Laura

14/07/2022

Why I Stopped Using a SIEM – and Why You Should Too

This post was written by Brad Freeman, SenseOn’s Director of Technology. Brad has over 15 years of experience conducting national cybersecurity investigations across critical national infrastructure and telecommunications sectors. He has led threat-hunting teams at corporations such as BT, managed Security Operations at EE, and performed incident response on offshore Oil & Gas platforms. Drawing upon his extensive experience and knowledge, including CISSP & CISM certifications, Brad leads the threat analytics team at SenseOn. He applies machine learning and artificial intelligence to automate the process of detecting and investigating cyber adversaries. He specialises in uncovering advanced actors within customer environments.

Today’s security operations centres (SOCs) face a massive challenge: how do you collect log data from different data sources and then aggregate and analyse it in a timely manner? Before attacks happen? In my experience, this is an incredibly difficult thing to do. 

Before becoming SenseOn’s co-founder and Director of Technology, I spent over a decade managing security teams at various organisations. Although I relished the challenge of providing frontline security across different verticals, these experiences were also connected by a common negative thread. 

No matter how much we invested in defence, there seemed to be an immense gap between what we wanted our security solutions to do and what they actually did. In no instance was this more true than how we used security information and event management (SIEM) systems. 

The Struggle to Find Business Value

Working as a security operations manager for an international energy company might be what first made me acutely aware of the downsides of SIEM tools. 

With a lot to lose from even a minor cyber attack, my organisation was keen to invest in cybersecurity. And, after a multi-year, several million-pound project to install a SIEM solution purchased from a well-established SIEM provider, my ten-person security team appeared to be well-equipped to deal with advanced threats — at least at first glance. 

Learn more: What you need to know about SIEM pricing.

Thanks to SIEM, my team would receive real-time event logs from tens of thousands of servers across our entire IT estate. Essentially, whenever anything remotely suspicious happened, we would get a notification about it. This would allow us to quickly investigate potential threats and security incidents. 

Unfortunately, while this might have sounded good in theory, receiving thousands of logs every day created a critical problem — how do you find business value among what turned out to be an immense amount of noise? As a result, the first thing we had to figure out was how to manage the overwhelming volume of security data that our SIEM solution sent us without compromising security. 

We spent countless hours trying to solve this problem. Working closely with our security engineering team, I spent most of my days writing “playbooks” on how to configure technical controls in a way that gave us value. 

To try and offload some of the stress SIEM was putting on experienced analysts, we created flow charts that would tell junior members of our security analytics team precisely what to do in case of a particular alert or sequence of alerts. The hope here was that senior analysts would then be freed up to do threat intelligence. 

Tied into our SIEM solution, which categorised ten types of alerts, this security framework seemed like a sensible way of mitigating the stress that came from having to deal with a barrage of alerts on a daily basis. 

Unfortunately, trying to extract value from SIEM software only proved to feed into an increasingly circular problem.

Read more: Why there’s no such thing as a low-cost SIEM

Real-World Headaches

We soon discovered that a SIEM is utterly unsuitable for real-world security environments, even with well-thought-out security orchestration and planning.

For example, to detect brute-force attacks, I configured our SIEM to alert us every time a user failed to enter a correct password more than a certain number of times each minute. Unfortunately, after doing so, my team soon received an extremely high number of alerts related to failed password entries. Our SIEM seemed to indicate that we were suffering from regular brute-force attacks but gave us no context. We also couldn’t find any other evidence of compromise. 

This meant that each time we received an alert, a member of my team had to take valuable time away from investigating other threat alerts to ask staff members questions like, “When did you last change your password?” Meanwhile, our SIEM would send us more alerts. As the backlog of tickets mounted, our capacity to investigate threats further decreased.

Read more: How to supercharge Microsoft Sentinel SIEM

After a prolonged investigation, we discovered that certain programs like Skype would keep a user’s old password in the background. Without users themselves doing anything, these programs would re-try old passwords every couple of milliseconds, resulting in hundreds of failed login attempts every time an employee changed their password. However, by the time we figured this out, our SIEM had encouraged us to waste a considerable amount of time searching for compromises it had invented. 

A similar situation would happen whenever an employee left the business. For whatever reason, their devices were often still plugged into our network after their departure. This resulted in our network constantly receiving connections from ex-employees trying to authenticate with the network. Although this was likely because individuals had hardcoded their credentials or logged into a server and kept the session alive, our SIEM gave us no insight into why this was happening. All it did was send us siloed alerts, forcing us to investigate non-existent problems.

Read more: 4 SIEM augmentation tools and why you need them

To try and stem this flow of false positive alarms, of which the above are only two examples that come to mind, I spent much of my time going back and forth with our security engineering department. We would repeatedly try to figure out where these alerts came from and engineer them away —  only for more alerts to pop up in their place. I lost count of how many times I said, “This use case is bad. Make changes here.” 

Struggling to Join the-Dots

As well as creating false positives and causing 46% of operational downtime, the fundamental flaw with SIEM systems is that whatever data they provide is completely siloed. They don’t give IT teams enough context to quickly get to the root cause of security alerts or queries. 

When the National Cyber Security Center (NCSC) published a report saying that sponsored state threat actors were using specific IP addresses to attack an organisation I was working with, this became an urgent issue for my team and me.

To figure out whether my organisation was being targeted, I entered a selection of the known threat actor IP addresses provided by the NCSC into my company’s SIEM. Shockingly, my SIEM solution reported that it found a thousand web requests to one of those IP addresses in the course of a day — all from a single device. 

The high frequency of connections immediately made me think I had uncovered command and control traffic (C2). Naturally, with patterns that match C2 going to what the NCSC has said is a nation-state threat actor, I panicked. 

Along with my best security analyst, we rushed to investigate the device where the traffic was coming from. However, we didn’t find any evidence that the computer was compromised. Of course, because advanced threat actors can move laterally without leaving traces, we still had no idea whether or not the device had been previously compromised.

Learn more: Reduce risk by adopting a compromised mindset.

Eventually, after days of focused, high-pressure investigation, we discovered that because the IP address that the NCSC highlighted was on a shared IP range, it shared its IP with over a thousand websites. One of our employees had used one of these sites to refresh sports results constantly. Thus, the siloed information that SIEM provided indicated an employee’s sports obsession, not C2. 

Going Beyond SIEM Limitations

The above examples are only a few of the times I can think of when out-of-context SIEM alerts led my team and me down a garden path —  taking us away from our actual jobs, creating vast amounts of stress, and, ultimately, hurting real security. 

This happened to me;  unfortunately, it still happens to security teams daily. The reason why is that a SIEM tool works like an email inbox. Alerts come through, and it’s up to you and your team to deal with them individually. 

I used to describe SIEM as “Excel on steroids” because, at the end of the day, although SIEM can give you endless logs, you still have to write your own rules. Out of the box, SIEM doesn’t do much to spot cyber threats and prevent security breaches from happening. 

We built SenseOn to succeed exactly where SIEM fails. Our self-driving cyber defence platform goes beyond the limitations of SIEM, improving visibility and security while lightening the load for security teams. By consolidating security controls like endpoint detection and response and SIEM, SenseOn can cluster disparate security observations to find real compromises. A single pane of glass also gives security teams a rich picture of alerts and vulnerabilities. 

As a tool for security monitoring, threat detection, and investigation, SenseOn shows analysts precisely where alerts are coming from at the user and application levels. This means that situations like those I described above never happen. 

Unlike SIEM tools, SenseOn doesn’t wake you at 3 AM for false alarms or send you hundreds of meaningless alerts each day. What SenseOn does is secure your entire IT estate, from endpoints to cloud servers, automating routine investigation while providing your analysts with data-rich insights into real threats. Through automation, SenseOn can also take immediate action on human analysts’ behalf, isolating devices and stopping the spread of time-sensitive malware like ransomware. 

Try a demo of SenseOn today.