Security Information and Event Management – SIEM – is the ability to collate and analyse events from across the network in real time in order to detect security incidents and attacks.

Many organisations first encounter SIEM when it becomes a compliance exercise in order meet the requirements of a security framework such as PCI-DSS or ISO 27001. SIEM is not a new idea – the core capabilities have been around since the mid 2000s.

Once it is fully up and running, a SIEM system provides security analysts and support engineers with a very useful means of seeing everything that is going on in the network ‘right now’ or at a chosen point in history.

All software and network devices produce event and log files and telemetry which describe what is happening to various levels of detail.  This varies from recording a successful login to Active Directory on a Windows PC through to noting that a DBA has dropped a table in an Oracle database or a router has been rebooted.

Because the technical disciplines that run our networks tend to be provided by different people and teams (the network engineers configure the firewalls, the DBA look after the SQL databases and the Systems Administrators support Windows Server configurations and so on)  the logs and support telemetry from all of those systems is similarly stored and analysed in those separate silos.  When considered from an engineering perspective, this segregation of log information makes a lot of sense; one of the main problems with reviewing system logs is that there are so many events produced in the normal operation of most systems it becomes very difficult to see the important messages hidden within all the noise.

However, when looking at the problem from a security perspective, the segregation becomes a problem.  By its very nature, security is a holistic exercises not a siloed endeavour.

A consistent finding in post mortem after security breaches is that the evidence of the breach was visible in the logs at the time of the attack but no-one spotted the evidence or realised the importance and reacted accordingly.  The Verizon Data Breach Investigation report of 2008 states that “In 82 percent of cases … the victim possessed the ability to discover the breach had they been more diligent in monitoring and analyzing event-related information available to them at the time of the incident.”

Consider a situation where an attacker is attempting to access an RDP server which is open to the internet. The attacker is using credential stuffing to attempt the breach and throttles the number of attempts per second to prevent the firewall from blocking the attack due to a sudden spike in traffic, however each connection attempt is logged by the firewall into its logs.  At the same time, the Windows Server hosting the RDP service logs each failed login attempt reporting event 4625 into the windows event logs.  When viewed in isolation in either the firewall logs or the windows logs, it is not obvious that anything is happening. However when the two event streams are viewed together and the failed Windows login attempts are correlated with the consistent stream of connection attempts seen in the firewall logs from the same IP address – then an alarm can be raised.  This is what SIEM systems do.  They correlate the activity from multiple systems so that you can see what is going on at the firewall, and the windows server, and the database and the storage arrays all at the same time – and seeing things going on at the same time, rather than in isolation, gives a very different perspective and allows attacks to be spotted that may otherwise have been missed.

The integrated event stream is also invaluable after the fact, when reviewing either an attack or some kind of system malfunction.  To realise that the performance problem reported by the application users happens at the same time as the Oracle database reports it is locking tables at the same time as the Windows server logs reports it has kicked off a scheduled backup brings an insight into the root cause of the problem.  When investigating problems in complex systems, context is king – and the ability to know what was going on at the same time in many different components of the system is invaluable to support staff.

For the correlation to work, it is essential that each device on the network knows what the time is – so an event that is timestamped as happening at 09:05:04 in the firewall logs happened at the same moment as an event showing at 09:05:04 in the server logs and not two seconds earlier or later.  Network devices and computers generally do not have a very reliable internal clock and so a reliable independent time source is needed.  This is recognised in PCI-DSS requirement 10.4, for example, which states that the time of critical systems must be synchronised and kept in alignment with technology such as NTP.

Not only do the different event logs have to agree what the time is so that it is easy to see when things happen in relation to each other across the network, the event logs and audit trails also have to be secured against change so that an attacker cannot alter the logs in order to hide the evidence of their activity in the network.

For this reason, SIEM systems typically are arranged to store copies of event logs from each device and system in a separate secure location – often on a security appliance within the data centre.  Event logs on computers and devices will have a finite amount of available storage space and so will automatically overwrite the oldest log files as needed.  By copying the logs to a central repository, the logs can be preserved and made available for forensic analysis any time after the attack occurred.

The challenge with the central log repository available to the SIEM system is the sheer volume of information and the risk of large numbers of false positive or negative alerts overwhelming the operations teams.  Modern SIEM systems have mature rules engines and correlation abilities and offer initial configurations that have rulesets ready to be used.  However, a period of ‘training’ is still needed in order to tune the system to the unique environment present in every network.  Much like tuning an Intrusion Detection System – the initial rules and alerts will need to fine tuned when the SIEM system is first installed.

SIEM provides a means for both security operations and support staff to take a holistic view of current and historic activity across the network – spot intrusion, attacks and system problems as well as conduct post mortem and forensic investigations after the fact.  The most import element though, is not the SIEM system’s ability to spot a problem and raise an alert but the need for a competent human to see the alert and respond in a timely and appropriate manner.