SIEM Fundamentals: Security Information and Event Management
A comprehensive introduction to SIEM systems covering architecture, log collection, correlation rules, deployment considerations, and how SIEM powers modern security operations centers.
The Target Breach That Nobody Noticed for Weeks
In 2013, attackers stole 40 million credit card numbers from Target — one of the largest retail breaches in history. Security teams had all the data: the SIEM alerts fired, the firewall logs showed data exfiltration, and endpoint telemetry captured the malware. But no one was watching. The SIEM generated thousands of alerts daily, and the critical signals drowned in the noise. This breach became the catalyst for modern SIEM deployments — centralized log correlation that actually gets investigated.
Security Information and Event Management (SIEM) is a technology that provides real-time analysis of security alerts generated by network hardware and applications. SIEM systems collect log data from multiple sources, normalize it, correlate events, and generate alerts for security teams.
Prerequisites
SIEM Architecture
Core Components
Log Collection Agents: Installed on endpoints to collect and forward logs. Agents can be installed on Windows (WinEventLog), Linux (syslog), and various appliances.
Log Aggregation Layer: Centralized servers that receive logs from all sources. This layer handles normalization — converting different log formats into a common schema.
Correlation Engine: The analytical core that processes normalized logs against rules to detect threats. Rules can be simple (single event match) or complex (multi-event sequences with time windows).
Storage Layer: Retains log data for compliance (often 1 year+) and forensic investigation. Must balance storage costs against retention requirements.
Dashboard and Alerting: Visualizes data, alerts analysts, and provides investigation interfaces.
Common SIEM Platforms
Log Collection and Normalization
Common Log Sources
Windows Event Logs — Security, System, Application (Event IDs)
Linux Syslog — auth.log, syslog, kern.log
Firewall Logs — Connection allow/deny, NAT translations
Web Server Logs — Apache/Nginx access and error logs
DNS Logs — Query and response logs
Cloud Logs — AWS CloudTrail, Azure Monitor, GCP Audit Logs
Endpoint Logs — EDR telemetry, process creation, file modifications
Log Forwarding
# Linux rsyslog to SIEM
# /etc/rsyslog.conf
*.* @siem-server:514
*.* @@siem-server:1514 # TCP with TLS
# Windows Event Forwarding (WEF)
wevtutil set-log "ForwardedEvents" /enabled:true
# Using Winlogbeat for Elastic
winlogbeat -e -c winlogbeat.yml
Correlation Rules
Rule Types
Atomic Rules: Single event matching a condition — "User logged in from a blocked country."
Threshold Rules: Multiple events within a timeframe — "10 failed logins in 5 minutes (brute force)."
Sequence Rules: Events occurring in a specific order — "User created → added to Domain Admins → logged in from new location."
Correlation Rules: Combining events from different sources — "Windows event ID 4688 (process creation) for cmd.exe preceded by suspicious network connection to known bad IP."
Rule Example
# Example correlation rule pseudocode
if (
event.source == "Windows Security Log" and
event.event_id == 4624 and # Successful logon
event.logon_type == 3 and # Network logon
event.account.is_privileged and
event.source_ip not in CORPORATE_RANGES and
count_events(event.account, timeframe="1h") > 5
):
alert("HIGH", "Possible lateral movement from suspicious IP")
SIEM Deployment Considerations
On-Premises vs Cloud
| Factor | On-Premises | Cloud (SIEM-as-a-Service) |
|--------|-------------|---------------------------|
| Upfront Cost | High (hardware, licensing) | Operational expense |
| Scalability | Limited by hardware | Elastic |
| Maintenance | Full responsibility | Vendor managed |
| Data Residency | Full control | Depends on provider |
| Integration | Manual | Built-in connectors |
Log Source Coverage
Effective SIEM requires collecting from all relevant sources. Common gaps include cloud environments, container orchestration (Kubernetes), IoT devices, and shadow IT.
Storage Planning
A typical organization generates 1-10 TB of logs per day. Storage must balance retention requirements (PCI DSS: 1 year, HIPAA: 6 years, SOX: 7 years) against cost. Hot storage (fast query, 30-90 days) and cold storage (cheap, long-term archive) strategies are standard.
Real-World Example: SIEM Implementation
Scenario: A mid-size company implements Splunk for SOC operations.
Common Mistakes
Best Practices
Related Tools
Related Articles
Summary
SIEM systems collect, normalize, and correlate security events from across the enterprise. Core components include log collection, aggregation, correlation engines, storage, and alerting. Effective SIEM requires comprehensive log coverage, well-tuned correlation rules, adequate storage, and integration with SOC workflows. Common pitfalls include alert fatigue, poor tuning, and blind spots in log coverage.