AdvancedSecurity Operations
Log Analysis: Extracting Intelligence from System Logs
Master log analysis techniques for security operations covering Windows Event Logs, Syslog, Apache logs, DNS logs, and cloud service logs with practical querying and correlation examples.
#Log Analysis#Windows Event Logs#Syslog#Security Monitoring#Forensics
Why Log Analysis Matters
Logs record every significant event in an IT environment. System logs track user authentication, process execution, network connections, file access, and configuration changes. Security professionals analyze logs to detect attacks, investigate incidents, troubleshoot issues, maintain compliance, and understand normal behavior to detect anomalies.
Prerequisites
Windows Event Logs
Key Event Logs
Security Log: Records authentication events, privilege use, and object access.
System Log: Records system events from services, drivers, and hardware.
Application Log: Records events from applications.
PowerShell Log: Script block logging (Event ID 4104) when enabled.
Analyzing Windows Logs
# Query last 10 security logon events
Get-WinEvent -LogName Security -MaxEvents 10 | Where-Object { $_.Id -eq 4624 }
# Find failed logon attempts in last 24 hours
Get-WinEvent -LogName Security -FilterXPath "*[System[EventID=4625]]" |
Where-Object { $_.TimeCreated -gt (Get-Date).AddDays(-1) } |
Group-Object { $_.Properties[5].Value } |
Sort-Object Count -Descending
# Export logs for analysis
wevtutil epl Security security_export.evtx
Key Event Fields
Each Windows event contains:
Linux Syslog
Key Log Files
/var/log/auth.log - Authentication events (Debian/Ubuntu)
/var/log/secure - Authentication events (RHEL/CentOS)
/var/log/syslog - General system messages
/var/log/kern.log - Kernel messages
/var/log/apache2/ - Apache web server logs
/var/log/nginx/ - Nginx web server logs
Analyzing Linux Logs
# Find failed SSH login attempts
grep "Failed password" /var/log/auth.log | awk '{print $1, $2, $11}' | sort | uniq -c | sort -rn
# Find successful SSH logins
grep "Accepted password" /var/log/auth.log | awk '{print $1, $2, $9, $11}'
# Monitor logs in real-time
tail -f /var/log/auth.log
# Use journalctl (systemd systems)
journalctl -u ssh.service --since "24 hours ago"
journalctl _COMM=sshd --output=json
Web Server Logs
Apache/Nginx Common Log Format
192.168.1.1 - - [01/Jun/2026:13:45:12 +0000] "GET /admin/login.php HTTP/1.1" 200 2345
Fields: IP address, identity, user, timestamp, request, status code, bytes sent.
Web Log Analysis
# Find most requested URLs
awk '{print $7}' /var/log/apache2/access.log | sort | uniq -c | sort -rn | head -20
# Find 404 errors (potential scanning)
awk '$9 == 404 {print $7}' /var/log/apache2/access.log | sort | uniq -c | sort -rn
# Find SQL injection attempts
grep -i "select|union|insert|--|'" /var/log/apache2/access.log
# Analyze response time anomalies
awk '{print $NF, $7}' /var/log/apache2/access.log | sort -rn | head -10
# Find brute force attempts on login pages
grep "POST /wp-login" /var/log/apache2/access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -10
DNS Log Analysis
DNS logs provide visibility into the domains clients are resolving. DNS is often used for command and control (C2), data exfiltration, and tunneling.
# Find top DNS queries
grep "query:" /var/log/named.log | awk '{print $(NF-1)}' | sort | uniq -c | sort -rn | head -20
# Find unusually long domain names (potential tunneling)
awk 'length($NF) > 50 {print $NF}' /var/log/named.log | sort -u
Suspicious DNS Patterns:
Cloud Service Logs
AWS CloudTrail
# Find IAM user creation
aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=CreateUser
# Find security group changes
aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=AuthorizeSecurityGroupIngress
# Query with jq
aws cloudtrail lookup-events --query "Events[?EventName=='ConsoleLogin'].{Time:EventTime,User:Username,IP:SourceIPAddress}" --output json
Azure Monitor
# Sign-in logs
SigninLogs
| where TimeGenerated > ago(7d)
| where ResultType != 0
| summarize FailedAttempts = count() by UserPrincipalName, AppDisplayName
| top 10 by FailedAttempts desc
Correlation Techniques
Time-Based Correlation
Align events from multiple sources by timestamp to understand attack chains:
14:01:01 — Firewall: Blocked connection from 1.2.3.4 to mail server
14:01:02 — Web server: POST to /owa/auth/login.aspx from 1.2.3.4
14:01:05 — Windows: Failed logon for admin@corp.local from 1.2.3.4
14:01:10 — Windows: Successful logon for admin@corp.local from 1.2.3.4
14:01:15 — EDR: PowerShell execution on mail server
→ Pattern suggests successful brute force attack on OWA
Session-Based Correlation
Group events by session ID or connection tuple (src_ip:port → dst_ip:port).
Real-World Example: Investigating an Incident with Logs
Scenario: Investigating a suspected data exfiltration.
Common Mistakes
Best Practices
Related Tools
Related Articles
Summary
Log analysis is the foundation of security operations. Windows Event Logs, Linux Syslog, web server logs, DNS logs, and cloud logs each provide unique visibility. Effective analysis requires understanding key event IDs, log formats, correlation techniques, and using the right tools (grep, awk, PowerShell, SIEM). Time synchronization, comprehensive collection, and baseline understanding are critical for effective log analysis.