AdvancedSecurity Operations

Security Monitoring: Building Detection Capabilities

Learn how to build effective security monitoring capabilities including alert triage, monitoring strategy, tool selection, and creating a detection roadmap for security operations.

#Security Monitoring#Detection#SOC#Alerting#Security Operations

The 76 Days That No One Was Watching

In 2014, attackers breached Sony Pictures and lurked undetected for 76 days. They exfiltrated 100 terabytes of data — emails, financial documents, unreleased films — and wiped thousands of workstations. The security monitoring systems were deployed but swimming in a sea of noise: thousands of alerts per day, most ignored, many from systems that hadn't been tuned in years. Effective monitoring provides visibility across the environment, enabling teams to detect threats before they cause damage.

Security monitoring is the continuous observation of systems, networks, and applications to detect security threats in real time. Effective monitoring provides visibility into what is happening across the environment.

Prerequisites

**SIEM Fundamentals** — Understanding of log management and correlation

**Log Analysis** — Knowledge of log sources and formats

**Networking Basics** — Network protocols and traffic patterns

Monitoring Strategy

Define Monitoring Objectives

Start with business and security objectives:

**Compliance** — Meet regulatory requirements (PCI DSS, HIPAA, SOX, GDPR)

**Threat Detection** — Detect known attack patterns and anomalies

**Incident Response** — Provide data for investigation and evidence

**Security Posture** — Measure control effectiveness over time

Identify Crown Jewels

Not all assets are equal. Focus monitoring on:

Domain Controllers

Database servers (customer data, financial records)

File servers with sensitive data

Email systems

Cloud management consoles

Code repositories (CI/CD pipelines)

Map Threats to Data Sources

For each threat scenario, identify the data sources needed for detection:

| Threat Scenario | Required Data Sources |

|----------------|---------------------|

| Ransomware deployment | Process logs, file system monitoring, EDR |

| Lateral movement | Logon events, network connections, service creation |

| Credential theft | LSASS access events, authentication logs |

| Data exfiltration | Network flows, DNS logs, proxy logs |

| Privilege escalation | Process logs, group membership changes, token manipulation |

Building a Detection Stack

Layer 1: Log Collection

Every system must generate security-relevant logs:

Windows: Security, System, PowerShell, Sysmon

Linux: auth.log, syslog, auditd

Firewalls: Connection logs, threat prevention logs

Endpoint: EDR telemetry, process creation, file changes

Layer 2: Aggregation and Normalization

SIEM or log management platform that:

Collects logs from all sources

Normalizes disparate formats into a common schema

Provides search and query capabilities

Retains data for compliance and forensics

Layer 3: Detection and Alerting

Correlation rules and detection logic:

Signature-based detection (known IOC patterns)

Behavioral detection (deviation from baseline)

Anomaly detection (statistical outliers)

Threat intelligence matching

Layer 4: Response and Investigation

SOAR (Security Orchestration, Automation, and Response) for automated actions, case management for tracking investigations, and playbooks for standardized response procedures.

Alert Triage

The SOC tier model structures alert triage:

Tier 1 (Triage): Review alerts, validate true/false positive, escalate confirmed incidents. Goal: 5-10 minutes per alert.

Tier 2 (Investigation): Deep investigation of escalated alerts, containment actions, evidence collection. Goal: 30-60 minutes per incident.

Tier 3 (Advanced): Complex investigations, reverse engineering, advanced forensics, threat hunting. Available for major incidents.

Triage Decision Process

Alert Received → Validate Alert → Is it a true positive?
  ├── Yes → Assess severity → Escalate to appropriate tier
  ├── No → Document false positive → Tune rule if needed
  └── Uncertain → Gather additional data → Reassess

Detection Engineering

Creating Detection Rules

# Splunk detection for suspicious PowerShell
index=windows EventCode=4104
| search ScriptBlockText="*DownloadString*" OR ScriptBlockText="*Invoke-Expression*" OR ScriptBlockText="*-enc*"
| table _time, host, UserName, ScriptBlockText
| eval risk_score = 
    case(match(ScriptBlockText, "DownloadString"), "High",
         match(ScriptBlockText, "Invoke-Expression"), "High",
         match(ScriptBlockText, "-enc"), "Critical",
         "Medium")

Rule Lifecycle

**Research**: Identify a detection opportunity from threat intelligence

**Develop**: Write and test the detection rule

**Validate**: Test against known true positives and false positives

**Deploy**: Push to production in a monitoring-only mode (no alerting)

**Tune**: Adjust based on observed results

**Activate**: Enable alerting after acceptable noise level

**Review**: Periodically review for effectiveness and relevance

Monitoring Coverage Assessment

Regularly assess monitoring coverage using frameworks like MITRE ATT&CK:

Technique: T1055 (Process Injection)
├── Detection 1: Sysmon Event ID 8 (CreateRemoteThread) → Covered
├── Detection 2: API monitoring → Covered
└── Detection 3: Memory scanning → Not covered (gap)

Real-World Example: Building a SOC

Scenario: A growing company builds its first SOC.

**Foundation**: Deploy SIEM with 50 critical log sources (firewalls, Windows DCs, cloud APIs)

**Initial Rules**: 50 out-of-the-box correlation rules for common threats

**Tuning**: First month focuses on reducing false positives — 40% of rules modified

**Playbooks**: Develop 10 incident response playbooks for common scenarios

**Staffing**: 3 analysts (Tier 1) on rotating shifts, 2 engineers (Tier 2), 1 lead (Tier 3)

**Coverage Assessment**: Map detection rules to MITRE ATT&CK — identify 30% coverage gap

**Expansion**: Add 20 custom rules targeting gap areas over next 3 months

**Metrics**: Mean time to detect (MTTD) reduced from 14 days to 2 hours

Common Mistakes

**Alerting on everything** — Overwhelms analysts; focus on high-fidelity alerts

**No false positive management** — Analysts ignore noisy alerts, missing real incidents

**Insufficient context in alerts** — Alerts without enough data for triage waste time

**No playbooks** — Inconsistent response; analysts figure out procedures individually

**Not measuring effectiveness** — Cannot improve without metrics (MTTD, MTTR)

**Ignoring visibility gaps** — Assuming all attacks are detected without coverage validation

Best Practices

**Start with high-value assets** — Protect crown jewels first

**Focus on quality over quantity** — 50 good rules beat 500 noisy ones

**Automate tier-1 triage** — SOAR handles repetitive low-confidence alerts

**Test detection coverage** — Use purple team exercises to validate

**Build playbooks** — Standardize response for common scenarios

**Measure and iterate** — Track MTTD, MTTR, false positive rate

**Tune continuously** — Rules drift over time as environment changes

**Use threat intelligence** — Prioritize monitoring based on current threats

Related Tools

**Splunk** — SIEM and monitoring platform

**TheHive** — Incident case management

**Shuffle** — Open-source SOAR

**Wazuh** — Open-source security monitoring

**ElastAlert** — Alerting for ELK Stack

**MITRE ATT&CK** — Detection coverage framework

SIEM Fundamentals: Security Information and Event Management

Detection Engineering: Creating Security Alerts and Rules

Incident Response: Structured Approach to Security Breaches

Threat Hunting: Proactive Cyber Defense Strategies

Log Analysis: Extracting Intelligence from System Logs

Summary

Security monitoring requires a strategic approach combining log collection, SIEM aggregation, detection rules, and alert triage. Focus on high-value assets, start with high-fidelity rules, build playbooks for consistent response, and continuously test and tune detection coverage. Measuring MTTD and MTTR tracks improvement over time.

Knowledge Check

What are the four layers of a detection stack?

What is the SOC tier model and how does triage work?

Why is alert quality more important than alert quantity?

How does the MITRE ATT&CK framework help assess monitoring coverage?

What metrics should a SOC track for effectiveness?

Frequently Asked Questions

What are the four layers of a detection stack?

The four layers are: (1) Log Collection from endpoints, firewalls, and cloud services; (2) Aggregation and Normalization in a SIEM; (3) Detection and Alerting via correlation rules; and (4) Response and Investigation using SOAR and case management tools.

What is the SOC tier model and how does triage work?

Tier 1 analysts review and validate alerts (5-10 minutes per alert), Tier 2 performs deep investigation and containment (30-60 minutes), and Tier 3 handles complex incidents, forensics, and threat hunting. Alerts flow from triage to investigation based on severity and complexity.

Why is alert quality more important than quantity?

High-quality, low-noise alerts prevent analyst fatigue and ensure real threats are not missed. Fifty well-tuned rules that produce high-fidelity alerts are far more effective than 500 noisy rules that overwhelm analysts and cause them to ignore alerting entirely.

How does MITRE ATT&CK help assess monitoring coverage?

MITRE ATT&CK provides a comprehensive matrix of adversary techniques. By mapping detection rules to ATT&CK techniques, organizations identify coverage gaps where no detection exists, prioritize new rule development, and validate that security controls detect the threats most relevant to their industry.

What is MTTD and MTTR and why do they matter?

Mean Time to Detect (MTTD) measures how quickly threats are identified; Mean Time to Respond (MTTR) measures how quickly they are contained. Both are key SOC performance metrics — mature organizations reduce MTTD from days to hours and MTTR from hours to minutes.

What are the common mistakes in security monitoring?

Common mistakes include alerting on everything (causing fatigue), not managing false positives, providing insufficient context in alerts, lacking response playbooks, not measuring effectiveness, and ignoring visibility gaps in monitoring coverage.

What are crown jewels in security monitoring?

Crown jewels are the highest-value assets requiring priority monitoring: domain controllers, database servers with sensitive data, file servers, email systems, cloud management consoles, and CI/CD pipelines. Protecting these assets first maximizes security impact with limited resources.

What is the difference between signature-based and behavioral detection?

Signature-based detection matches known attack patterns (IOCs, YARA rules) and catches known threats. Behavioral detection identifies deviations from normal baselines, catching novel and zero-day attacks. Effective monitoring uses both approaches together.

How do you build effective incident response playbooks?

Playbooks standardize response for common scenarios (ransomware, phishing, data exfiltration). Include step-by-step procedures, roles and responsibilities, escalation paths, communication templates, and evidence collection checklists. Test and update playbooks quarterly through tabletop exercises.

What is SOAR and how does it improve security monitoring?

SOAR (Security Orchestration, Automation, and Response) automates repetitive Tier 1 tasks like alert enrichment, IP reputation checks, and containment actions. It reduces analyst workload by 60-80% for low-confidence alerts, allowing focus on high-priority investigations.