Automated detection systems are essential, but they are not infallible. Even the most sophisticated security platforms have blind spots: novel attack techniques, living-off-the-land tactics, and slow-and-low adversaries that deliberately avoid triggering detection thresholds. Threat hunting fills this gap by applying human intuition, creativity, and domain expertise to proactively search for threats that automated systems miss.
This guide provides a practical framework for building and operating a threat hunting programme, covering methodology, data requirements, tooling, playbook development, and programme metrics.
What Is Threat Hunting?
Threat hunting is the proactive, hypothesis-driven search for threats within an organisation's environment that have evaded existing detection mechanisms. Unlike incident response (which reacts to known alerts) or vulnerability management (which addresses potential weaknesses), threat hunting assumes that an adversary may already be present and seeks to find evidence of their activity.
The key characteristics that distinguish threat hunting from other security functions:
Proactive: Threat hunting is initiated by the hunter, not triggered by an alert. Hunters search for threats that have not been detected, not threats that have already raised alarms.
Hypothesis-driven: Effective threat hunting begins with a hypothesis ("If an adversary were present in our environment, what evidence would they leave?") and systematically tests that hypothesis against available data.
Iterative: Threat hunts refine their hypotheses based on findings. Initial results lead to follow-up questions, which lead to deeper investigation. A hunt that finds no threats but identifies telemetry gaps or detection blind spots is still valuable.
Human-led: Whilst hunters use tools and automation to process data, the analytical process (developing hypotheses, interpreting results, recognising patterns) requires human expertise.
Reactive vs Proactive Hunting
Threat hunting exists on a maturity spectrum from reactive to proactive.
Reactive Hunting (Intelligence-Driven)
Reactive hunting uses external intelligence (threat reports, indicators of compromise (IOCs), vulnerability disclosures) to search for specific threats. When a new threat report describes a campaign targeting your industry, reactive hunting searches your environment for the indicators and techniques described in the report.
Reactive hunting is valuable and accessible. It requires less analytical expertise than proactive hunting and produces clear, measurable results. However, it is limited by the intelligence available: you can only hunt for threats that someone else has already discovered and documented.
Proactive Hunting (Hypothesis-Driven)
Proactive hunting develops original hypotheses based on knowledge of the environment, threat actor behaviour, and attack techniques. Rather than searching for specific indicators, proactive hunters look for behavioural patterns and anomalies that could indicate adversary presence.
Proactive hunting is more demanding but more rewarding. It can discover novel threats, zero-day exploitation, and adversaries who deliberately avoid known indicators. It also generates intelligence that feeds back into detection engineering, improving automated detection capabilities over time.
Developing Hunt Hypotheses
The quality of a threat hunt depends on the quality of its hypothesis. Effective hypotheses are specific, testable, and grounded in realistic threat scenarios.
Sources of Hypotheses
Threat intelligence: Reports of campaigns targeting your industry, sector, or geography. Focus on the tactics, techniques, and procedures (TTPs) described in these reports rather than specific indicators, which are ephemeral.
MITRE ATT&CK framework: The framework provides a structured catalogue of adversary techniques. Hunters can systematically work through relevant techniques, developing hypotheses for each: "If an adversary used T1053 (Scheduled Tasks) for persistence, what evidence would we expect to see in our environment?"
Environmental knowledge: Understanding of your organisation's specific infrastructure, crown jewels, and attack surface. What are the most likely targets? What are the most probable attack paths? Where are the gaps in your detection coverage?
Anomaly investigation: Statistical anomalies identified through data analysis (unusual authentication patterns, unexpected network connections, atypical process execution) can seed hypotheses worth investigating.
Hypothesis Structure
A well-formed hunt hypothesis includes:
- The threat scenario: What adversary activity are we looking for?
- Expected evidence: What data artefacts would this activity produce?
- Data sources required: Where would we find this evidence?
- Analysis approach: How will we search for and evaluate the evidence?
- Success criteria: What constitutes a positive finding versus a false lead?
Example hypothesis: "If an adversary has compromised a privileged Active Directory account and is conducting reconnaissance, we would expect to see LDAP queries from unusual source hosts, targeting sensitive AD objects (AdminSDHolder, domain controller computer objects, Group Policy objects) outside of normal IT administration windows."
Essential Data Sources
Threat hunting requires thorough, high-fidelity data. The effectiveness of any hunt is constrained by the available telemetry.
Endpoint Data
Endpoint telemetry is the foundation of most threat hunts. Essential data includes process execution events with full command-line arguments, parent-child process relationships, file creation, modification, and deletion events, registry modifications, network connections initiated by processes, module loading (particularly DLLs loaded by legitimate processes), and user session events.
Network Data
Network telemetry provides visibility into communications that endpoint agents may not capture. Key sources include DNS query logs with full domain names and response codes, NetFlow or IPFIX data showing connection metadata, proxy and firewall logs with URL-level visibility, TLS certificate metadata for encrypted connections, and network detection system alerts.
Identity Data
Identity telemetry is critical for detecting credential abuse, privilege escalation, and lateral movement. Sources include Active Directory authentication logs (Event IDs 4624, 4625, 4648, 4768, 4769, 4776), Azure AD and cloud identity provider sign-in logs, VPN and remote access authentication events, privileged access events (sudo, runas, privilege escalation), and service account activity logs.
Hunt Playbooks
Hunt playbooks codify hunting procedures for specific threat scenarios, making hunts repeatable, teachable, and measurable. Below are three example playbooks covering common threat hunting objectives.
Playbook 1: Lateral Movement Detection
Hypothesis: An adversary with initial access is moving laterally through the environment using legitimate remote administration tools and protocols.
Data sources: Endpoint process execution logs, Windows authentication events, network connection metadata.
Hunt procedure:
- Identify all remote execution tool usage (PsExec, WMI, WinRM, RDP, SSH) across the environment
- Baseline normal patterns: which accounts, source hosts, and destination hosts are typical for each tool?
- Identify anomalies: unusual source-destination pairs, accounts using remote tools for the first time, usage outside normal business hours
- Investigate anomalies: correlate with authentication events to determine whether the access was preceded by legitimate login, whether MFA was used, and whether subsequent activity is consistent with the user's role
- Document findings and update detection rules if gaps are identified
Playbook 2: Persistence Mechanism Discovery
Hypothesis: An adversary has established persistence through mechanisms that our automated detection does not cover.
Data sources: Registry modification events, scheduled task creation, service installation, startup item changes, WMI subscription events.
Hunt procedure:
- Enumerate all persistence mechanisms created within the hunt timeframe (30-90 days)
- Baseline against known-good: approved software installations, IT-managed scheduled tasks, expected service configurations
- Investigate unknowns: persistence mechanisms that do not match approved software or IT operations
- Analyse binaries and scripts referenced by persistence mechanisms using static and dynamic analysis
- Check network connections for any suspicious persistence mechanisms to identify potential C2 communication
Playbook 3: Data Exfiltration Indicators
Hypothesis: An adversary is staging and exfiltrating sensitive data using legitimate cloud services or encrypted channels.
Data sources: Network metadata, proxy logs, DNS queries, endpoint file access events, cloud storage API logs.
Hunt procedure:
- Identify large or sustained outbound data transfers to cloud storage services (Dropbox, Google Drive, OneDrive, Mega, AWS S3)
- Baseline normal usage: which users and hosts regularly transfer data to these services, and what volumes are typical?
- Identify anomalies: unusual volumes, new users or hosts, transfers outside business hours, data transfers to personal accounts
- Investigate DNS-based exfiltration: look for high-frequency DNS queries to unusual domains, abnormally long subdomain strings, or elevated query volumes from individual hosts
- Analyse endpoint file access patterns preceding suspicious transfers: bulk file access, access to sensitive directories, or archive creation before upload
Tooling for Threat Hunting
Effective threat hunting requires tools that provide flexible data access, powerful search and query capabilities, and visualisation to support pattern recognition.
Query platforms: Tools that provide fast search across large volumes of security telemetry: SIEM query languages, Elastic/Kibana, Splunk SPL, or dedicated hunting platforms.
Notebook environments: Jupyter notebooks with Python libraries (pandas, NetworkX, matplotlib) enable custom analysis and visualisation that predefined dashboards cannot provide.
Unified platforms: SenseOn provides a particularly effective hunting environment because all telemetry (endpoint, network, cloud, and identity) is available through a single query interface. Hunters do not need to pivot between separate tools or manually correlate data from different systems. The cross-domain correlation engine's behavioural baselines also provide the anomaly context that hunters need: deviations from established baselines are surfaced automatically, giving hunters pre-filtered starting points for investigation.
Programme Metrics
Measuring threat hunting effectiveness ensures programme accountability and continuous improvement.
Hunt coverage: What percentage of relevant MITRE ATT&CK techniques have been hunted in the past quarter? This metric drives systematic coverage of the threat landscape.
Findings per hunt: How many confirmed threats, suspicious activities, or detection gaps does each hunt identify? This metric validates that hunts are productive.
Mean time to hunt (MTTH): How quickly can a new hunt be initiated after a relevant threat intelligence report is published? This measures programme agility.
Detection rule improvements: How many new or improved detection rules result from hunting findings? This measures the feedback loop between hunting and automated detection.
Dwell time reduction: Over time, does proactive hunting reduce the average time that threats persist undetected in the environment? This is the ultimate measure of hunting programme impact.
Building Your Programme
Start small. A single analyst dedicating one day per week to structured hunting, following documented playbooks against available telemetry, delivers more value than an ambitious programme that never launches.
Focus initial hunts on high-impact scenarios: lateral movement, persistence mechanisms, and data exfiltration. These techniques appear in nearly every intrusion and produce clear, actionable findings.
Invest in telemetry first. The most skilled hunter cannot find threats in data that does not exist. Ensure complete endpoint, network, and identity telemetry before investing in advanced hunting tools or techniques.
