⭐ Day 14 – Log Enrichment, Field Extraction & Metadata

Today students learn how to transform raw logs → structured data → security meaning in Splunk.

This is VERY important because:

• SIEM alerts depend on correct fields
• Threat hunting depends on good field extraction
• Dashboards depend on structured data
• MITRE mapping depends on knowing what each field means

🧠 1. What Is Log Enrichment? (Simple Explanation)

Log Enrichment = adding extra security context to raw logs.

Before enrichment:

IP=192.168.1.50

After enrichment:

IP=192.168.1.50
GeoLocation=India
Reputation=Malicious
User=tom
DeviceType=Windows10

This makes detection and hunting much easier.

🔥 2. Why Field Extraction Matters?

Field Extraction = splitting raw logs into readable fields.

Example raw log:

Failed password for root from 10.0.0.5 port 4588

Splunk extracts:

• user = root
• src_ip = 10.0.0.5
• port = 4588
• action = failed password

This allows queries like:

stats count by src_ip

🧱 3. Three Types of Fields Every SOC Analyst Must Know

A. Source

Where logs came from.
Examples:

• WinEventLog:Security
• /var/log/auth.log
• firewall logs

B. Sourcetype

What type of log it is.
Examples:

• WinEventLog:System
• linux_secure
• Sysmon

C. Fields

Actual extracted data:

• user
• IP
• port
• status
• command
• EventCode

Understanding these transforms raw logs into security detection.

🔍 4. Splunk Tools for Field Extraction

✔ Field Extractor (UI tool)

Go to:

Settings → Fields → Field Extractions

Upload sample logs → highlight → Splunk generates regex.

✔ Field Discovery

In search bar:

index=*

Then click on Fields (left side)
→ Shows auto-detected fields.

✔ Rex Command

Manual extraction:

| rex "Failed password for (?<user>\w+) from (?<src_ip>\d+\.\d+\.\d+\.\d+)"

This creates user and src_ip fields.

⚙️ 5. Sample Field Extraction for Linux Logs

Raw log:

Failed password for root from 10.0.0.5 port 4588

Splunk rex:

| rex "Failed password for (?<user>\w+) from (?<src_ip>\S+) port (?<port>\d+)"

Now you can run:

stats count by src_ip

⚙️ 6. Sample Field Extraction for Windows Logs

Windows logs already have fields, such as:

• Account_Name
• Workstation_Name
• Logon_Type
• IpAddress

But you can extract extra fields.

Example:

| rex "Account Name:\s+(?<account>\S+)"

🧩 7. Log Enrichment Techniques

✔ Add GeoIP info

| iplocation IpAddress

Adds:

• City
• Country
• Region
• Latitude/Longitude

✔ Add threat intel reputation

If you have threat feeds:

| lookup bad_ips ip OUTPUT reputation

✔ Add device type

| lookup device_inventory host OUTPUT device_type

Useful for:

• identifying servers
• identifying critical systems

🛠 8. Hands-On Lab (Day 14)

Task 1 – Extract Linux fields

index=* "Failed password"
| rex "Failed password for (?<user>\w+) from (?<ip>\S+)"
| stats count by user, ip

Task 2 – Extract Windows fields manually

index=* EventCode=4624
| rex "Logon Type:\s+(?<logtype>\d+)"
| stats count by logtype

Task 3 – Add GeoIP enrichment

index=* IpAddress=* 
| iplocation IpAddress
| table IpAddress Country City

Task 4 – Create enriched dashboard panel

Panel shows:

• IP
• Country
• City
• Count

index=* IpAddress=*
| iplocation IpAddress
| stats count by IpAddress, Country, City

Task 5 – Extract suspicious PowerShell commands

index=* powershell "EncodedCommand"
| rex "CommandLine:\s+(?<cmd>.+)"
| table _time user cmd

🎤 Trainer Script

Say this:

“Field extraction converts raw logs into intelligence.
Enrichment adds context — country, device type, risk.
A SOC analyst must understand fields to build detections and dashboards.”

📝 Homework (Day 14)

Students must extract:

• IP
• User
• Port
• Action

From one log of their choice, AND:

• Write MITRE technique
• Write one enriched dashboard query