Elastic

The ELK Stack and its components

The ELK Stack

The Elastic Stack is a collection of open-source software components developed by Elastic. The "stack" consists of three core products:

1. Elasticsearch

2. Logstash

3. Kibana

When used together, the Elastic Stack becomes a very versatile toolset that can be used for a wide range of use cases, including log analysis, application performance monitoring, security analysis, and business intelligence. The components work together allowing for the simplied collection, processing, and analysis of data from various sources.

Elasticsearch

The central component of the Elastic Stack, it is an open-source, distributed search and analytics engine designed for storing and indexing large volumes of data. Its highly scalable system can store and search various data types. It is built on top of Apache Lucene, an open source search engine library, and provides a simple REST API for indexing, searching, and analyzing.

Logstash

Logstash is a data ingestion and processing tool primarily used to collect, process, and transform data from various sources and prepare it for storage in Elasticsearch. Its designed to handle various data types including logs, metrics, events, and other data. It has a collection of input, filter, and output plugins that can be used to collect data from many sources, parse and transform it, and then send it to various destinations.

Kibana

The interactive graphical and visual front-end powering the Elastic Stack. Users can create interactive dashboards, visualizations, and reports based on data in Elasticsearch. It has the ability to search and filter log events.

Beats

Although Beats is not mandatory, it can be crucial in providing efficient and secure data collection capabilities. Beats are lightweight data "shippers" that collect various types of data from different endpoints and can then forward that data directly to Elasticsearch or Logstash for further processing. They can be tailored for specific use cases like collecting system logs, network traffic data, and metrics from servers and applications.

🧠 Elasticsearch & Kibana Cheatsheet

🔍 Basic Query Syntax

Goal	KQL Example	Lucene Example
Match exact term	field: "value"	field:"value"
Wildcard match	field: val*	field:val*
Contains substring	field: *val*	field:*val*
Match multiple values	field: ("val1" or "val2")	field:(val1 OR val2)
Must not match	NOT field: "value"	-field:"value"
Must match both	field1: "val1" AND field2: "val2"	field1:val1 AND field2:val2
Match one of many fields	field1:val OR field2:val	field1:val OR field2:val
Exists	field:*	_exists_:field
Does not exist	NOT field:*	-_exists_:field

📅 Date & Time Queries

Goal	KQL Example	Lucene Example
Last 24 hours	@timestamp > now-24h	@timestamp:[now-24h TO now]
Specific date	@timestamp >= "2024-07-01"	@timestamp:[2024-07-01 TO *]
Date range	@timestamp >= "2024-06-01" and @timestamp <= "2024-06-30"	@timestamp:[2024-06-01 TO 2024-06-30]
Relative time	@timestamp < now-15m	@timestamp:{* TO now-15m}

🔢 Numeric Queries

Goal	KQL Example	Lucene Example
Equals	bytes: 1000	bytes:1000
Greater than	bytes > 5000	bytes:{5000 TO *}
Less than	bytes < 2000	bytes:{* TO 2000}
Range inclusive	bytes >= 1000 and bytes <= 5000	bytes:[1000 TO 5000]

📁 String Matching Tricks

Goal	KQL Example	Notes
Prefix match	filename: "doc*"	Matches doc, document, docs, etc.
Contains word	message: "*error*"	Must be a keyword or text field
Case insensitive	user.keyword: "admin"	Ensure field is .keyword mapped
Fuzzy search	N/A	user:adm~

🧩 Advanced Tricks

Goal	KQL Example
Group with parentheses	(user:admin OR user:root) AND action:login
Boolean logic with fields	status:500 AND (path:/api/* OR path:/admin/*)
Nested field search	nested_field.property: "value"
Escaping special characters	message: "path\\: /usr/bin"

🧠 Common Useful Kibana Queries

A collection of practical queries for real-world use in detection, hunting, and investigation.

🔐 Authentication Events

Goal	KQL Query
Failed logins (Windows)	event.code: 4625
Successful logins (Windows)	event.code: 4624
Logon type 10 (Remote Desktop)	event.code: 4624 AND winlog.event_data.LogonType: 10
Multiple failed logins by same user	event.code: 4625 AND user.name: "jdoe"
Logins outside business hours	event.code: 4624 AND NOT @timestamp >= "08:00:00" AND NOT @timestamp <= "18:00:00"

📦 Process Execution

Goal	KQL Query
Suspicious PowerShell usage	process.name: powershell.exe AND process.args: "*EncodedCommand*"
Any use of cmd.exe	process.name: "cmd.exe"
Any use of certutil (often LOLBin)	process.name: "certutil.exe"
Parent-child relationship (cmd from Excel)	process.parent.name: excel.exe AND process.name: cmd.exe

🌐 Network / Web Activity

Goal	KQL Query
Connections to foreign IPs	destination.geo.country_name != "United States"
DNS queries for suspicious domains	dns.question.name: "*.ru" OR dns.question.name: "*.xyz"
Access to internal web admin portals	url.path: "/admin" OR url.path: "/login"
Unusual ports used	destination.port >= 1024 AND destination.port != 3389

🧩 File Access / Execution

Goal	KQL Query
Executables written to temp folder	file.path: "C:\\Users\\*\\AppData\\Local\\Temp\\*.exe"
Any .exe file creation	file.name: "*.exe" AND event.action: "created"
Use of USB device (Windows)	event.code: 4663 AND object.name: "*Removable*"
Downloads of .bat, .ps1, .vbs	url.full: "*.bat" OR url.full: "*.ps1" OR url.full: "*.vbs"

👤 User & Account Behavior

Goal	KQL Query
Privileged user actions	user.name: ("Administrator" or "root" or "SYSTEM")
New user account creation	event.code: 4720 (Windows)
Group membership changes	event.code: 4732 OR event.code: 4728
User account deletion	event.code: 4726

🚨 Suspicious Behavior Hunting

Goal	KQL Query
Base64 or encoded strings in commands	process.args: "*JVBER*" OR process.args: "*base64*"
Large file transfers	file.size > 100000000 (e.g., >100MB)
Scheduled task creation (Windows)	event.code: 4698
Registry modifications (Windows)	event.code: 4657
Suspicious parent/child combos	process.parent.name: "explorer.exe" AND process.name: "cmd.exe"

🧪 Misc / Operational Use

Goal	KQL Query
Check if a field exists	fieldname: *
Detect log source type	agent.type: "winlogbeat" or event.module: "suricata"
View only logs from a single host	host.name: "WIN-HOST-01"
Find all events with certain term	message: "*suspicious*"
Events with no destination IP	NOT destination.ip:*

💡 Tips

• Use filters to narrow down timeframe (top-right of Kibana).

• Combine fields using AND, OR, and parentheses.

• Always check .keyword field variants for exact matches.

• Use Discover to test queries, then export to visualizations.