Using the KQL what is the best strategy to detect randmised, possibly malicious domains…
- User
Using the KQL what is the best strategy to detect randmised, possibly malicious domains in URL?
Assistant
A great question about threat hunting with Kusto Query Language (KQL)!
Detecting randomized, possibly malicious domains in URLs is a challenging task, but KQL provides an excellent toolkit to tackle it. Here's a step-by-step strategy to help you identify suspicious domains:
Understand the characteristics of randomized malicious domains
Before we dive into the KQL queries, let's outline the common traits of randomized malicious domains:
- High entropy: Malicious domains often have randomly generated names with a mix of letters, numbers, and special characters, making them look unlike human-readable words.
- Unusual length: Randomized domains tend to be longer or shorter than legitimate domains.
- Low reputation: Newly registered or rarely seen domains are more likely to be malicious.
- Suspicious TLDs (Top-Level Domains): Malicious actors often abuse less common or newly introduced TLDs.
- Domain age: New domains (e.g., registered in the last 30 days) are more likely to be malicious.
KQL strategy
We'll use a combination of KQL functions, operators, and techniques to detect anomalous domains. Assume you have a table named
HttpEvent(e.g., from Azure Monitor, Microsoft Defender for Endpoint, or a custom log source) with a columnUrlcontaining the URLs to analyze.Step 1: Extract domain from URL
Use the
parse_url()function to extract the domain from theUrlcolumn:HttpEvent | extend Domain = tostring(parse_url(Url).Host)Step 2: Calculate domain entropy
Entropy measures the randomness of a string. We'll use a simple entropy calculation ( Shannon Entropy ) using
strlen()andmvexpand:HttpEvent | extend Domain = tostring(parse_url(Url).Host) | extend Entropy = strlen(Domain) * (1 - (array_length(mvexpand split(Domain, ''), typeof(string)) | summarize sum(itemCount) by item | project (sum_itemCount * log2(sum_itemCount)) | summarize sum_sum_itemCount_log2 | extend result = -1 * sum_sum_itemCount_log2 / strlen(Domain)) )Simplify the entropy calculation (optional)
If the previous query is too complex, use a simplified version:
HttpEvent | extend Domain = tostring(parse_url(Url).Host) | extend Entropy = strlen(replace_regex(Domain, @"[a-zA-Z0-9]", "", 0)) / strlen(Domain)This calculates the ratio of non-alphanumeric characters to the total length.
Step 3: Identify unusual domain lengths
Flag domains with lengths outside the common range (e.g., 5-25 characters):
HttpEvent | extend Domain = tostring(parse_url(Url).Host) | where strlen(Domain) < 5 or strlen(Domain) > 25Step 4: Analyze TLDs
Extract the TLD using
split()and flag uncommon TLDs (e.g., appearing < 1% of the time):HttpEvent | extend Domain = tostring(parse_url(Url).Host) | extend Tld = tostring(split(Domain, '.')[-1]) | summarize TldCount = count() by Tld | where TldCount < toint(count(Tld) * 0.01) // 1% threshold | join kind=inner (HttpEvent) on TldStep 5: Assess domain age (if you have WHOIS data)
If you have access to WHOIS data (e.g., from a separate table
WhoisDatawith columnsDomainandCreationDate), join and filter on new domains:HttpEvent | extend Domain = tostring(parse_url(Url).Host) | join kind=inner (WhoisData) on Domain | where CreationDate > ago(30d) // 30 days thresholdStep 6: Combine the factors
Unite the previous steps using
unionanddistinctto get a unified list of suspicious domains:let HighEntropyDomains = HttpEvent | extend Domain = tostring(parse_url(Url).Host) | extend Entropy = ... (use one of the entropy calculations) | where Entropy > 3.5; // adjust the threshold let UnusualLengthDomains = HttpEvent | extend Domain = tostring(parse_url(Url).Host) | where strlen(Domain) < 5 or strlen(Domain) > 25; let SuspiciousTlds = HttpEvent | extend Domain = tostring(parse_url(Url).Host) | extend Tld = tostring(split(Domain, '.')[-1]) | summarize TldCount = count() by Tld | where TldCount < toint(count(Tld) * 0.01) | join kind=inner (HttpEvent) on Tld; let NewDomains = HttpEvent | extend Domain = tostring(parse_url(Url).Host) | join kind=inner (WhoisData) on Domain | where CreationDate > ago(30d); union HighEntropyDomains, UnusualLengthDomains, SuspiciousTlds, NewDomains | distinct Domain | summarize Count = count() by Domain | where Count > 1 // adjust the threshold (e.g., 2, 3) | sort by Count descInterpret the results
The final output will show domains that triggered one or more of the detection rules. Investigate the top results (highest
Countvalues), focusing on:- Domains with high entropy, unusual lengths, or suspicious TLDs.
- Newly registered domains (if WHOIS data is available).
Refine and automate
- Store the KQL query as a function (e.g.,
DetectSuspiciousDomains()) for easy reuse. - Schedule the query to run periodically (e.g., every 4 hours) and store results in a separate table (e.g.,
SuspiciousDomains). - Integrate the results with your incident response process or threat intelligence platform.
- Continuously refine the thresholds and rules based on false positives and emerging threat patterns.
By following this strategy, you'll effectively leverage KQL to detect randomized, potentially malicious domains in your URL data. Happy hunting!
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.