How to configure Logstash to get Application, System, Security and Sysmon logs from…
- User
How to configure Logstash to get Application, System, Security and Sysmon logs from Winlogbeat and send them to Elasticsearch?
- Assistant
To configure Logstash to receive logs from Winlogbeat and send them to Elasticsearch, you need to set up both Winlogbeat and Logstash properly. Below are the steps to achieve this:
Step 1: Configure Winlogbeat
-
Install Winlogbeat: Download and install Winlogbeat on your Windows machine. The installation files can be obtained from the official Elastic downloads page.
-
Edit the Winlogbeat configuration file (
winlogbeat.yml):- Define the event logs you want to collect with Winlogbeat, such as Application, System, Security, and Sysmon logs.
- Configure the output to Logstash.
winlogbeat.event_logs: - name: Application - name: System - name: Security - name: Microsoft-Windows-Sysmon/Operational output.logstash: hosts: ["your-logstash-server:5044"] ssl: certificate_authorities: ["path/to/ca.crt"]- Start Winlogbeat: Use the following PowerShell command to start Winlogbeat:
Start-Service winlogbeat
Step 2: Configure Logstash
-
Install Logstash: Download and install Logstash from the official Elastic downloads page.
-
Create a Logstash configuration file (for example,
winlogbeat.conf):- Define input settings to receive data from Winlogbeat.
- Optionally, add filters for processing the logs.
- Define the output settings to send processed logs to Elasticsearch.
input { beats { port => 5044 ssl => true ssl_certificate => "/path/to/logstash.crt" ssl_key => "/path/to/logstash.key" } } filter { if [log][level] == "ERROR" { mutate { add_field => { "severity" => "high" } } } # Add additional filters as needed } output { elasticsearch { hosts => ["http://your-elasticsearch-server:9200"] index => "winlogbeat-%{+YYYY.MM.dd}" } stdout { codec => rubydebug } }- Start Logstash: Use the following command to start Logstash with the configuration file:
bin/logstash -f /path/to/winlogbeat.conf
Step 3: Verify the Setup
-
Check Winlogbeat Logs: Ensure Winlogbeat is running without errors and logs are being sent to Logstash.
Get-EventLog -LogName Application -Newest 10 -
Check Logstash Logs: Verify that Logstash is receiving and processing events correctly.
-
Check Elasticsearch: Verify that the logs are present in Elasticsearch by using Kibana or querying Elasticsearch directly.
curl -X GET "localhost:9200/winlogbeat-*/_search" -H 'Content-Type: application/json' -d' { "query": { "match_all": {} } } '
By following these steps, you will have Winlogbeat configured to send logs to Logstash, which will then forward them to Elasticsearch for storage and analysis. Adjust paths, hosts, and SSL settings according to your environment specifics. You can also add more advanced filtering and transformations in the Logstash configuration to meet your needs.
-
- User
i dont use sertificate for logstash. And where to store winlogbeat.conf for logstash?
- Assistant
If you are not using SSL/TLS for encrypting data between Winlogbeat and Logstash, simply omit the SSL-related configurations in both Winlogbeat and Logstash configurations. As for storing the Logstash configuration file, you can place it in the Logstash configuration directory or any directory you prefer, and then start Logstash with the configuration file path.
Step 1: Configure Winlogbeat without SSL
Edit the
winlogbeat.ymlconfiguration file:winlogbeat.event_logs: - name: Application - name: System - name: Security - name: Microsoft-Windows-Sysmon/Operational output.logstash: hosts: ["your-logstash-server:5044"]Step 2: Configure Logstash without SSL
- Create the Logstash Configuration File:
You can name the configuration file as desired, for example,
winlogbeat.conf. This file can be stored in the Logstash configuration directory (typically located in/etc/logstash/conf.d/on Linux, orC:\Logstash\config\on Windows, depending on your installation).
Here is a sample Logstash configuration file without SSL settings:
input { beats { port => 5044 } } filter { if [log][level] == "ERROR" { mutate { add_field => { "severity" => "high" } } } # Add additional filters as needed } output { elasticsearch { hosts => ["http://your-elasticsearch-server:9200"] index => "winlogbeat-%{+YYYY.MM.dd}" } stdout { codec => rubydebug } }-
Store the Configuration File: Place the
winlogbeat.conffile in the appropriate directory. For instance:- On Linux:
/etc/logstash/conf.d/winlogbeat.conf - On Windows:
C:\Logstash\config\winlogbeat.conf
- On Linux:
-
Start Logstash: Start Logstash with the configuration file.
# For Linux bin/logstash -f /etc/logstash/conf.d/winlogbeat.conf # For Windows bin\logstash.bat -f C:\Logstash\config\winlogbeat.conf
Step 3: Verify the Setup
-
Check Winlogbeat Logs: Ensure Winlogbeat is running without errors and logs are being sent to Logstash.
Get-EventLog -LogName Application -Newest 10 -
Check Logstash Logs: Verify that Logstash is running and receiving events correctly.
-
Check Elasticsearch: Verify that the logs are present in Elasticsearch by using Kibana or querying Elasticsearch directly.
curl -X GET "http://your-elasticsearch-server:9200/winlogbeat-*/_search" -H 'Content-Type: application/json' -d' { "query": { "match_all": {} } } '
By following these steps, Winlogbeat will send the event logs to Logstash, and Logstash will forward them to Elasticsearch without requiring SSL/TLS configuration. This basic setup helps you get started without additional complications introduced by SSL/TLS.
- Create the Logstash Configuration File:
You can name the configuration file as desired, for example,
- User
How to check Winlogbeat service works good
- Assistant
Ensuring that Winlogbeat is functioning correctly involves several steps, such as checking the service status, inspecting log files, and verifying that logs are being sent to Logstash or Elasticsearch. Here is a comprehensive guide to help you verify that Winlogbeat is working properly on a Windows machine:
1. Check Winlogbeat Service Status
Open PowerShell with administrative privileges and run the following command:
Get-Service -Name winlogbeatYou should see the status of the Winlogbeat service. It should be "Running" if the service is active.
2. Check Winlogbeat Logs
Winlogbeat logs are usually stored in the
logsdirectory within the Winlogbeat installation directory (e.g.,C:\Program Files\Winlogbeat\logs). Inspect these logs for any errors or warnings.Open the latest log file in a text editor or use PowerShell to view the last few lines:
Get-Content "C:\Program Files\Winlogbeat\logs\winlogbeat" -Tail 50Look for any error messages or issues that might indicate problems.
3. Verify Events Being Sent
You can verify that events are being picked up by Winlogbeat by using PowerShell to read from the event logs specified in your
winlogbeat.ymlconfiguration file:Get-WinEvent -LogName Application -MaxEvents 5 Get-WinEvent -LogName System -MaxEvents 5 Get-WinEvent -LogName Security -MaxEvents 5 Get-WinEvent -LogName Microsoft-Windows-Sysmon/Operational -MaxEvents 54. Check Data in Logstash or Elasticsearch
When Using Logstash
- Logstash Configuration: Ensure that Logstash is running and correctly configured to receive data from Winlogbeat.
- Verify Data Reception: Open the Logstash logs or use a tool like
netcatto ensure that Logstash is receiving data on port 5044.
When Using Elasticsearch Directly
- Kibana: Navigate to Kibana and check if Winlogbeat indices are being created.
- Elasticsearch Query: Use a curl command or any REST client to query Elasticsearch and check if Winlogbeat data is present.
Example with
curl:curl -X GET "http://your-elasticsearch-server:9200/winlogbeat-*/_search" -H 'Content-Type: application/json' -d' { "query": { "match_all": {} } } '5. Verify Output in Kibana
- Open Kibana using your browser.
- Navigate to Discover.
- Select the
winlogbeat-*index pattern (you might need to create it if it doesn't exist). - Check if the events are visible in Kibana.
Summary
By checking service status, inspecting logs, verifying event capture, and ensuring data is being received by Logstash or Elasticsearch, you can confirm that Winlogbeat is functioning correctly. If errors or issues are found during these checks, they can typically be resolved by adjusting configurations and rechecking system permissions.