Learn how to pipe Lightrun Logs to Splunk’s HTTP Event Collector (HEC) using the FluentBit Log Processor.¶
Splunk allows you to search, analyze, and visualize all of your data, providing valuable insights on your infrastructure that you can act on. By integrating Lightrun with Splunk using Fluent Bit as a Log collector, you will be able to add real-time logs to your live application on demand and gain actionable insights on the data with Splunk.
This tutorial assumes that you have:
- A Lightrun account. Create your free Lightrun account.
- Access to a Splunk Enterprise/Cloud service account. Register for a Splunk free trial account here.
- Fluent Bit is installed and running. See Getting started with Fluent Bit for Fluent Bit installation instructions.
- Basic understanding of Fluent Bit concepts and terminologies.
Configure Splunk HEC¶
The first step in this tutorial is to set up Splunk to receive data through HTTPS.
To do that:
- Log in to your Splunk’s cloud dashboard.
- Go to Settings > Data Inputs > HTTP Event Collector > Global Settings.
- Ensure that your HTTP port number is 8080 and click Enable SSL.
- Go back to the HTTP Event Collector page and click New Token.
- Follow the steps to create an authentication token.
- Copy and save the token in a safe place.
Configure Fluent Bit¶
Fluent Bit has two main configuration components:
INPUT- Define where data should be collected from.
OUTPUT- Define where the collected data is sent to.
Add the following to your Fluent Bit configuration file as your INPUT.
[INPUT] name tail path /tmp/lightrun_file_handler_logs1.log
%TEMP% on Windows OS.
This configuration tells Fluent Bit to read from the
/tmp/lightrun_file_handler_logs1.log file as the input with the Tail input plugin.
Add the following to your Fluent Bit configuration file as your OUTPUT.
[OUTPUT] Name splunk Match * Host <SPLUNK_HOSTNAME/IP> Port 8088 Splunk_token <AUTH_TOKEN> TLS ON tls_verify off splunk_send_raw_data on
<SPLUNK_HOSTNAME/IP>to the hostname of the target Splunk service
<AUTH_TOKEN>to your Splunk authentication token.
This configuration tells Fluent Bit to send the collected data to Splunk Events Collector with the Fluent Bit Splunk plugin.
Fluent Bit configuration files are based on a strict Indented Mode, which means that each configuration file must follow the same pattern of alignment. By default, an indentation level is four spaces from left to right.
Confirm your integration¶
To confirm your integration, restart Fluent Bit and add a Lightrun log to your application. Your Lightrun Logs data should appear in your Splunk event console.