8.6.4. Quick creation of a POC Logstash

8.6.4.1. Introduction

This procedure describes how to quickly create a Logstash Proof Of Concept.

8.6.4.2. Prerequisites

  • User : member of Administrator group

8.6.4.3. Preliminary operations

8.6.4.4. Procedure

A POC with a Logstash docker can be done in minutes.
The following commands, given as an indication, should facilitate this task.

Important

The controls are given as an indication to mount a demonstrator quickly. It does not follow the best practices necessary for the development of a production component.

  • On a Linux machine with docker, run the following commands to retrieve the default Logstash configuration files: (procedure tested with Logstash version 7.13.1)

    bash
mkdir logstash_docker
cd logstash_docker
sudo docker run --name="logstash_tmp" --rm -d -it docker.elastic.co/logstash/logstash:7.13.1
sudo docker cp logstash_tmp:/usr/share/logstash/config config
sudo docker cp logstash_tmp:/usr/share/logstash/pipeline pipeline
sudo docker rm -f logstash_tmp
A logstash_docker folder has been created with two subfolders: config and pipeline.
In config, parameters can be kept by default, except for the xpack.monitoring.elasticsearch.hosts parameter which must be commented in logstash.yaml.
  • In the pipeline folder, replace the default pipeline with the pipeline described in the section above.
    A docker using these configuration files and this pipeline can then be started:
    sudo docker run --name="logstash_export" --rm -d -it -p $LOGSTASH_PORT:$LOGSTASH_PORT/$PROTOCOL -v $(pwd)/config/:/usr/share/logstash/config/ -v $(pwd)/pipeline:/usr/share/logstash/pipeline/ -v $(pwd)/output:/usr/share/logstash/data/output/ --user $(id -u):$(id -g) docker.elastic.co/logstash/logstash:7.13.1

Logstash will then create an output directory in which the received logs will be written, with one JSON per line.