# Graylog & ElasticSearch

Graylog is an open-source log management system that collects, indexes, and analyzes log data from various sources, providing real-time monitoring and alerting. Elasticsearch is a distributed search and analytics engine that efficiently indexes and queries large volumes of structured and unstructured data. Graylog often uses Elasticsearch as its storage and indexing backend for fast log searching and analysis. We will be installing the Graylog & Elastic Search on the Ubuntu 24.04LTS

***

## Installing MongoDB Database (Community Edition)

Mongodb is the database system service which can be integrated by the graylog and which can be installed from the below given URL

{% hint style="info" %}
<https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-debian/>
{% endhint %}

#### Installation Steps :&#x20;

1. Importing the key pairs :&#x20;

```bash
sudo apt-get install gnupg curl
curl -fsSL https://www.mongodb.org/static/pgp/server-8.0.asc | \
   sudo gpg -o /usr/share/keyrings/mongodb-server-8.0.gpg \
   --dearmor
```

2. Creating the list file&#x20;

{% code overflow="wrap" %}

```bash
echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-8.0.gpg ] https://repo.mongodb.org/apt/ubuntu noble/mongodb-org/8.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-8.0.list
```

{% endcode %}

3. Reloading the packages database & installing mongodb

```bash
sudo apt-get update
sudo apt-get install -y mongodb-org
```

4. Registering services & enabling on startup

```bash
sudo systemctl start mongod
sudo systemctl enable mongod
```

Here we have installed the mongoDB database and now we will be installing elastic search.

***

## Installing Elastic Search

1. Installing necessary packages&#x20;

```bash
sudo apt install default-jdk apt-transport-https
```

2. Installing the elastic search GPG Key

{% code overflow="wrap" %}

```bash
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
```

{% endcode %}

3. **Add the Elasticsearch repository to your sources list:**

{% code overflow="wrap" %}

```bash
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
```

{% endcode %}

4. Updating the packages & installing Elastic search

```bash
sudo apt update 
sudo apt install elasticsearch
```

Now the elastic search is installed and now we will configure the elastic search to enable the nodes & clusters to integrate with the graylog server

The configuration file of elasticsearch is located at `/etc/elasticsearch/elasticsearch.yml`  we will add the following lines at the end of config file to enable the cluster :&#x20;

```bash
sudo nano /etc/elasticsearch/elasticsearch.yml
## END OF THE CONFIG FILE
---------------------------------------
## CLUSTER CONFIG 
cluster.name: graylog
action.auto_create_index: false
```

We will also set the value of parameter `xpack.security.enabled: false` to disable security pack feature as it might not give outputs as expected.

```bash
sudo nano /etc/elasticsearch/elasticsearch.yml
# Enable security features
xpack.security.enabled: false
```

Now we will restart the service of elastic search&#x20;

```bash
sudo systemctl restart elasticsearch
```

Now we will be creating the new password for the user elastic so that we can manage it using the default credentials.

{% code overflow="wrap" %}

```bash
cd /usr/share/elasticsearch && sudo ./bin/elasticsearch-reset-password -u elastic
```

{% endcode %}

This will ask the option to display the password as plain text in the console then press `y` and note the password displayed at the terminal. Now we will check wheather our elastic node is working perfect or not :&#x20;

{% code overflow="wrap" %}

```bash
curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:<PASSWORD_GENERATED> http://localhost:9200
```

{% endcode %}

The output should look like :&#x20;

{% code overflow="wrap" %}

```bash
{
  "name" : "Legion",
  "cluster_name" : "graylog",
  "cluster_uuid" : "HY1TK3vQRb6_wCpKPNFHlg",
  "version" : {
    "number" : "8.17.2",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "747663ddda3421467150de0e4301e8d4bc636b0c",
    "build_date" : "2025-02-05T22:10:57.067596412Z",
    "build_snapshot" : false,
    "lucene_version" : "9.12.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
```

{% endcode %}

<figure><img src="/files/dNbxV0ncBLnxUgRb2DhS" alt=""><figcaption></figcaption></figure>

Now we have successfully configured the elastic search node and now we will start configuring the graylog datanode & the graylog server.

***

## Installing Graylog-datanode & Graylog Server

Firstly we will fetch the local repositories file from the graylog repo server&#x20;

{% code overflow="wrap" %}

```bash
wget https://packages.graylog2.org/repo/packages/graylog-6.1-repository_latest.deb
```

{% endcode %}

Now we will install the `.deb` package which was downloaded above using&#x20;

```bash
sudo dpkg -i <package_name_graylog>.deb
```

Once installed the debian package we will now update the repositories & install the necessary packages&#x20;

```bash
sudo apt install -y graylog-server graylog-datanode
```

Once installed we will start configuring graylog server and integrate with the elastic search datanode which was deployed above.

To run Graylog Data Node, the value of the "vm.max\_map\_count" sysctl setting needs to be at least 262144 or more. Run the following commands to set and persist the value.

```bash
 sudo sysctl -w vm.max_map_count=262144
```

If the value is not the same we can do this by this command :&#x20;

```bash
echo "vm.max_map_count = 262144" | sudo tee /etc/sysctl.d/99-graylog-datanode.conf
```

***

## Configuring Graylog-server

The config file is located on `/etc/graylog/server/server.conf` , Firstly we will generate the 96 characters long random key and the encrypted password string to feed as input in the config file and then edit certian parameters to run the graylog server.

**Generate a secret key** for password hashing:

```bash
pwgen -N 1 -s 96
```

**Generate an admin password hash**:

```bash
echo -n 'YourPassword' | sha256sum
```

We will enter this vaules in the config files where the parameters are `password_secret` & `root_password_sha2`&#x20;

{% code overflow="wrap" %}

```bash
## PASSWORD SECRET
password_secret = RPASN987AEGyMaatdfPRWQRXp5yTwhtDolY8UHxVA1wiNbzF5aaaaavruEBt0Bpz70356wMG30qRLO8SSAiUg8bza3t33vyr
## ROOT PASSWD
root_password_sha2 = 6f2cb9dd8f4b65e24e1c3f3fa5bc5aaaaa49237f11abceacd45bbcb74d621c25
```

{% endcode %}

**Set Bind Address**:

```bash
http_bind_address = 0.0.0.0:9000
```

**Connect Graylog with Elasticsearch**

```bash
elasticsearch_hosts = http://127.0.0.1:9200
```

Start & Enable services:&#x20;

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now graylog-server graylog-datanode
sudo systemctl restart graylog-server graylog-datanode
```

***

## Configuring datanode

Create your `password_secret` with the following command:

```bash
< /dev/urandom tr -dc A-Z-a-z-0-9 | head -c${1:-96};echo;
```

Edit the config file:

```bash
sudo nano /etc/graylog/datanode/datanode.conf
```

Enter the key just generated in the password\_secret to enable the datanode otherwise the datanode might not initialize

{% code overflow="wrap" %}

```bash
password_secret =DvM-BWV8HEMmKtxtzkG1VnWtVEL9PX9WaFfAICkNDdELcKTMsFYbs18oWpzBFYv42zGz
```

{% endcode %}

Ensure the following parameters are configured properly :&#x20;

```bash
elasticsearch_hosts = http://127.0.0.1:9200
```

Save and restart:

```bash
sudo systemctl restart graylog-datanode
```

***

## Accessing the Greylog UI

```bash
http://localhost:9000
```

***

## Troubleshooting

#### Elastic search version 8.x not supported , Downgrade to 7.10.2

**Step 1: Uninstall Elasticsearch 8.x**

```bash
sudo systemctl stop elasticsearch
sudo apt remove --purge elasticsearch -y
sudo rm -rf /var/lib/elasticsearch /etc/elasticsearch
```

#### **Step 2: Install Elasticsearch 7.10.2**

{% code overflow="wrap" %}

```bash
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.2-amd64.deb
sudo dpkg -i elasticsearch-7.10.2-amd64.deb
```

{% endcode %}

#### **Step 3: Configure Elasticsearch**

Edit the config file:

```bash
sudo nano /etc/elasticsearch/elasticsearch.yml
```

Modify:

```
cluster.name: graylog
network.host: 127.0.0.1
http.port: 9200
discovery.type: single-node
```

Save and restart:

```bash
sudo systemctl enable --now elasticsearch
```

#### **Step 4: Verify Elasticsearch**

```bash
curl -X GET "http://127.0.0.1:9200"
```

Now restart Graylog:

```bash
sudo systemctl restart graylog-server
```

***


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://ghoulsec.gitbook.io/ghoulsec-vault/security-operations/graylog-and-elasticsearch.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.