# Managing multiple Wazuh clusters with Cross-Cluster Search

**Cross-Cluster Search (CCS) in Wazuh** enables querying and viewing alerts from remote Wazuh clusters at a centralized location, while keeping log data at the source cluster. This setup is trusted by remote clusters and allows security alerts to be monitored via a single Wazuh dashboard, even in a remote Security Operations Center (SOC).

**Use Case:**

A cybersecurity operations company provides managed SOC services to two customers (Organizations A and B). Each customer requires separate log management with tightly controlled access and data retention within their own environment.

**Solution Overview:**

* **CCS Environment**: This environment is used only for querying data from remote Wazuh clusters. It contains the Wazuh indexer and dashboard but no agents or Wazuh server. It is trusted by clusters A and B.
* **Cluster A**: Contains a Wazuh server and indexer for Organization A, with no dashboard (visualization occurs in the CCS environment). It trusts the CCS environment for data access.
* **Cluster B**: Contains a Wazuh server and indexer for Organization B, also with no dashboard, but trusts the CCS environment for data access.

<figure><img src="/files/OVpQPNsRkJhubYawRWyS" alt=""><figcaption></figcaption></figure>

The table below shows a further breakdown of the infrastructure.

| **Cluster** | **Component**   | **Node name**       | **IP address**  |
| ----------- | --------------- | ------------------- | --------------- |
| CCS         | Wazuh dashboard | ccs-wazuh-dashboard | 192.168.146.154 |
|             | Wazuh indexer   | ccs-wazuh-indexer-1 |                 |
| Cluster A   | Wazuh server    | ca-wazuh-server-1   | 192.168.146.155 |
|             | Wazuh indexer   | ca-wazuh-indexer-1  |                 |
| Cluster B   | Wazuh server    | cb-wazuh-server-1   | 192.168.146.156 |
|             | Wazuh indexer   | cb-wazuh-indexer-1  |                 |

Ensure network connectivity between the remote clusters and the CCS environment, irrespective of the network address.

***

## CSS Environment Configuration :&#x20;

A Cross-Cluster Search (CCS) environment is authorized to perform search operations on remote Wazuh clusters. It is typically made up of the Wazuh indexer and the Wazuh dashboard. The Wazuh indexer node establishes trust with indexer nodes in remote clusters, enabling it to search for log data on these indexers. The data on the Wazuh indexer nodes in the remote clusters are displayed on the Wazuh dashboard in the CCS environment.&#x20;

### Generating certificates

Download the `wazuh-certs-tool.sh` script and the `config.yml` configuration file. The `wazuh-certs-tool.sh` script is used to generate certificates for the cluster. The `config.yml` file defines the IP addresses and node names of the Wazuh central components to be deployed.

```bash
# ENSURE YOU ARE CHANGING URLS BASED ON LATEST STABLE VERSION OF WAZUH
curl -sO https://packages.wazuh.com/4.9/config.yml
curl -sO https://packages.wazuh.com/4.9/wazuh-certs-tool.sh
```

Edit the `config.yml` file and replace the name and IP variables with the corresponding node names and IP addresses:

```bash
nodes:
  # Wazuh indexer nodes
  indexer:
    - name: ccs-wazuh-indexer-1
      ip: "192.168.146.154"
  # Wazuh dashboard nodes
  dashboard:
    - name: ccs-wazuh-dashboard
      ip: "192.168.146.154"
```

> **Note:** Comment out or remove the Wazuh server configuration since it is not in use.

Run the `wazuh-certs-tool.sh` script with option `-A` to create the `root-ca`, `admin`, and node certificates:

```bash
bash ./wazuh-certs-tool.sh -A
# THE OUTPUT SHOULD LOOK LIKE 
25/08/2024 15:22:57 INFO: Generating the root certificate.
25/08/2024 15:22:57 INFO: Generating Admin certificates.
25/08/2024 15:22:58 INFO: Admin certificates created.
25/08/2024 15:22:58 INFO: Generating Wazuh indexer certificates.
25/08/2024 15:22:58 INFO: Wazuh indexer certificates created.
25/08/2024 15:22:58 INFO: Generating Wazuh dashboard certificates.
25/08/2024 15:22:59 INFO: Wazuh dashboard certificates created.
```

Copy the root CA certificate to the working directory of a node in cluster A and cluster B using the `scp` utility. The root CA is used later in the remote clusters to generate certificates.

Here we will pass the root certificates file into the nodes to ensure the connections are encrypted between master & worker nodes

{% code overflow="wrap" %}

```bash
scp -r wazuh-certificates/root-ca.* <USER_NAME>@<IP_ADDRESS>:<DESTINATION_DIRECTORY>
## MY COMMANDS
#scp -r wazuh-certificates/root-ca.* clustera@192.168.146.155:/home/clustera
#scp -r wazuh-certificates/root-ca.* clustern@192.168.146.156:/home/clusterb
```

{% endcode %}

Compress all the certificate files and remove the uncompressed version to allow for easier transfer to other component nodes within the CCS environment if need be:

```bash
tar -cvf ./wazuh-certificates.tar -C ./wazuh-certificates/ .
```

***

### Installing & configuring Wazuh indexer

Perform the following configuration steps on the Wazuh indexer node for the CCS environment. In my case the machine I used is ubuntu , you can install the indexer from the manual for any other distributions of linux

{% code overflow="wrap" %}

```bash
apt-get install debconf adduser procps
## ADDING WAZUH REPO
apt-get install gnupg apt-transport-httpscurl -s https://packages.wazuh.com/key/GPG-KEY-WAZUH | gpg --no-default-keyring --keyring gnupg-ring:/usr/share/keyrings/wazuh.gpg --import && chmod 644 /usr/share/keyrings/wazuh.gpg
## ADDING REPO SOURCE FILE
echo "deb [signed-by=/usr/share/keyrings/wazuh.gpg] https://packages.wazuh.com/4.x/apt/ stable main" | tee -a /etc/apt/sources.list.d/wazuh.list
## UPDATING PACKAGES 
apt-get update
```

{% endcode %}

* Installing Wazuh Indexer

```bash
apt-get -y install wazuh-indexer
```

Edit the `/etc/wazuh-indexer/opensearch.yml` configuration file:

{% code overflow="wrap" %}

```bash
network.host: "192.168.146.154" <-- CHANGE THIS -->
node.name: "ccs-wazuh-indexer-1" <-- CHANGE THIS ->

cluster.initial_master_nodes:
- "ccs-wazuh-indexer-1" <-- CHANGE THIS --> 
cluster.name: "ccs-cluster" <-- CHANGE THIS --> 
node.max_local_storage_nodes: "3"

plugins.security.nodes_dn:
- "CN=ccs-wazuh-indexer-1,OU=Wazuh,O=Wazuh,L=California,C=US" <-- CHANGE THIS -->
#- "CN=node-2,OU=Wazuh,O=Wazuh,L=California,C=US"
#- "CN=node-3,OU=Wazuh,O=Wazuh,L=California,C=US"
plugins.security.restapi.roles_enabled:
- "all_access"
- "security_rest_api_access"
```

{% endcode %}

The lines `<— CHANGE THIS — >` has to be modified & need to enter the correct details we entered in the `config.yaml` file when deploying the certificates on the start

Replace the values of:

* `network.host` with the IP address of the Wazuh indexer node.
* `node.name` with the name of the Wazuh indexer node set in the `config.yml` file.
* `cluster.initial_master_nodes` with the name of the Wazuh indexer node in the CCS environment.&#x20;
* `cluster.name` with the name of the cluster. For example, `ccs-cluster`.
* `plugins.security.nodes_dn` with the distinguished name (DN) of the certificate of the CCS Wazuh indexer nodes. The common name (CN) is to be replaced with the node name of the CCS Wazuh indexer node provided in the `config.yml` file.

&#x20;Run the following commands, replacing `<INDEXER_NODE_NAME>` with the value of `node.name` configured in the `/etc/wazuh-indexer/opensearch.yml` file:

```bash
NODE_NAME=ccs-wazuh-indexer-1 <-- CHANGE THIS -->
mkdir /etc/wazuh-indexer/certs
tar -xf ./wazuh-certificates.tar -C /etc/wazuh-indexer/certs/ ./$NODE_NAME.pem ./$NODE_NAME-key.pem ./admin.pem ./admin-key.pem ./root-ca.pem
mv -n /etc/wazuh-indexer/certs/$NODE_NAME.pem /etc/wazuh-indexer/certs/indexer.pem
mv -n /etc/wazuh-indexer/certs/$NODE_NAME-key.pem /etc/wazuh-indexer/certs/indexer-key.pem
chmod 500 /etc/wazuh-indexer/certs
chmod 400 /etc/wazuh-indexer/certs/*
chown -R wazuh-indexer:wazuh-indexer /etc/wazuh-indexer/certs
```

Enable and start the Wazuh indexer service:

```bash
systemctl daemon-reload
systemctl enable wazuh-indexer
systemctl start wazuh-indexer
```

Run the Wazuh indexer `indexer-security-init.sh` script on the Wazuh indexer node to load the new certificate information and initialize the Wazuh indexer:

```bash
/usr/share/wazuh-indexer/bin/indexer-security-init.sh
```

The output should look like :&#x20;

{% code overflow="wrap" %}

```bash
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
Security Admin v7
Will connect to 192.168.186.60:9200 ... done
Connected as "CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US"
OpenSearch Version: 2.10.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: ccs-cluster
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
SUCC: Expected 10 config types for node {"updated_config_types":["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"],"updated_config_size":10,"message":null} is 10 (["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"]) due to: null
Done with success
```

{% endcode %}

`DONE WITH SUCCESS & Cluster State Green` defines the indexer is correctly configured

Check that the Wazuh indexer is initialized and properly set up by running the following command:

```bash
curl -k -u admin:admin https://192.168.146.154:9200
```

The output should look like :&#x20;

```bash
{
  "name" : "ccs-wazuh-indexer-1",
  "cluster_name" : "ccs-cluster",
  "cluster_uuid" : "JkH8XuirS6m6PgdZFEQV2g",
  "version" : {
    "number" : "7.10.2",
    "build_type" : "rpm",
    "build_hash" : "eee49cb340edc6c4d489bcd9324dda571fc8dc03",
    "build_date" : "2023-09-20T23:54:29.889267151Z",
    "build_snapshot" : false,
    "lucene_version" : "9.7.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}
```

***

### Installing & Configuring Wazuh Dashboard

```bash
apt-get install debhelper tar curl libcap2-bin #debhelper version 9 or later
apt-get install gnupg apt-transport-https
## INSTALLING GPG KEY
curl -s https://packages.wazuh.com/key/GPG-KEY-WAZUH | gpg --no-default-keyring --keyring gnupg-ring:/usr/share/keyrings/wazuh.gpg --import && chmod 644 /usr/share/keyrings/wazuh.gpg
## ADDING THE REPOSITORY
echo "deb [signed-by=/usr/share/keyrings/wazuh.gpg] https://packages.wazuh.com/4.x/apt/ stable main" | tee -a /etc/apt/sources.list.d/wazuh.list
## UPATING PACKAGES
apt-get update
apt-get -y install wazuh-dashboard
```

&#x20;Edit the `/etc/wazuh-dashboard/opensearch_dashboards.yml` file to add the URL value of the Wazuh indexer in the `opensearch.hosts` section.

```bash
server.host: 0.0.0.0
   server.port: 443
   opensearch.hosts: "https://192.168.146.154:9200" <--CHANGE THIS-->
   opensearch.ssl.verificationMode: certificate
```

Replace `<DASHBOARD_NODE_NAME>` with your Wazuh dashboard node name, the same one set in `config.yml` when creating the certificates, and move the certificates to their corresponding directories:

```bash
NODE_NAME=ccs-wazuh-dashboard <--CHANGE THIS-->
mkdir /etc/wazuh-dashboard/certs
tar -xf ./wazuh-certificates.tar -C /etc/wazuh-dashboard/certs/ ./$NODE_NAME.pem ./$NODE_NAME-key.pem ./root-ca.pem
mv -n /etc/wazuh-dashboard/certs/$NODE_NAME.pem /etc/wazuh-dashboard/certs/dashboard.pem
mv -n /etc/wazuh-dashboard/certs/$NODE_NAME-key.pem /etc/wazuh-dashboard/certs/dashboard-key.pem
chmod 500 /etc/wazuh-dashboard/certs
chmod 400 /etc/wazuh-dashboard/certs/*
chown -R wazuh-dashboard:wazuh-dashboard /etc/wazuh-dashboard/certs
```

Enable and start the Wazuh dashboard service:

```bash
systemctl daemon-reload
systemctl enable wazuh-dashboard
systemctl start wazuh-dashboard
```

Edit the `/usr/share/wazuh-dashboard/data/wazuh/config/wazuh.yml` file and replace the `hosts` section with a similar configuration as below:

```bash
hosts:
  - Cluster A:
      url: https://192.168.146.155 <--CHANGE THIS-->
      port: 55000
      username: wazuh-wui
      password: wazuh-wui
      run_as: true
  - Cluster B:
      url: https://192.168.146.156 <--CHANGE THIS-->
      port: 55000
      username: wazuh-wui
      password: wazuh-wui
      run_as: true
```

Restart the Wazuh dashboard service to apply the changes:

```bash
systemctl restart wazuh-dashboard
```

> **Note:** Accessing the Wazuh dashboard at this configuration stage will produce API errors since the Wazuh servers have not been installed yet. Complete all the steps before accessing the dashboard in the [Set up Cross-Cluster Search](https://wazuh.com/blog/managing-multiple-wazuh-clusters-with-cross-cluster-search/#set-up-cross-cluster-search) section.

In the next section we will configure the Clusters & API connections


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://ghoulsec.gitbook.io/ghoulsec-vault/security-operations/wazuh-open-source-xdr.-open-source-siem./installation/managing-multiple-wazuh-clusters-with-cross-cluster-search.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.