Upgrade Your Splunk to a Deployment Server

I show in this tutorial how to upgrade your Splunk to a deployment server.

Here’s What You Need

• Splunk Enterprise – Trial license running on Ubuntu 18.04
• Windows 2016 Server – Evaluation license

Getting Started

When installing the universal forwarder on the Windows server there are two options. Choose the second option, because we are setting up the deployment server-client deployment in this tutorial. However our first app that will be deployed will define a receiving port on the Splunk server anyway. This is an easy way to demonstrate how deployment servers work.

Follow the installation options and make note of the two options to get data from the Windows server to the Splunk server.

1. Setup a receiving port on the Splunk server, port 9997.
2. Define the deployment server ip address or hostname.

The Splunk User

It’s best practice to run Splunk and its services on *nix systems as a non-root user.

# change to splunk user
su - splunk

# give splunk user(non root user) permissions to all files
sudo chown -R splunk:splunk $SPLUNK_HOME

Activate the Deployment Server

Create a new folder named sendtoindexer. In the folder create a file, outputs.conf and fill it with the lines shown below. The contents of outputs.conf:

[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = 192.168.56.99:9997

[tcpout-server://192.168.56.99:9997]

We’re now going to move the app into the deployment-apps folder. Doing this turns your standard Splunk search head/indexer into a deployment server.

# copy outputs.conf to the new sendtoindexer app
sudo cp outputs.conf /opt/splunk/etc/apps/sendtoindexer

# copy the add folder to the right spot
sudo cp /opt/splunk/etc/apps/sendtoindexer /$SPLUNK_HOME/etc/deployment-apps

# restart Splunk
sudo reboot

Moving just one app into the $SPLUNK_HOME/etc/deployment-apps folder is all it takes to activate the deployment server. The indexer is now additionally a deployment server to be used to deploy to listeners, otherwise known as clients.

And what do clients do? They phone home to the deployment server, looking for new updates. By calling home to the deployment server, the server can keep track of active connections to its clients. The deployment server requires a check in from clients.

Apps are configuration files really. A working setup will display the number of clients connecting and the number of apps deployed, as seen below.

Now that the app is assigned to the new Universal Forwarders server class the next time the client phones home it will automatically receive the app.

The next step is go to Settings > Forwarder Management. Click edit on the Universal Forwarders server class, then edit clients. This page is where you need to give the Splunk deployment server the ip address of the client which is the Windows server.

TIP: If you need to change the ip address of the deployment server after installing the universal forwarder make the change on this configuration file. $SPLUNK_HOME/etc/system/local/deploymentclient.conf

[target-broker:deploymentServer]
targetUri = 192.168.56.99:8089

Open the Windows Services app and restart SplunkForwarder Service.

What we have just deployed is the simple sendtoindexer app. Look again at the outputs.conf file. The file is merely defining a tcp out server which is the ip address of our deployment server.

sudo tar -C /opt/splunk/etc/deployment-apps/ -xzf splunk-app-for-windows-infrastructure_151.tgz

1. Assign an app a server class
2. Assign a server class the client host(s)

That means I created a new server class and named it Domain Controllers. I then extracted the download file for the Splunk App for Windows Infrastructure into the $SPLUNK_HOME/etc/deployment-apps directory. Then I assigned the app to the server class. Lastly, I defined the Windows server ip address as the client for the Domain Controllers server class.

Install the app on the search head/indexer by going to manage apps > install app from file. Switch to the new app in Splunk Web to get started.