Tutorial: Install Logstash and Kibana on a Windows server

https://www.ulyaoth.net/resources/tutorial-install-logstash-and-kibana-on-a-windows-server.34/

This guide should give you a good example of how to setup Logstash on a Windows environment for anyone that is not interested in using Linux.

This guide was setup with taking into consideration that you have a single server for your "loghost" and not run anything else on it, if you do run other website on your IIS then please make sure you put the "reverse proxy" from this guide on the vhost and not on all of IIS as I do below.

This guide was tested on:
Windows 10 Professional
Windows 2012 Standard
Windows Server 2016 Technical Preview 3

In this guide I will show that it is also possible to run Logstash on a Windows machine and use IIS as web server.

If your are looking for the Linux version of this guide please have a look here:
How to install Logstash & Kibana on Fedora using Rsyslog as shipper.
How to install Logstash & Kibana on Fedora using logstash-forwarder as shipper.

So lets start!

Step 1: Download Logstash, Kibana and ElasticSearch.
Simply go to "https://www.elasticsearch.org/overview/elkdownloads/"

Elasticsearch: https://download.elasticsearch.org/...p/elasticsearch/2.0.0/elasticsearch-2.0.0.zip
Logstash: https://download.elastic.co/logstash/logstash/logstash-2.0.0.zip
Kibana: https://download.elastic.co/kibana/kibana/kibana-4.2.0-windows.zip
nxlog: http://sourceforge.net/projects/nxlog-ce/files/

On the nxlog website choose the latest "msi" file, at the time of writing this guide it was: nxlog-ce-2.8.1248

Feel free to download the newest version!

Step 2: Extract all packages
I created myself a folder called "ulyaoth" in "c:\ulyaoth\" and extracted all folders there to make it more easy.

So for me it looks like this now:
c:\ulyaoth\elasticsearch
c:\ulyaoth\logstash
c:\ulyaoth\kibana

Please be aware this is the directory structure I use, if you change the naming of the directories or files you have to update everything correct in the guide to fit your situation.

Step 3: Download the JDK version of Java and install it.
Go to the java website: https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

Accept the license and then download: "Windows x64 (jdk-8u60-windows-x64.exe)" package.
Now install it!

(Since Oracle updates Java frequently please download the latest and ignore if my version above is slightly older)

Step 4: Add the JAVA_HOME variable to the server
Now right click on "This PC" and choose "Properties" on the right bottom site next to your computer and full computer name click on Change settings. On the window that opens go to the Advanced tab and click on "Environment Variables" at the bottom box called "System Variables" click on new and add the following:
Variable Name: JAVA_HOME
Variable value: C:\Program Files\Java\jdk1.8.0_66

Should look like this:


(Same as I wrote with Step 3 if your Java version is newer please update the JAVA_HOME entry accordingly)

Step 5: Download the required configuration file for Logstash.
logstash.conf

Place this file in:
C:\ulyaoth\logstash\bin

Also make sure to open the actual config and see if you need to change anything that will fit your situation or naming.

Step 6: Install IIS
Windows Server:
Go to "Server Manager" and choose "Add Roles and Features Wizard" from the list here choose "Web Server (IIS)" now go further and let it install.

Windows 10:
On Windows 10 you should go to your "Control Panel" then go to "Program and Features" and then click on "Turn Windows features on or off" here you will find "Internet Information Service" which is IIS.

Step 7: Install Microsoft Web Platform Installer 5.0
Go to the following website: http://www.microsoft.com/web/downloads/platform.aspx

Once you have downloaded the file called "wpilauncher.exe" simply click it and at one point you come at a Window with applications and you "Add" button on the right. In the top is a search window, search here for ARR you should see a window as here below:


On this screen choose "Application Request Routing 3.0" and then click on "Add" and then
Search also for "Rewrite" and click "Add" behind "URL Rewrite 2.o.

Now press the "Install" in the button and a window as below should appear:


now simply click on "I Accept" and everything will install for you, a window like this appears:


When everything is finished correctly you see the following:


Just press on "Finish" to complete the installation and then on "Exit" to close the whole app.

Step 8: Open IIS Manager and you should see a window as below.


Step 9: Create a new website for Kibana as shown below
Right click on "sites" in the left part of IIS Manager and click "Add Website".

Fill it in something like this:


It should automatically start, if you do not have a certificate then choose "http" at the Binding part.

Step 10: Create a reverse proxy in IIS to Kibana.
Click on your newly created name in the left window mine is called "loghost.ulyaoth.net" and then in the left window allot of icons will appear:


Now double click on the icon "Rewrite URL" and then in the right window click on "Add Rules(s)...", and a window should popup that looks like this:


Now click on "Reverse Proxy" and then click OK in the bottom a new window will popup as below:

Now make sure you fill everything in exactly as I did above:
Inbound Rules: http://127.0.01:5601

And if you did choose "https" for your website then also enable the option: Enable SSL Offloading

Now you should have everything as I did so go ahead and press on the "OK" button.

You probably will get a warning that you are about to use ARR so please go ahead and accept this.

To explain it in short the ip you entered "127.0.0.1" and port "5601" this is where Kibana will run under and you are basically making tunnel to get data from that internal program to people that will connect to your domain. If you change Kibana's port then you of-course have to update the above reverse proxy rule.

Step 11: Start elasticsearch and put it on autostart
Open a console and go to "c:\ulyaoth\elasticsearch\bin\"
now type the following command:

Code:

service install

You should see something like:


Now type the following:

Code:

service manager

You should see the elasticsearch service manager:


You have to change on the tab the "Startup type" from Manual to Automatic and then press Apply this should make Elasticsearch start automatically on server boot.

This window contains some more options such as how much memory Elasticsearch will use you can find this on the "Java" tab I would suggest to make this fitting for your server if you have a servers that will handle a huge amount of logs I would increase the "Maximum Memory Pool: 1024" at least to a higher amount.

Before you close the window make sure to press "Start" so it actually will run right now 

This is everything to start ElasticSearch automatically on boot to test that it is working open a browser and go to the url "127.0.0.1:9200".

If you see a json string something like what you see below in the picture then it means your ElasticSearch is running:


Step 12: Create a Logstash startup bat file
Now you might wonder why create a additional bat file when Logstash does already provide this!? Well the reason for this step is that some others that use this more extensive on windows have some Java freezes after running Logstash for a while and creating this file will resolve that issue.

"run.bat" should simply contain this line:

Code:

logstash.bat agent -f logstash.conf

Now place your "run.bat" in the folder: C:\ulyaoth\logstash\bin

Step 13: Start Logstash & Autostart it
For this step we need another small program to create a proper windows service so please go ahead and download "NSSM" (the Non-Sucking Service Manager) from: http://nssm.cc/

Code:

https://nssm.cc/release/nssm-2.24.zip

Once you have the zip file simply unzip it and copy the file from the unzipped folder you now have: "nssm-2.24\win64" (nssm.exe) to "C:\ulyaoth\logstash\bin" so it should result in you having "C:\ulyaoth\logstash\bin\nssm.exe".

Now open a Command Prompt and type:

Code:

cd C:\ulyaoth\logstash\bin

And then type the following:

Code:

nssm install logstash

You will now see a GUI to create a server fill in the following:
Path: C:\ulyaoth\logstash\bin\run.bat
Startup directory: C:\ulyaoth\logstash\bin

It should look like this:


If everything looks the same then go to the "Details" and add the following:
Display name: Logstash

Also make sure that "Startup type" is on Automatic.

Now the following step is additional but recommended if you are 100% going to use Elasticsearch, click on the "Dependencies" tab and add the following: elasticsearch-service-x64

The reason for this step is that Logstash can do difficult if you have a Elasticsearch output but it is not actually running.

Now this is all so press on the "Install service" button to finish setting up a service for Logstash.


If you wish to adjust the memory Logstash does use then simply open the file "C:\ulyaoth\logstash\bin\logstash.bat" and the change the following two lines accordingly to the amount of memory you wish it to use:

Code:

set LS_MIN_MEM=256mset LS_MAX_MEM=1g

Step 14: Start Kibana & Autostart it
In the previous step you downloaded "nssm.exe" you will need to copy the same file once more but this time copy it to: "C:\ulyaoth\kibana\bin\nssm.exe"

Now open a Command Prompt and type:

Code:

cd C:\ulyaoth\kibana\bin

And then type the following:

Code:

nssm install kibana

You will now see a GUI to create a server fill in the following:
Path: C:\ulyaoth\kibana\bin\kibana.bat
Startup directory: C:\ulyaoth\kibana\bin

It should look like this:


If everything looks the same then go to the "Details" and add the following:
Display name: Kibana

Also make sure that "Startup type" is on Automatic.

Now the following step is additional but recommended if you are 100% going to use Elasticsearch, click on the "Dependencies" tab and add the following:
elasticsearch-service-x64
logstash

The reason for this step is that Kibana won't start without elasticsearch, Logstash requirement is not really required but since I only use Kibana with Logstash I see no reason to not add it.

Now this is all so press on the "Install service" button to finish setting up a service for Kibana.


If you wish to adjust the settings of Kibana such as running it on a different port or IP simply go to "C:\ulyaoth\kibana\config\kibana.yml" and play around with the available settings.

Step 15: Install nxlog
Double click the msi file and you should see a window like this:


Now press on "Install" and you will see the following window:


Just wait till the installation finishes and a new window shows like this:


On this window press on "Finish" and that should be it to install nxlog.

Step 16: Download the nxlog config file.
Download my github config file for nxlog:
nxlog.conf

And place this file inside: "C:\Program Files (x86)\nxlog\conf"
You have to overwrite the existing empty one.

Now open the file and change the information such as IP / port to fit your environment.

Depending on your IIS setup you might need to change the exact IIS location also I am talking about this line:

Code:

File    "C:\inetpub\logs\LogFiles\W3SVC1\u_ex*.log"

Step 17: Start Logstash, Kibana and nxlog
Open the program "services.msc" and now start the programs in the following order:

• Logstash
• Kibana
• nxlog

If you did everything correct then Kibana should be running so lets test it by going to "http://loghost.ulyaoth.net/" or the website name you did choose and you should see that Kibana is started:


Now to configure Kibana fill the window above in as you can see below on the picture:


When you have filled everything in exactly as me above then press on "Create" sometimes you not see the same options this means Logstash did not put anything in Elasticsearch yet so just refresh the website a few times to generate logs and it will eventually work.

Well that is all to configure Kibana now click in the top on "Discover" and you will see the final result with the IIS logs from Kibana as below.


A small note sometimes it happens that because you use a reverse proxy that IIS is actually not logging you must enable reverse proxy logging in that case. Alternative just make nxlog push some other kind of log to Elasticsarch.

This was everything you now have successfully installed Logstash, Elasticsearch, Kibana and we used Logstash-Forwarder as a shipper on Windows! As final I would probably restart the server to make sure all services will start automatically.

If you have any suggestions to improve this guide then please feel free to or update the configs on GitHub or to provider me the information so I can update the guide so it can help others!, just click the "Ask Questions / Get Support" button