Logstash Log Shipper Agent Setup

Steps

Windows

1. Download Logstash on application server.  This requires a Java JDK to be on the server with a JAVA_HOME environment variable configured.
2. Unpackage Logstash to C:\ drive.  This will be your Logstash Agent home directory (C:\logstash-1.4.2).
3. Make the directory C:\logstash-1.4.2\log
4. Download nssm on application server.
5. Unzip nssm to C:\.  This is used to create a Windows service for the Logstash Agent (C:\nssm-2.24).
   1. If server is 32-bit OS, place a copy of the C:\nssm-2.24\win32\nssm.exe file in C:\logstash-1.4.2\bin
   2. If server is 64-bit OS, place a copy of the C:\nssm-2.24\win64\nssm.exe file in C:\logstash-1.4.2\bin
6. Create a logstash.conf file in C:\logstash-1.4.2\bin using a plain text editor
   

   • logstash.conf template

     input {    file {        path => "C:\PATH_TO_LOG_HERE.log"        type => "NAME_OF_LOG_HERE"    }}output {    if "_grokparsefailure" not in [tags] {        redis {            host => "10.50.101.51"            data_type => "list"            key => "logstash"            codec => "json"        }    }}

7. From command prompt, navigate to C:\logstash-1.4.2\bin

8. Run the following

   nssm install logstash

9. In the NSSM Service Installer window, enter the following
   1. On the Application tab
      1. Path – C:\logstash-1.4.2\bin\logstash.bat
      2. Startup Directory – C:\logstash-1.4.2\bin\
      3. Arguments – agent -f C:/logstash-1.4.2/bin/logstash.conf
      
   2. On the I/O tab
      1. Input (stdin) – C:\logstash-1.4.2\log\logstash.log
      2. Output (stdout) – C:\logstash-1.4.2\log\logstash.log
      3. Error (stderr) – C:\logstash-1.4.2\log\logstash.log
      
   3. On the File rotation tab
      1. Check Rotate files
      2. Check Rotate while service is running
      3. Restrict rotation to files bigger than – 50000000 bytes
      
10. Click Install Service.  If successful, should get the following
    
11. Open services.msc and ensure newly created "logstash" service is not started/running

12. Open C:\logstash-1.4.2\bin\logstash.bat in a plain text editor

13. Ensure the following and save

    if "%LS_MAX_MEM%" == "" (REM set LS_MAX_MEM=1gset LS_MAX_MEM=500m)

14. Download the logstash contrib plugin
15. Unpackage the contents of the logstash contrib plugin into the Logstash Agent home directory (C:\logstash-1.4.2).  Overwrite folders/files as necessary.

16. Open the C:\logstash-1.4.2\lib\logstash\inputs\eventlog.rb file in a plain text editor, make the following edit, and save the file

    #     while#       notification = events.NextEvent#   comment out the above and add the below per https://github.com/elasticsearch/logstash/pull/1674/files?diff=split      while true        begin          notification = events.NextEvent(1000) #timeout is 1000 ms        rescue          next        end

17. Open services.msc and start the logstash service