logstash 读取多个系统相同文件shipper端

来源：互联网发布：淘宝客服售后聊天技巧编辑：程序博客网时间：2024/06/06 01:45
input {        file {                type => "weblogic_10.4.32.48_access"                path => ["/data01/zzpt/weblogic-10.4.32.48-access*"]                start_position=>"beginning"                codec=>plain {                 charset=>"GBK"               }        }           file {                 type => "weblogic_10.4.32.111_access"                 path => ["/data01/hdxt/weblogic-10.4.32.111-access*"]                   start_position=>"beginning"                codec=>plain {                 charset=>"GBK"               }        }  }filter {        mutate {                       add_field =>["messages","%{type}-%{message}"]                       remove_field =>["message"]                }    grok {        match =>[              "messages","\s*%{IPORHOST:clientip}\s*\-\s*\-\s*\[(?<time>(\S+\s+).*?)\]\s+\"%{WORD:verb}\s+%{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:http_status_code} %{NUMBER:bytes}"        ]    }    # mutate {   #      remove_field =>["messages"]   #  }}output {        redis {                host => "10.4.145.93"                data_type => "list"                key => "%{type}:redis"                port=>"6379"                password => "1234567"        }         stdout {                        codec => rubydebug                } }        测试logstash 配置；[elk@rsyslog logstash]$ logstash -f logstash_weblogic_access_log.conf   127.0.0.1:6379> keys *(empty list or set)模拟数据:[elk@rsyslog zzpt]$ echo "10.4.32.48test001" >>weblogic-10.4.32.48-access.2017-06-29127.0.0.1:6379> keys *1) "weblogic_10.4.32.48_access:redis"127.0.0.1:6379> keys *1) "weblogic_10.4.32.48_access:redis"127.0.0.1:6379> BLPOP "weblogic_10.4.32.48_access:redis" 01) "weblogic_10.4.32.48_access:redis"2) "{\"@version\":\"1\",\"@timestamp\":\"2017-07-05T00:31:52.649Z\",\"path\":\"/data01/zzpt/weblogic-10.4.32.48-access.2017-06-29\",\"host\":\"rsyslog\",\"type\":\"weblogic_10.4.32.48_access\",\"messages\":\"weblogic_10.4.32.48_access-10.4.32.48test001\",\"tags\":[\"_grokparsefailure\"]}"[elk@rsyslog hdxt]$ echo "10.4.32.111--tlcb999" >> weblogic-10.4.32.111-access.2017-03-18127.0.0.1:6379> keys *(empty list or set)127.0.0.1:6379> keys *1) "weblogic_10.4.32.111_access:redis"127.0.0.1:6379> LLEN "weblogic_10.4.32.111_access:redis"(integer) 1127.0.0.1:6379> brpop "weblogic_10.4.32.111_access:redis" 01) "weblogic_10.4.32.111_access:redis"2) "{\"@version\":\"1\",\"@timestamp\":\"2017-07-05T00:37:25.443Z\",\"path\":\"/data01/hdxt/weblogic-10.4.32.111-access.2017-03-18\",\"host\":\"rsyslog\",\"type\":\"weblogic_10.4.32.111_access\",\"messages\":\"weblogic_10.4.32.111_access-10.4.32.111--tlcb999\",\"tags\":[\"_grokparsefailure\"]}"
阅读全文
0 0