logstash同步mysql数据,多表同步配制
来源:互联网 发布:内存稳定性测试软件 编辑:程序博客网 时间:2024/06/11 10:42
input {
stdin {
}
jdbc {
# mysql jdbc connection string to our backup databse
jdbc_connection_string => "jdbc:mysql://192.168.42.147:3306/test"
# the user we wish to excute our statement as
jdbc_user => "root"
jdbc_password => "root"
# the path to our downloaded jdbc driver
jdbc_driver_library => "/home/hadoop/opt/logstash-2.3.2/mysql-connector-java-5.1.29.jar"
# the name of the driver class for mysql
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
statement_filepath => "jdbc.sql"
schedule => "* * * * *"
type => "gjhz"
}
jdbc {
# mysql jdbc connection string to our backup databse
jdbc_connection_string => "jdbc:mysql://192.168.42.147:3306/test"
# the user we wish to excute our statement as
jdbc_user => "root"
jdbc_password => "root"
# the path to our downloaded jdbc driver
jdbc_driver_library => "/home/hadoop/opt/logstash-2.3.2/mysql-connector-java-5.1.29.jar"
# the name of the driver class for mysql
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
statement_filepath => "jdbc2.sql"
schedule => "* * * * *"
type => "rjfw"
}
}
filter {
json {
source => "message"
remove_field => ["message"]
}
}
output {
if [type] == "gjhz"{
elasticsearch {
hosts => "192.168.42.190:9200"
index => "zkr"
document_id => "%{a_id}"
}
}else{
elasticsearch {
hosts => "192.168.42.190:9200"
index => "zkr"
document_id => "%{d_id}"
}
}
stdout {
codec => json_lines
}
}
stdin {
}
jdbc {
# mysql jdbc connection string to our backup databse
jdbc_connection_string => "jdbc:mysql://192.168.42.147:3306/test"
# the user we wish to excute our statement as
jdbc_user => "root"
jdbc_password => "root"
# the path to our downloaded jdbc driver
jdbc_driver_library => "/home/hadoop/opt/logstash-2.3.2/mysql-connector-java-5.1.29.jar"
# the name of the driver class for mysql
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
statement_filepath => "jdbc.sql"
schedule => "* * * * *"
type => "gjhz"
}
jdbc {
# mysql jdbc connection string to our backup databse
jdbc_connection_string => "jdbc:mysql://192.168.42.147:3306/test"
# the user we wish to excute our statement as
jdbc_user => "root"
jdbc_password => "root"
# the path to our downloaded jdbc driver
jdbc_driver_library => "/home/hadoop/opt/logstash-2.3.2/mysql-connector-java-5.1.29.jar"
# the name of the driver class for mysql
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
statement_filepath => "jdbc2.sql"
schedule => "* * * * *"
type => "rjfw"
}
}
filter {
json {
source => "message"
remove_field => ["message"]
}
}
output {
if [type] == "gjhz"{
elasticsearch {
hosts => "192.168.42.190:9200"
index => "zkr"
document_id => "%{a_id}"
}
}else{
elasticsearch {
hosts => "192.168.42.190:9200"
index => "zkr"
document_id => "%{d_id}"
}
}
stdout {
codec => json_lines
}
}
阅读全文
0 0
- logstash同步mysql数据,多表同步配制
- logstash jdbc同步mysql多表数据到elasticsearch
- ElasticSearch5+logstash的logstash-input-jdbc实现mysql数据同步
- ElasticSearch5+logstash的logstash-input-jdbc实现mysql数据同步
- mysql 半同步复制配制
- logstash-output-mongodb实现Mysql到Mongodb数据同步
- Logstash的logstash-input-jdbc插件mysql数据同步ElasticSearch及词库
- Logstash实时同步mysql数据库至elastic
- elasticsearch5.2.1使用logstash同步mysql
- UbuntuLinux下用rsync进行数据备份和同步配制
- MYSQL 5 同步 MYSQL数据同步备份
- 设置 MySql 数据同步
- 设置 MySql 数据同步
- 设置 MySql 数据同步
- 设置 MySql 数据同步
- MySQL数据同步
- mySQL数据同步设置
- Mysql 的数据同步
- Zuul:智能路由和过滤(译)
- 来说说<Bridge>桥接模式
- Celery学习笔记(一)
- 对初学者建议“Android开发的10大技巧”
- 代码检视工具Gerrit的日常使用
- logstash同步mysql数据,多表同步配制
- 科普:论文上附有的 arXiv 是干嘛的
- 生产者消费者模式下的并发无锁环形缓冲区
- 扩增子分析流程QIIME. 1 使用Docker配置QIIME
- 并查集详解 (转)
- 图片相关工具类
- ajax跨域遇到的问题
- [Leetcode] 232. Implement Queue using Stacks 解题报告
- R极简教程-5:包