logstash-out-elasticsearch 设置geo_point格式
来源:互联网 发布:volvos90怎么样知乎 编辑:程序博客网 时间:2024/05/21 15:03
此项目是logstash从kafka拉取数据,并取到elastisearch.并进行格式处理。保存geo_point类型,可以进行空间数据查询
配置文件如下:
es.conf
input {
kafka {
type => "test"
auto_offset_reset => "smallest"
group_id => "mon9"
topic_id => "db100"
zk_connect => "192.168.199.6:2181,192.168.199.7:2181,192.168.199.8:2181"
}
}
filter {
mutate {
split => { "message" => "," }
add_field => {
"id" => "%{message[1]}"
"DZBM_s" => "%{message[2]}"
"DZQC_s" => "%{message[3]}"
"SJYTMC_s" => "%{message[4]}"
"FWYT_s" => "%{message[5]}"
"lon" => "%{message[6]}"
"lat" => "%{message[7]}"
}
remove_field => [ "message", "host" ]
}
mutate {
convert => { "lon" => "float" }
convert => { "lat" => "float" }
}
mutate {
rename => {
"lon" => "[geo][lon]"
"lat" => "[geo][lat]"
}
}
}
output {
elasticsearch {
index => "base_map"
template => "map_template.json"
document_type => "node_points"
hosts => ["192.168.199.10:9200", "192.168.199.11:9200"]
}
}
map_template.json:
{
"template": "base_map",
"order": 1,
"settings": {
"number_of_shards": 1
},
"mappings": {
"node_points" : {
"properties" : {
"geo" : { "type" : "geo_point" }
}
}
}
}
注意:
mapping文件路径一定要写正确,否则不能映射
- logstash-out-elasticsearch 设置geo_point格式
- logstash-out-mongodb实现elasticsearch到Mongodb的数据同步
- elasticsearch geo_point 地理位置过滤 按经度排序
- elasticsearch geo_point 字段 经纬度地址字段返回距离信息
- 各大数据库的配置 & logstash-out-mongodb实现elasticsearch到Mongodb的数据同步
- LogStash+ElasticSearch性能问题?
- elasticsearch+logstash+kibana 初探
- Elasticsearch & Logstash -- 一些经验总结
- logstash+elasticsearch+kibana3
- Elasticsearch+Logstash+Kibana配置
- elasticsearch+logstash+kibana+marvel
- kafka-logstash-elasticsearch
- logstash-kafka-elasticsearch
- 搭建elasticsearch+logstash+kibana
- logstash Output plugins >> elasticsearch
- elasticsearch, logstash 安装
- Elasticsearch logstash kibane 搭建
- ELK-ElasticSearch+Logstash+Kibana
- ThinkPHP单字母函数(快捷方法)使用总结
- SVD和PCA
- 类型检测
- LeetCode 23. Merge k Sorted Lists 【hard】
- spring注解方式读取properties文件中值
- logstash-out-elasticsearch 设置geo_point格式
- Kinect for Windows 体系构架
- update语句更新时和其他的表联合查询
- javascript优缺点
- Dubbo与Zookeeper、SpringMVC整合和使用
- linux中常见的>/dev/null 2>&1的作用
- 【自定义View】自定义View之炫丽CheckBox
- Android开发内存管理
- 为什么24阶魔尺照着感觉就能扭出正方体