.NET 分布式 搜索 elasticsearch.net NEST
来源:互联网 发布:饭菜先生软件出问题 编辑:程序博客网 时间:2024/05/17 23:31
我们公司(电商)目前使用 elasticsearch。对于一个3亿数据量索引查询速度都是在毫秒级别,不过实现都是基于JAVA的client. 本人因为是.NET出身,虽然现在做JAVA。但是对.NET情有独钟。所以闲暇时间研究了下 elasticsearch的.NET 的CLIENT的NEST.希望.NET人员多使用 elasticsearch 来做搜索服务 解决.NET一直没有好的搜索服务。本人以前也研究过 Lucene.Net。但是无法实现集群及分布式。
本人也参考了很多前辈博客和和本公司做搜索的同事 和 google。走了很多弯路。所以希望后来人别走很多弯路。
Elasticsearch一般至少最少3台服务。2台服务器时候1一台是主分片。另外一台永远都是副本所以在服务,无法实现集群式搜索。只能实现主从备份模式。
所以服务器建议3台以上。这样分片才会平衡到3台服务器上。
参考资料
https://www.nuget.org/packages/Nest
http://blog.csdn.net/huwei2003/article/details/40980929
https://www.elastic.co/guide/en/elasticsearch/client/net-api/1.x/nest.html
安装Elasticsearch
网上已经有很多资料了。所以这里不再写了
1、VS2012安装NEST
PM> Install-Package NEST -version 1.7.2
我安装的是1.7.2的客户端版本。
2、创建连接
public static ElasticClient GetElasticClient(string indexName) { var connectString = "http://127.0.0.1:9200"; var nodesStr = connectString.Split('|'); var nodes = nodesStr.Select(s => new Uri(s)).ToList(); var connectionPool = new SniffingConnectionPool(nodes); var settings = new ConnectionSettings(connectionPool); if (!string.IsNullOrWhiteSpace(indexName)) { settings.SetDefaultIndex(indexName); } var client = new ElasticClient(settings); return client; }
3、创建索引
创建索引只需要调用一次即可。 索引结构如下:
[ElasticType(IdProperty = "id", Name = "order")]public class Order{ [ElasticProperty(Name = "id", Type = FieldType.Long, Index = FieldIndexOption.NotAnalyzed)] public long Id { get; set; }public string Content { get; set; }[ElasticProperty(Name = "firstname", Type = FieldType.String, Index = FieldIndexOption.Analyzed)]public string Firstname { get; set; }[ElasticProperty(Name = "lastname", Type = FieldType.String, Index = FieldIndexOption.NotAnalyzed)]public string Lastname { get; set; } [ElasticProperty(Name = "totalprice", Type = FieldType.Double, Index = FieldIndexOption.NotAnalyzed)]public Double TotalPrice { get; set; }[ElasticProperty(Name = "createtime", Type = FieldType.Date, Index = FieldIndexOption.Analyzed)]public DateTime Createtime { get; set; } }
[ElasticType(IdProperty = “id”, Name = “order”)]
这个IdProperty字段属性是搜索在做分片时候需要HASH对象。所以选择这个可以根据数据库的分区键或者数据库主键来选择具体哪个字段来做IdProperty。一般都是选择数据库主键
Content:这个字段未定义属性。 所以这个字段不能拿来做查询条件。
要做查询条件的字段都必须加上
[ElasticProperty(Name = “lastname”, Type = FieldType.String, Index = FieldIndexOption.NotAnalyzed)]
或者
[ElasticProperty(Name = “createtime”, Type = FieldType.Date, Index = FieldIndexOption.Analyzed)]
这两个的区别是
NotAnalyzed 是不分词只做索引字段
Analyzed 作为查询字段并且分词
所以创建索引必须规划好每个字段是否分词及是否查询。
/// <summary> /// 创建索引 /// </summary> /// <param name="indexName"></param> private static void CreateIndex(string indexName) { var client = ElasticsearchHelper.GetElasticClient(""); //client 不能给 default client.CreateIndex(r => r.Index(indexName).NumberOfReplicas(1).NumberOfShards(20)); client.Map<Order>(m => m.MapFromAttributes()); }
这里是创建ORDER的索引并且每个分片一个副本 总共20个分片。
20分片实际要根据业务数据量多少来决定。
4、删除索引
/// <summary> /// 删除索引 /// </summary> private static void DeleteIndex() { var client = ElasticsearchHelper.GetElasticClient("order"); var result = client.DeleteIndex(r => r.Index("order")); System.Console.WriteLine(result.IsValid); }
5、写入数据
private static void WriteIndex() { var client = ElasticsearchHelper.GetElasticClient("order"); long i = GetMax(); while (true) { i++; SetMax(i); Order order = new Order() { Id = i, Firstname = "我爱天安门", Lastname = "外派", TotalPrice =Convert.ToDouble(i), Content = "天使3412341234" + i.ToString(), Createtime = DateTime.Now }; var addResult = client.Index(order); System.Console.WriteLine("第" + i + "行记录" + addResult.Created); } } static long GetMax() { long reslt = 0; try { string k = System.IO.File.ReadAllText("1.txt"); reslt = Convert.ToInt64(k); } catch (Exception) { } return reslt; } static long SetMax(long max) { long reslt = 0; try { System.IO.File.WriteAllText("1.txt", max.ToString()); } catch (Exception) { } return reslt; }
6、查询
#region 求和var client = ElasticsearchHelper.GetElasticClient("order");QueryContainer termQuery = new TermQuery() { Field = "lastname", Value = "求和" };var result = client.Search<Order>(s => s .Aggregations(a => a .Sum("my_sum_agg", sa => sa .Field(p => p.TotalPrice) ) ) );var agg = result.Aggs.Sum("my_sum_agg");var searchResults = client.Search<Order>(s => s // .Query(termQuery) //带筛选条件 .Aggregations(r => r.Terms("firstname", r1 => r1.Field(r2 => r2.Lastname) .OrderAscending("sumprice") .Aggregations(y => y.Sum("sumprice", y1 => y1.Field(y2 => y2.TotalPrice))))));var carTypes = searchResults.Aggs.Terms("firstname"); List<double> re=new List<double>(); foreach (var carType in carTypes.Items) { string key = carType.Key; System.Console.WriteLine("key:" + key + " total:" + carType.Sum("sumprice").Value); re.Add((double)carType.Sum("sumprice").Value); } //List<SumTotalPrice> orders = searchResults.Documents.ToList(); // System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total); //System.Console.WriteLine(searchResults.RequestInformation); System.Console.ReadLine(); #endregion #region 多条件精确匹配 var client = ElasticsearchHelper.GetElasticClient("order"); QueryContainer termQuery = new TermQuery() { Field = "firstname", Value = "测试" }; QueryContainer wholeWordQuery = new TermQuery() { Field = "lastname", Value = "addddd" }; //非索引字段。无法查询 此查询条件无效 QueryContainer termQuery1 = new TermQuery() { Field = "content", Value = " 天空是百度的1696" }; termQuery = (termQuery || wholeWordQuery) || termQuery1; var searchResults = client.Search<Order>(s => s .From(0) .Size(10) .Query(termQuery) ); List<Order> orders = searchResults.Documents.ToList(); System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total); System.Console.ReadLine(); #endregion #region 精确匹配 var client = ElasticsearchHelper.GetElasticClient("order"); var searchResults = client.Search<Order>(s => s .From(0) .Size(10) .Query(q => q.Term(r => r.OnField(k => k.Firstname).Value("棉花"))) .Highlight(h => h.OnFields(e => e.OnField("firstname") .PreTags("<b style='color:black'>") .PostTags("</b>"))) .Sort(r => r.Descending().OnField(q => q.Createtime)) ); List<Order> orders = searchResults.Documents.ToList(); System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total); System.Console.ReadLine(); #endregion // //查询索引字段 带 裤的 类似 数据库的 like '裤%' var client = ElasticsearchHelper.GetElasticClient("order"); var searchResults = client.Search<Order>(s => s .From(0) .Size(10) .QueryString("裤*") ); System.Console.WriteLine(" total:" + searchResults.Total); System.Console.ReadLine(); #region ID查询 var searchResults = client.Search<Order>(s => s .From(0) .Size(10) .Query(q => q.Ids(r=>r.Values(19))) ); #endregion #region 时间范围查询 var client = ElasticsearchHelper.GetElasticClient("order"); // QueryContainer termQuery = new TermQuery() { Field = "firstname", Value = "测试" }; var format = "yyyy-MM-dd'T'HH:mm:ss.fff+08:00"; var lowerBound = DateTime.Now.AddMinutes(-20); var searchResults = client.Search<Order>(s => s // .Query(r => termQuery && r.Range(st => st.OnField(st1 => st1.Createtime).GreaterOrEquals(lowerBound, format))) .Query(r => r.Range(st => st.OnField(st1 => st1.Createtime).GreaterOrEquals(lowerBound, format))) .SortDescending(r=>r.Createtime) .From(0) .Size(10) ); List<Order> orders = searchResults.Documents.ToList(); var k= searchResults.RequestInformation; System.Console.WriteLine(orders.Count() + " total:" + searchResults.Total); System.Console.ReadLine(); #endregion
7、删除数据
private static void DeleteRowById(long id) { var client = ElasticsearchHelper.GetElasticClient("order"); client.Delete<Order>(r => r.Id(id)); }
- .NET 分布式 搜索 elasticsearch.net NEST
- .NET 分布式 搜索 elasticsearch.net NEST
- Elasticsearch .net client NEST使用说明
- NEST.net Client For Elasticsearch简单应用
- Elasticsearch.Net Nest 5.0.0 用法
- ElasticSearch NEST搜索
- ElasticSearch.net NEST批量创建修改删除索引完整示例
- Elasticsearch(十)【NEST高级客户端--搜索查询】
- 分布式搜索elasticsearch java API 之(一)------与集群交互 - 云端分布式搜索技术 - 博客频道 - CSDN.NET
- ElasticSearch NEST
- plainelastic.net elasticsearch实现搜索业务
- 分布式搜索Elasticsearch
- Elasticsearch分布式搜索
- 分布式搜索Elasticsearch 概述
- 分布式搜索elasticsearch 基本概念
- [Elasticsearch] 分布式搜索
- [Elasticsearch] 分布式搜索
- [Elasticsearch] 分布式搜索
- java异常信息
- 如何制作商场超市地摊清仓大甩卖广告语录音
- 10034---Jaxb2 实现JavaBean与xml互转
- Django model字段类型清单
- 算法题:最多覆盖点数
- .NET 分布式 搜索 elasticsearch.net NEST
- 那些想你的日子
- 2012.3.23
- 截取部分图片并显示-ios例子[转载]
- 大图片中截取其中一部分小图显示 两图合成一图[转载]
- iOS代码搜索网站
- \"No previous prototype for function\" warning警告错误
- Please ensure that adb is correctly located 问题的解决
- 第11章非阻塞I/O