Elasticsearch (ES) (上万字）详细学习总结

ops/2024/9/23 6:23:49/

一、认识ES

二、ES相关安装和部署(elasticsearch 、kbana、ik)

这部分的内容可以查看我之前写的Docker常用部署这篇文章

三、Mapping映射

3.1 Mapping映射属性

3.2 索引库操作 

3.2.1 遵循Restful规范说明

3.2.2 具体使用方式说明 

 3.2.3增删改查示例

#创建
PUT /heima
{"mappings":{"properties": {"info":{"type": "text","analyzer": "ik_smart"},"email": {"type": "keyword","index": false},"name": {"type": "object","properties": {"firstName":{"type": "keyword"},"lastName": {"type": "keyword"}}}}}
}#查询
GET /heima#删除
DELETE /heima#添加
PUT /heima/_mapping
{"properties":{"age":{"type":"byte"}}
}

3.2.4 总结

3.3 文档的操作

3.3.1 文档的增删改查示例

#新增文档（如果已经存在则进行修改）
POST /heima/_doc/1
{"info": "Java入门到精通","email": "csh@qq.com","name": {"firstName": "云","lastName": "赵"}
}#查询文档
GET /heima/_doc/1#删除文档
DELETE /heima/_doc/1#全量修改（直接覆盖之前的文档，如果不存在该文档则会创建新的文档）
PUT /heima/_doc/1
{"info": "Python入门到精通","email": "csh@163.com","name": {"firstName": "白","lastName": "李"}
}#增量修改（局部修改）
POST /heima/_update/1
{"doc":{"email":"333@163.com"}
}

3.3.2 文档批量处理

#批量添加
POST /_bulk
{"index": {"_index" : "heima" , "_id":1}}
{"info":"Java入门到精通","email":"csh@qq.com","name":{"firstName": "云","lastName": "赵"}}
{"index": {"_index" : "heima" , "_id":2}}
{"info":"Python入门到精通","email":"csh@163.com","name":{"firstName": "白","lastName": "李"}}#批量删除
POST /heima/_bulk
{"delete": {"_index" : "heima" , "_id":1}}
{"delete": {"_index" : "heima" , "_id":2}}

3.3.3 总结

四、JavaRestClient（索引库）

4.1 客户端初始化

 第一步：

引入依赖：

        <dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-high-level-client</artifactId></dependency>

第二步：

在Spring-boot-dependency中找出<elaticsearch.version>7.17.0</elaticsearch.version>，

将其复制在父工程的<properties></properties>中修改版本号好为7.12.1（可根据自己所需要的版本进行修改）

第三步：通过RestHighLevelClient使用ES

建立ES连接示例：
 

package com.etc.search;import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;@SpringBootTest
class TripSearchApplicationTests {private RestHighLevelClient client;@Testvoid testConnection() {System.out.println("client: "+client);}@BeforeEachvoid setUp(){client = new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.92.136:9200")));}@AfterEachvoid tearDown() throws IOException{if(client != null){client.close();}}}

4.2 商品映射Mapping

 kibana创建示例：

PUT /item
{"mappings": {"properties": {"id":{"type": "keyword"},"name":{"type": "text","analyzer": "ik_smart"},"price":{"type": "integer"},"image":{"type": "keyword","index": false},"category":{"type":"keyword"},"brand":{"type": "keyword"},"sold":{"type": "integer"},"commentCount":{"type": "integer","index": false},"isAD":{"type": "boolean"},"updateTime":{"type": "date"}}}
}

4.3 使用Java客户端示例：

package com.etc.search;import org.apache.http.HttpHost;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.common.xcontent.XContentType;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.web.bind.annotation.DeleteMapping;import java.io.IOException;@SpringBootTest
class TripSearchApplicationTests {private RestHighLevelClient client;@Testvoid testConnection() {System.out.println("client: "+client);}@Testvoid testCreateIndex() throws IOException{CreateIndexRequest request = new CreateIndexRequest("items");request.source(MAPPING_TEMPLATE, XContentType.JSON);client.indices().create(request, RequestOptions.DEFAULT);}@Testvoid testGetIndex() throws IOException{GetIndexRequest request = new GetIndexRequest("items");boolean exists =  client.indices().exists(request, RequestOptions.DEFAULT);System.out.println("exists:"+exists);}@Testvoid testDeleteIndex() throws IOException{DeleteIndexRequest request = new DeleteIndexRequest("items");client.indices().delete(request, RequestOptions.DEFAULT);}@BeforeEachvoid setUp(){client = new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.92.136:9200")));}@AfterEachvoid tearDown() throws IOException{if(client != null){client.close();}}private static final String MAPPING_TEMPLATE = "{\n" +"  \"mappings\": {\n" +"    \"properties\": {\n" +"      \"id\":{\n" +"        \"type\": \"keyword\"\n" +"      },\n" +"      \"name\":{\n" +"        \"type\": \"text\",\n" +"        \"analyzer\": \"ik_smart\"\n" +"      },\n" +"      \"price\":{\n" +"        \"type\": \"integer\"\n" +"      },\n" +"      \"image\":{\n" +"        \"type\": \"keyword\",\n" +"        \"index\": false\n" +"      },\n" +"      \"category\":{\n" +"        \"type\":\"keyword\"\n" +"      },\n" +"      \"brand\":{\n" +"        \"type\": \"keyword\"\n" +"      },\n" +"      \"sold\":{\n" +"        \"type\": \"integer\"\n" +"      },\n" +"      \"commentCount\":{\n" +"        \"type\": \"integer\",\n" +"        \"index\": false\n" +"      },\n" +"      \"isAD\":{\n" +"        \"type\": \"boolean\"\n" +"      },\n" +"      \"updateTime\":{\n" +"        \"type\": \"date\"\n" +"      }\n" +"    }\n" +"  }\n" +"}";}

4.4 总结

五、JavaRestClient（文档）

5.1 使用Java客户端示例：

 示例代码：

package com.etc.search;import org.apache.http.HttpHost;import org.bouncycastle.cert.ocsp.Req;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.common.xcontent.XContentType;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;@SpringBootTest
class TripSearchApplicationTests {private RestHighLevelClient client;@Testvoid testConnection() {System.out.println("client: "+client);}private static final String DOCUMENT_TEMPLATE = "{\n" +"  \"id\":\"1\",\n" +"  \"name\":\"csh\",\n" +"  \"price\":\"11\"\n" +"}";@Testvoid testCreateDocument() throws IOException{IndexRequest request = new IndexRequest("items").id("1");request.source(DOCUMENT_TEMPLATE,XContentType.JSON);client.index(request,RequestOptions.DEFAULT);}@Testvoid testDeleteDocument() throws IOException{DeleteRequest request = new DeleteRequest("items","1");client.delete(request,RequestOptions.DEFAULT);}@Testvoid testGetDocument() throws IOException{GetRequest request = new GetRequest("items","1");GetResponse response =  client.get(request, RequestOptions.DEFAULT);String json = response.getSourceAsString();System.out.println(json);}@Testvoid testUpdateDocument() throws IOException{UpdateRequest request = new UpdateRequest("items","1");request.doc("name","csh2024","price",2024);client.update(request,RequestOptions.DEFAULT);}@BeforeEachvoid setUp(){client = new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.92.136:9200")));}@AfterEachvoid tearDown() throws IOException{if(client != null){client.close();}}}

5.2 文档批量操作

5.3 总结：

 

 六、DSL查询

6.1 DSL查询介绍

6.2 DSL查询所有 

6.3 DSL叶子查询

 match查询单个字段，需要更改key和value的值，而multi_match查询多个字段，key值无需改变。

6.4 复合查询

6.4.1 复合查询的bool查询

 6.4.2 复合查询的排序和分页

 示例：先按照销量降序排列，若销量一直则按升序排列

GET /items/_search
{"query": {"match_all": {}},"sort": [{"sold": {"order": "desc"}, "price": {"order": "asc"}}]
}

 

from的计算公式：(页数-1)*页条；

ES对from和size进行限制，两者相加不能超过10000 

 6.5 高亮

6.6 搜索总结

七、JavaRestClient （搜索）

7.1 查询所有

示例：查询所有（match_all）

package com.etc.search;import org.apache.http.HttpHost;import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;@SpringBootTest
class TripSearchApplicationTests {private RestHighLevelClient client;@Testvoid testMatchAll() throws IOException{SearchRequest request = new SearchRequest("items");request.source().query(QueryBuilders.matchAllQuery());SearchResponse response = client.search(request,RequestOptions.DEFAULT);System.out.println("response = "+ response);}@BeforeEachvoid setUp(){client = new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.92.136:9200")));}@AfterEachvoid tearDown() throws IOException{if(client != null){client.close();}}}

7.2 解析查询结果

7.3 构建查询条件

 7.3.1 构建查询条件(matchQuery,multiMatchQuery)

  7.3.2 构建查询条件(termQuery,rangeQuery) 

  7.3.3 构建查询条件(bool查询)  

   示例：使用JavaRestClient实现以下的搜索：

  • 搜索关键字为脱脂牛奶
  • 品牌为德亚
  • 价格必须低于300

代码：

package com.etc.search;import net.minidev.json.JSONUtil;
import org.apache.http.HttpHost;import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;@SpringBootTest
class TripSearchApplicationTests {private RestHighLevelClient client;@Testvoid testSearch() throws IOException{SearchRequest request = new SearchRequest("items");request.source().query(QueryBuilders.boolQuery().must(QueryBuilders.matchQuery("name","脱脂牛奶")).filter(QueryBuilders.termQuery("brand","德亚")).filter(QueryBuilders.rangeQuery("price").lt(300)));SearchResponse response = client.search(request,RequestOptions.DEFAULT);parseResponseResult(response);}private static void parseResponseResult(SearchResponse response) {//总条数SearchHits searchHits = response.getHits();long total = searchHits.getTotalHits().value;System.out.println("total = "+total);//命中条数SearchHit[] hits = searchHits.getHits();for (SearchHit hit: hits){String json = hit.getSourceAsString();System.out.println("doc = "+ json);}}@BeforeEachvoid setUp(){client = new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.92.136:9200")));}@AfterEachvoid tearDown() throws IOException{if(client != null){client.close();}}}

  7.3.4 排序和分页

7.4 高亮

高亮显示结果解析

 示例：解析结果方法

private static void parseResponseResult(SearchResponse response) {//总条数SearchHits searchHits = response.getHits();long total = searchHits.getTotalHits().value;System.out.println("total = "+total);//命中条数SearchHit[] hits = searchHits.getHits();for (SearchHit hit: hits){String json = hit.getSourceAsString();//转换为ItemDocItemDoc doc = JSONUtil.toBean(json,ItemDoc.class);//处理高亮结果Map<String, HighlightField> hfs = hit.getHighlightFields();if(hfs != null && !hfs.isEmpty()){HighlightField hf = hfs.get("name");String hfName = hf.getFragments()[0].string();doc.setName(hfName);}System.out.println("doc = "+ doc);}}

八、数据聚合

8.1 数据聚合的介绍

注意：参与聚合的字段必须是Keyword、数值、日期、布尔类型的字段 

8.2 DSL聚合

 

8.3 JavaRestClient数据聚合 

package com.etc.search;import org.apache.http.HttpHost;import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.search.aggregations.AggregationBuilders;
import org.elasticsearch.search.aggregations.Aggregations;
import org.elasticsearch.search.aggregations.bucket.terms.Terms;import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;import java.io.IOException;
import java.util.List;
import java.util.Map;@SpringBootTest
class TripSearchApplicationTests {private RestHighLevelClient client;@Testvoid testAgg() throws IOException{SearchRequest request = new SearchRequest("items");request.source().size(0);String brandAggName  = "brandAgg";request.source().aggregation(AggregationBuilders.terms(brandAggName).field("brand").size(10));SearchResponse response = client.search(request,RequestOptions.DEFAULT);Aggregations aggregations = response.getAggregations();Terms brandTerms  = aggregations.get(brandAggName);List<? extends Terms.Bucket> buckets = brandTerms.getBuckets();for (Terms.Bucket bucket :buckets){System.out.println("brand: "+bucket.getKeyAsString());System.out.println("count: "+bucket.getDocCount());}}@BeforeEachvoid setUp(){client = new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.92.136:9200")));}@AfterEachvoid tearDown() throws IOException{if(client != null){client.close();}}}


http://www.ppmy.cn/ops/45789.html

相关文章

不借助三方平台自主搭建量化回测系统 ——以海龟交易策略为例

不借助三方平台自主搭建量化回测系统 ——以海龟交易策略为例

数量技术宅团队在CSDN学院推出了量化投资系列课程 欢迎有兴趣系统学习量化投资的同学&#xff0c;点击下方链接报名&#xff1a; 量化投资速成营&#xff08;入门课程&#xff09; Python股票量化投资 Python期货量化投资 Python数字货币量化投资 C语言CTP期货交易系统开…
阅读更多...
云动态摘要 2024-05-31

云动态摘要 2024-05-31

给您带来云厂商的最新动态&#xff0c;最新产品资讯和最新优惠更新。 最新优惠与活动 [1.5折起]年中盛惠--AI分会场 腾讯云 2024-05-30 人脸核身、语音识别、文字识别、数智人、腾讯混元等热门AI产品特惠&#xff0c;1.5折起 云服务器ECS试用产品续用 阿里云 2024-04-14 云…
阅读更多...
MySQL统计字符长度：CHAR_LENGTH(str)

MySQL统计字符长度：CHAR_LENGTH(str)

对于SQL表&#xff0c;用于计算字符串中字符数的最佳函数是 CHAR_LENGTH(str)&#xff0c;它返回字符串 str 的长度。 另一个常用的函数 LENGTH(str) 在这个问题中也适用&#xff0c;因为列 content 只包含英文字符&#xff0c;没有特殊字符。否则&#xff0c;LENGTH() 可能会返…
阅读更多...
身份认证与口令攻击

身份认证与口令攻击

身份认证与口令攻击 身份认证身份认证的五种方式口令认证静态口令动态口令(一次性口令)动态口令分类 密码学认证一次性口令认证S/KEY协议改进的S/KEY协议 其于共享密钥的认证 口令行为规律和口令猜测口令规律口令猜测 口令破解操作系统口令破解Windows密码存储机制Windows密码破…
阅读更多...
C语言--toupper/tolower/isupper/islower函数介绍

C语言--toupper/tolower/isupper/islower函数介绍

介绍 toupper&#xff08;&#xff09; 是一个用于将字符转换为大写的 C/C 函数&#xff0c;它位于 头文件中。使用 toupper 函数很简单&#xff0c;只需要将要转换的字符作为参数传递给它即可。 同理&#xff1a;tolower&#xff08;&#xff09;是将一个字符转换为小写的函数…
阅读更多...
C++之类的三种继承修饰符（public、private、protected）总结

C++之类的三种继承修饰符（public、private、protected）总结

1、前言 前文博客介绍了修饰符public、private、protected在类中成员变量和函数访问权限限制的总结&#xff0c;主要如下&#xff1a; public&#xff08;公有&#xff09;: 公有成员在类的内部和外部都可以被访问。 private&#xff08;私有&#xff09;: 私有成员只能在类的内…
阅读更多...
3D效果轮播图

3D效果轮播图

3D效果轮播图 代码 <!DOCTYPE html> <html lang"en"><head><meta charset"UTF-8"><meta name"viewport" content"widthdevice-width, initial-scale1.0"><title>Document</title><sty…
阅读更多...
数据结构的快速排序（c语言版）

数据结构的快速排序（c语言版）

一.快速排序的概念 1.快排的基本概念 快速排序是一种常用的排序算法,它是基于分治策略的一种高效排序算法。它的基本思想如下: 从数列中挑出一个元素作为基准(pivot)。将所有小于基准值的元素放在基准前面,所有大于基准值的元素放在基准后面。这个过程称为分区(partition)操作…
阅读更多...
最新文章

Copyright @ 2022~2023