Elasticsearch-DSL高级查询操作

news/2024/12/22 17:57:09/

一、禁用元数据和过滤数据

1、禁用元数据_source

GET product/_search
{"_source": false, "query": {"match_all": {}}
}

查询结果不显示元数据
禁用之前:

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 5,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "product","_type" : "_doc","_id" : "1","_score" : 1.0,"_source" : {"name" : "xiaomi phone","desc" : "shouji zhong de zhandouji","date" : "2021-06-01","price" : 3999,"tags" : ["xingjiabi","fashao","buka"]}},{"_index" : "product","_type" : "_doc","_id" : "2","_score" : 1.0,"_source" : {"name" : "xiaomi nfc phone","desc" : "zhichi quangongneng nfc,shouji zhong de jianjiji","date" : "2021-06-02","price" : 4999,"tags" : ["xingjiabi","fashao","gongjiaoka"]}},{"_index" : "product","_type" : "_doc","_id" : "3","_score" : 1.0,"_source" : {"name" : "nfc phone","desc" : "shouji zhong de hongzhaji","date" : "2021-06-03","price" : 2999,"tags" : ["xingjiabi","fashao","menjinka"]}},{"_index" : "product","_type" : "_doc","_id" : "4","_score" : 1.0,"_source" : {"name" : "xiaomi erji","desc" : "erji zhong de huangmenji","date" : "2021-04-15","price" : 999,"tags" : ["low","bufangshui","yinzhicha"]}},{"_index" : "product","_type" : "_doc","_id" : "5","_score" : 1.0,"_source" : {"name" : "hongmi erji","desc" : "erji zhong de kendeji 2021-06-01","date" : "2021-04-16","price" : 399,"tags" : ["lowbee","xuhangduan","zhiliangx"]}}]}
}

禁用之后:

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 5,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "product","_type" : "_doc","_id" : "1","_score" : 1.0},{"_index" : "product","_type" : "_doc","_id" : "2","_score" : 1.0},{"_index" : "product","_type" : "_doc","_id" : "3","_score" : 1.0},{"_index" : "product","_type" : "_doc","_id" : "4","_score" : 1.0},{"_index" : "product","_type" : "_doc","_id" : "5","_score" : 1.0}]}
}

2、数据源过滤器

Including：结果中返回哪些field
Excluding：结果中不要返回哪些field，不返回的field不代表不能通过该字段进行检索，因为元数据不存在不代表索引不存在

两种实现方式，
1：在创建索引的时候，mapping中配置；
这样配置映射，在查询的时候只显示name和price，不显示desc和tags

PUT product2
{"mappings": {"_source": {"includes": ["name","price"],"excludes": ["desc","tags"]}}
}

查看映射信息:GET product2/_mapping

{"product2" : {"mappings" : {"_source" : {"includes" : ["name","price"],"excludes" : ["desc","tags"]},"properties" : {"desc" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"name" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"owner" : {"properties" : {"age" : {"type" : "long"},"name" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"sex" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}}}},"price" : {"type" : "long"},"tags" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}}}}}
}

插入数据:

PUT /product2/_doc/1
{"owner":{"name":"zhangsan","sex":"男","age":18},"name": "hongmi erji","desc": "erji zhong de kendeji","price": 399,"tags": ["lowbee","xuhangduan","zhiliangx"]
}

查询数据:
GET product2/_search
可以看到查询的结果没有上面excludes的数据

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "product2","_type" : "_doc","_id" : "1","_score" : 1.0,"_source" : {"price" : 399,"name" : "hongmi erji"}}]}
}

2：在写get search查询的时候指定；
基于上面的测试数据，先DELETE product2删除索引 再重新PUT /product2/_doc/1创建索引直接自动映射。
两种写法:
1.“_source”: 直接写展示的字段,
只展示owner.name和owner.sex

GET product2/_search
{"_source": ["owner.name","owner.sex"], "query": {"match_all": {}}
}

结果:

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "product2","_type" : "_doc","_id" : "1","_score" : 1.0,"_source" : {"owner" : {"sex" : "男","name" : "zhangsan"}}}]}
}

2.source里用includes和excludes

GET product2/_search
{"_source": {"includes": ["owner.*","name"],"excludes": ["name", "desc","price"]},"query": {"match_all": {}}
}

结果:

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1,"relation" : "eq"},"max_score" : 1.0,"hits" : [{"_index" : "product2","_type" : "_doc","_id" : "1","_score" : 1.0,"_source" : {"owner" : {"sex" : "男","name" : "zhangsan","age" : 18}}}]}
}

二、query string search

1.查看索引的结构
GET product/_mapping

2.查询索引的数据 默认10条
GET product/_search

3.查询索引的数据 限制条数20条
GET /product/_search?size=20

4.查询name分词后含有nfc的数据
GET /product/_search?q=name:nfc

5.查询前20条数据并且按照价格降序排列
GET /product/_search?from=0&size=20&sort=price:desc

6.createtime的数据类型是date，不会索引，所以这里是精准匹配createtime:2020-08-19的数据
GET /product/_search?q=createtime:2020-08-19

7.查询所有text分词后的词条中包含炮这个单词的
GET /product/_search?q=炮

三、全文检索-Fulltext query

查询模板:

GET index/_search
{"query": {"match": {"field": "searchContent"}}
}

造测试数据:
put mapping 就像关系型数据库的表结构：ddl语句

PUT product
{"mappings" : {"properties" : {"createtime" : {"type" : "date"},"date" : {"type" : "date"},"desc" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}},"analyzer":"ik_max_word"},"lv" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"name" : {"type" : "text","analyzer":"ik_max_word","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"price" : {"type" : "long"},"tags" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"type" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}}}}
}

插入数据:就像关系型数据库的insert

PUT /product/_doc/1
{"name" : "小米手机","desc" :  "手机中的战斗机","price" :  3999,"lv":"旗舰机","type":"手机","createtime":"2020-10-01T08:00:00Z","tags": [ "性价比", "发烧", "不卡顿" ]
}
PUT /product/_doc/2
{"name" : "小米NFC手机","desc" :  "支持全功能NFC，手机中的滑翔机","price" :  4999,"lv":"旗舰机","type":"手机","createtime":"2020-05-21T08:00:00Z","tags": [ "性价比", "发烧", "公交卡" ]
}
PUT /product/_doc/3
{"name" : "NFC手机","desc" :  "手机中的轰炸机","price" :  2999,"lv":"高端机","type":"手机","createtime":"2020-06-20","tags": [ "性价比", "快充", "门禁卡" ]
}
PUT /product/_doc/4
{"name" : "小米耳机","desc" :  "耳机中的黄焖鸡","price" :  999,"lv":"百元机","type":"耳机","createtime":"2020-06-23","tags": [ "降噪", "防水", "蓝牙" ]
}
PUT /product/_doc/5
{"name" : "红米耳机","desc" :  "耳机中的肯德基","price" :  399,"type":"耳机","lv":"百元机","createtime":"2020-07-20","tags": [ "防火", "低音炮", "听声辨位" ]
}
PUT /product/_doc/6
{"name" : "小米手机10","desc" :  "充电贼快掉电更快，超级无敌望远镜，高刷电竞屏","price" :  "","lv":"旗舰机","type":"手机","createtime":"2020-07-27","tags": [ "120HZ刷新率", "120W快充", "120倍变焦" ]
}
PUT /product/_doc/7
{"name" : "挨炮 SE2","desc" :  "除了CPU，一无是处","price" :  "3299","lv":"旗舰机","type":"手机","createtime":"2020-07-21","tags": [ "割韭菜", "割韭菜", "割新韭菜" ]
}
PUT /product/_doc/8
{"name" : "XS Max","desc" :  "听说要出新款12手机了，终于可以换掉手中的4S了","price" :  4399,"lv":"旗舰机","type":"手机","createtime":"2020-08-19","tags": [ "5V1A", "4G全网通", "大" ]
}
PUT /product/_doc/9
{"name" : "小米电视","desc" :  "70寸性价比只选，不要一万八，要不要八千八，只要两千九百九十八","price" :  2998,"lv":"高端机","type":"耳机","createtime":"2020-08-16","tags": [ "巨馍", "家庭影院", "游戏" ]
}
PUT /product/_doc/10
{"name" : "红米电视","desc" :  "我比上边那个更划算，我也2998，我也70寸，但是我更好看","price" :  2999,"type":"电视","lv":"高端机","createtime":"2020-08-28","tags": [ "大片", "蓝光8K", "超薄" ]
}
PUT /product/_doc/11
{"name": "红米电视","desc": "我比上边那个更划算，我也2998，我也70寸，但是我更好看","price": 2998,"type": "电视","lv": "高端机","createtime": "2020-08-28","tags": ["大片","蓝光8K","超薄"]
}

在这里插入图片描述

在构造mapping映射的时候，对text类型的字段指定了"analyzer":"ik_max_word"分词器，这里用的是IK分词器，插入数据会对该字段进行分词，建立倒排索引。*“type” : “keyword”*是用来后续精准查询的时候通过field.keyword来精准匹配。

1、query->match->text类型字段
进行全文搜索，会对查询的文本进行分词。
query match 这个name会被分词 name是txt类型 会被分词 所以搜索条件被分词后会和这个查询字段的词项进行匹配 匹配到的都返回
查询条件和索引中的字段数据都会进行分词 后 进行匹配 按照score返回

GET product/_search?_source=false
{"query": {"match": {"name": "NFC手机"}}
}

query->match->text.keyword类型字段
name是text类型字段，name.keyword做为查询条件不会进行分词，直接和索引数据中的name进行匹配，id为3的数据可以查询匹配。

GET product/_search
{"query": {"match": {"name.keyword": "NFC手机"}}
}

2、query->match_all查询全部数据
默认查询返回10条，这里指定20条，禁用元数据不返回太多

GET product/_search?size=20&_source=false
{"query": {"match_all": {}}
}

3、query->multi_match 多个字段匹配
多个字段匹配 name或者desc 包含 query中的任意一个就行，name或者desc分词后的数据包含手机就返回

GET product/_search?size=20&_source=false
{"query": {"multi_match": {"query": "手机","fields": ["name","desc"]}}
}

4、query->match_phrase 短语查询
搜索与指定短语匹配的文档，保留短语中词语的相对位置。
name的分词器是ik_max_word,看下name会被分为哪些词

GET _analyze
{"analyzer": "ik_max_word","text": "小米NFC手机"
}
结果:
{"tokens" : [{"token" : "小米","start_offset" : 0,"end_offset" : 2,"type" : "CN_WORD","position" : 0},{"token" : "nfc","start_offset" : 2,"end_offset" : 5,"type" : "ENGLISH","position" : 1},{"token" : "手机","start_offset" : 5,"end_offset" : 7,"type" : "CN_WORD","position" : 2}]
}GET _analyze
{"analyzer": "ik_max_word","text": "NFC手机"
}结果:
{"tokens" : [{"token" : "nfc","start_offset" : 0,"end_offset" : 3,"type" : "ENGLISH","position" : 0},{"token" : "手机","start_offset" : 3,"end_offset" : 5,"type" : "CN_WORD","position" : 1}]
}

短语查询 索引里面name字段要有NFC手机这个短语 顺序不能颠倒，NFC手机会被分为nfc 手机
分词后能和索引字段name分词后的数据匹配到且顺序不乱 就可以做为结果展示

GET product/_search
{"query": {"match_phrase": {"name": "NFC手机"}}
}

结果:

{"took" : 5,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 2,"relation" : "eq"},"max_score" : 2.8616219,"hits" : [{"_index" : "product","_type" : "_doc","_id" : "3","_score" : 2.8616219},{"_index" : "product","_type" : "_doc","_id" : "2","_score" : 2.4492486}]}
}

5、Term 对字段进行精确匹配。

GET /my_index/_search
{"query": {  // "query"定义查询条件"term": { // "term"查询执行精确匹配"field_name": "exact_value" // "field_name"是要匹配的字段; "exact_value"是精确查询的精确值，通常用于keyword标签或其他不分析的文本字段}}
}

6、Bool 多条件组合查询
组合多个查询条件，支持must（必须）、should（至少一个）和must_not（必须不）关键字。
match支持全文检索，对查询条件分词然后匹配索引中的分词后的词项
term精准查询，不会分词检索，和非text类型或者text.keyword使用
range gte大于等于lte小于等于
minimum_should_match should默认至少满足一个，这里表示至少满足的数量自己控制

GET product/_search
{"query": {"bool": {"must": [{"match": {"name": "手机"}},{"match": {"desc": "手机"}}],"should": [{"term": {"type.keyword": {"value": "手机"}}},{"range": {"price": {"gte": 100,"lte": 300}}}],"minimum_should_match": 2,"must_not": [{"range": {"price": {"gte": 2999,"lte": 4500}}}]}}
}

filter：条件过滤查询，过滤满足条件的数据 不计算相关度得分

GET product/_search
{"query": {"bool": {"filter": [{"term": {"type.keyword": {"value": "手机"}}}]}}
}

7、terms
索引中tags含有性价比或者大片任意一个就行

GET product/_search
{"query": {"terms": {"tags.keyword": [ "性价比", "大片" ],"boost": 2.0}}
}

8、constant_score 意为固定得分
避免算分 提高性能

GET product/_search
{"query": {"constant_score": {"filter": {"term": {"type.keyword": "手机"}},"boost": 1.2}}
}

9、(must或者filter)和should组合 这时should满足0也行 如果should单用 要至少满足一个

GET product/_search
{"query": {"bool": {"filter": [{"range": {"price": {"gte": 10,"lte": 4000}}}],"should": [{"match": {"name": "哈哈哈哈哈哈哈哈哈哈哈哈"}},{"range": {"price": {"gte": 4001,"lte": 9000}}}],"minimum_should_match": 1}}
}

minimum_should_match不设置或者设置为0,即使should两个条件一个都不符合也可以查出数据


http://www.ppmy.cn/news/1557253.html

相关文章

【Qt】显示类控件：QLabel、QLCDNumber、QProgressBar、QCalendarWidget

【Qt】显示类控件：QLabel、QLCDNumber、QProgressBar、QCalendarWidget

目录 QLabel QFrame 例子： textFormat pixmap、scaledContents alignment wordWrap、indent、margin buddy QLCDNumber 例子： QTimer QProgressBar 例子： QCalendarWidget 例子： QLabel 标签控件，用来显示…
阅读更多...
故障诊断 | 一个小创新：特征提取+KAN分类

故障诊断 | 一个小创新：特征提取+KAN分类

往期精彩内容： Python-凯斯西储大学（CWRU）轴承数据解读与分类处理 基于FFT CNN - BiGRU-Attention 时域、频域特征注意力融合的轴承故障识别模型-CSDN博客 基于FFT CNN - Transformer 时域、频域特征融合的轴承故障识别模型-CSDN博客 P…
阅读更多...
使用CNN模型训练图片识别（键盘，椅子，眼镜，水杯，鼠标）

使用CNN模型训练图片识别（键盘，椅子，眼镜，水杯，鼠标）

首先是环境： 我是在Anaconda3中的Jupyter Notebook (tensorflow)中进行训练，环境各位自行安装 数据集： 本次数据集五个类型（键盘，椅子，眼镜，水杯，鼠标）我收集了每个接近两…
阅读更多...
Neo4j插入数据逐级提升速度4倍又4倍

Neo4j插入数据逐级提升速度4倍又4倍

语雀版：https://www.yuque.com/xw76/back/dtukgqfkfwg1d6yo 目录 背景介绍初始方案Node()创建事务批量提交记录Node是否存在生成Cypher语句执行数据库参数优化切换成85k个三元组测试建索引（很显著！！！）MATCH…
阅读更多...
UE5 Lyra项目源码分析-角色配置说明

UE5 Lyra项目源码分析-角色配置说明

在上一篇里，我们研究了关卡的配置如何在GameMode实现加载的，并稍微理解了Lyra是如何实现的模块化，由于模块化太彻底，所以理解起来有些难，在这一篇里，我们看一下配置里面比较想了解的一块。就是角色是如何配…
阅读更多...
阿里数据仓库-数据模型建设方法总结

阿里数据仓库-数据模型建设方法总结

一、大数据领域建模综述 1.1 为什么需要数据建模 有结构地分类组织和存储是我们面临的一个挑战。 数据模型强调从业务、数据存取和使用角度合理存储数据。 数据模型方法，以便在性能、成本、效率之间取得最佳平衡 成本：良好的数据模型能极大地减少不必要的数据冗余，也能实现…
阅读更多...
【ETCD】【源码阅读】深入分析 applierV3backend.Apply`方法源码

【ETCD】【源码阅读】深入分析 applierV3backend.Apply`方法源码

applierV3backend的Apply主要负责将 Raft 请求 (pb.InternalRaftRequest) 应用到 Etcd 的后端存储中。它处理各种不同类型的请求，并且根据请求的具体内容调用相应的处理逻辑。 版本【release 文章目录 一、完整源码二、方法详解1. 定义和初始化2. 记录操作开始时间并…
阅读更多...
Linux系统安全与应用： 筑牢防线，高效运维

Linux系统安全与应用： 筑牢防线，高效运维

在当今数字化时代，Linux系统凭借其卓越的稳定性与强大的性能，广泛应用于各类关键业务场景，从数据中心服务器到云计算平台，从网络设备到嵌入式系统，无处不在。因此，确保Linux系统安全稳定运行，成为重中之重。本文将深入解析Linux系统安全与应用的关键要点，涵盖账户安全，…
阅读更多...
最新文章

Copyright @ 2022~2023