英伟达基于Mistral 7B开发新一代Embedding模型——NV-Embed-v2

news/2024/11/16 20:28:47/

在这里插入图片描述

我们介绍的 NV-Embed-v2 是一种通用嵌入模型,它在大规模文本嵌入基准(MTEB 基准)(截至 2024 年 8 月 30 日)的 56 项文本嵌入任务中以 72.31 的高分排名第一。此外,它还在检索子类别中排名第一(在 15 项任务中获得 62.65 分),这对 RAG 技术的发展至关重要。

NV-Embed-v2 采用了多项新设计,包括让 LLM 关注潜在向量,以获得更好的池化嵌入输出,并展示了一种两阶段指令调整方法,以提高检索和非检索任务的准确性。此外,NV-Embed-v2 还采用了一种新颖的硬阴性挖掘方法,该方法考虑了正相关性得分,能更好地去除假阴性。

有关更多技术细节,请参阅我们的论文: NV-Embed:将 LLM 训练为通用嵌入模型的改进技术。

型号详情

  • 仅用于解码器的基本 LLM:Mistral-7B-v0.1
  • 池类型: Latent-Attention
  • 嵌入尺寸: 4096

如何使用

所需软件包

如果遇到问题,请尝试安装以下 python 软件包

pip uninstall -y transformer-engine
pip install torch==2.2.0
pip install transformers==4.42.4
pip install flash-attn==2.2.0
pip install sentence-transformers==2.7.0

以下是如何使用 Huggingface-transformer 和 Sentence-transformer 对查询和段落进行编码的示例。

HuggingFace Transformers

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel# Each query needs to be accompanied by an corresponding instruction describing the task.
task_name_to_instruct = {"example": "Given a question, retrieve passages that answer the question",}query_prefix = "Instruct: "+task_name_to_instruct["example"]+"\nQuery: "
queries = ['are judo throws allowed in wrestling?', 'how to become a radiology technician in michigan?']# No instruction needed for retrieval passages
passage_prefix = ""
passages = ["Since you're reading this, you are probably someone from a judo background or someone who is just wondering how judo techniques can be applied under wrestling rules. So without further ado, let's get to the question. Are Judo throws allowed in wrestling? Yes, judo throws are allowed in freestyle and folkstyle wrestling. You only need to be careful to follow the slam rules when executing judo throws. In wrestling, a slam is lifting and returning an opponent to the mat with unnecessary force.","Below are the basic steps to becoming a radiologic technologist in Michigan:Earn a high school diploma. As with most careers in health care, a high school education is the first step to finding entry-level employment. Taking classes in math and science, such as anatomy, biology, chemistry, physiology, and physics, can help prepare students for their college studies and future careers.Earn an associate degree. Entry-level radiologic positions typically require at least an Associate of Applied Science. Before enrolling in one of these degree programs, students should make sure it has been properly accredited by the Joint Review Committee on Education in Radiologic Technology (JRCERT).Get licensed or certified in the state of Michigan."
]# load model with tokenizer
model = AutoModel.from_pretrained('nvidia/NV-Embed-v2', trust_remote_code=True)# get the embeddings
max_length = 32768
query_embeddings = model.encode(queries, instruction=query_prefix, max_length=max_length)
passage_embeddings = model.encode(passages, instruction=passage_prefix, max_length=max_length)# normalize embeddings
query_embeddings = F.normalize(query_embeddings, p=2, dim=1)
passage_embeddings = F.normalize(passage_embeddings, p=2, dim=1)# get the embeddings with DataLoader (spliting the datasets into multiple mini-batches)
# batch_size=2
# query_embeddings = model._do_encode(queries, batch_size=batch_size, instruction=query_prefix, max_length=max_length, num_workers=32, return_numpy=True)
# passage_embeddings = model._do_encode(passages, batch_size=batch_size, instruction=passage_prefix, max_length=max_length, num_workers=32, return_numpy=True)scores = (query_embeddings @ passage_embeddings.T) * 100
print(scores.tolist())
# [[87.42693328857422, 0.46283677220344543], [0.965264618396759, 86.03721618652344]]

Sentence-Transformers

import torch
from sentence_transformers import SentenceTransformer# Each query needs to be accompanied by an corresponding instruction describing the task.
task_name_to_instruct = {"example": "Given a question, retrieve passages that answer the question",}query_prefix = "Instruct: "+task_name_to_instruct["example"]+"\nQuery: "
queries = ['are judo throws allowed in wrestling?', 'how to become a radiology technician in michigan?']# No instruction needed for retrieval passages
passages = ["Since you're reading this, you are probably someone from a judo background or someone who is just wondering how judo techniques can be applied under wrestling rules. So without further ado, let's get to the question. Are Judo throws allowed in wrestling? Yes, judo throws are allowed in freestyle and folkstyle wrestling. You only need to be careful to follow the slam rules when executing judo throws. In wrestling, a slam is lifting and returning an opponent to the mat with unnecessary force.","Below are the basic steps to becoming a radiologic technologist in Michigan:Earn a high school diploma. As with most careers in health care, a high school education is the first step to finding entry-level employment. Taking classes in math and science, such as anatomy, biology, chemistry, physiology, and physics, can help prepare students for their college studies and future careers.Earn an associate degree. Entry-level radiologic positions typically require at least an Associate of Applied Science. Before enrolling in one of these degree programs, students should make sure it has been properly accredited by the Joint Review Committee on Education in Radiologic Technology (JRCERT).Get licensed or certified in the state of Michigan."
]# load model with tokenizer
model = SentenceTransformer('nvidia/NV-Embed-v2', trust_remote_code=True)
model.max_seq_length = 32768
model.tokenizer.padding_side="right"def add_eos(input_examples):input_examples = [input_example + model.tokenizer.eos_token for input_example in input_examples]return input_examples# get the embeddings
batch_size = 2
query_embeddings = model.encode(add_eos(queries), batch_size=batch_size, prompt=query_prefix, normalize_embeddings=True)
passage_embeddings = model.encode(add_eos(passages), batch_size=batch_size, normalize_embeddings=True)scores = (query_embeddings @ passage_embeddings.T) * 100
print(scores.tolist())

MTEB 基准的指令模板

对于检索、STS 和摘要的 MTEB 子任务,请使用 instructions.json 中的指令前缀模板。 对于分类、聚类和重排,请使用 NV-Embed 论文表 7 中提供的说明。 7 中提供的说明。

instructions.json

{"ClimateFEVER":{"query": "Given a claim about climate change, retrieve documents that support or refute the claim","corpus": ""},"HotpotQA":{"query": "Given a multi-hop question, retrieve documents that can help answer the question","corpus": ""},"FEVER":{"query": "Given a claim, retrieve documents that support or refute the claim","corpus": ""},"MSMARCO":{"query": "Given a web search query, retrieve relevant passages that answer the query","corpus": ""},"DBPedia":{"query": "Given a query, retrieve relevant entity descriptions from DBPedia","corpus": ""},"NQ":{"query": "Given a question, retrieve passages that answer the question","corpus": ""},"QuoraRetrieval":{"query": "Given a question, retrieve questions that are semantically equivalent to the given question","corpus": "Given a question, retrieve questions that are semantically equivalent to the given question"},"SCIDOCS":{"query": "Given a scientific paper title, retrieve paper abstracts that are cited by the given paper","corpus": ""},"TRECCOVID":{"query": "Given a query on COVID-19, retrieve documents that answer the query","corpus": ""},"Touche2020":{"query": "Given a question, retrieve passages that answer the question","corpus": ""},"SciFact":{"query": "Given a scientific claim, retrieve documents that support or refute the claim","corpus": ""},"NFCorpus":{"query": "Given a question, retrieve relevant documents that answer the question","corpus": ""},"ArguAna":{"query": "Given a claim, retrieve documents that support or refute the claim","corpus": ""},"FiQA2018":{"query": "Given a financial question, retrieve relevant passages that answer the query","corpus": ""},"STS":{"text": "Retrieve semantically similar text"},"SUMM":{"text": "Given a news summary, retrieve other semantically similar summaries"}
}

如何启用多 GPU(注意,这是 HuggingFace Transformers的情况)

from transformers import AutoModel
from torch.nn import DataParallelembedding_model = AutoModel.from_pretrained("nvidia/NV-Embed-v2")
for module_key, module in embedding_model._modules.items():embedding_model._modules[module_key] = DataParallel(module)

http://www.ppmy.cn/news/1547535.html

相关文章

ollama+springboot ai+vue+elementUI整合

1. 下载安装ollama (1) 官网下载地址:https://github.com/ollama/ollama 这里以window版本为主,下载链接为:https://ollama.com/download/OllamaSetup.exe。 安装完毕后,桌面小图标有一个小图标,表示已安装成功&…

【OceanBase 诊断调优】—— ocp上针对OB租户CPU消耗计算逻辑

指标介绍 租户 CPU 使用量 * 100 / 租户 CPU 分配量。 指标参数说明 指标项指标名称单位租户 CPU 消耗ob_tenant_cpu_percent% 计算表达式 sum(rate(ob_sysstat{stat_id"140013",LABELS}[INTERVAL])) by (GBLABELS) / sum(ob_sysstat{stat_id"140005"…

Wxml2Canvas小程序将dom转为图片,bug总结

1.显示文字 标签上面使用 data-type"text" 加上class名 <view data-type"text" class"my_draw_canvas"><text data-type"text" class"center my_draw_canvas" data-text"企业出游证明">企业出游证明…

JVM——类加载器、类加载器的分类

类加载器是java虚拟机提供给应用程序去 实现获取类和接口字节码数据 的技术 类加载器的分类&#xff1a; 一类是 Java代码中实现的一类是 Java虚拟机底层源代码实现的 通常可以细分为三大类&#xff1a;jdk8版本中的 java代码中的 扩展类加载器&#xff1a;Extension 允许扩…

基于Java Springboot在线教育学习系统

一、作品包含 源码数据库设计文档万字PPT全套环境和工具资源部署教程 二、项目技术 前端技术&#xff1a;Html、Css、Js、Vue、Element-ui 数据库&#xff1a;MySQL 后端技术&#xff1a;Java、Spring Boot、MyBatis 三、运行环境 开发工具&#xff1a;IDEA/eclipse 数据…

操作系统——同步

笔记内容及图片整理自XJTUSE “操作系统” 课程ppt&#xff0c;仅供学习交流使用&#xff0c;谢谢。 背景 解决有界缓冲区问题的共享内存方法在并发变量上存在竞争条件&#xff0c;即多个并发进程访问和操作同一个共享数据&#xff0c;从而其执行结果与特定访问次序有关。这种…

高级java每日一道面试题-2024年11月07日-Redis篇-Redis有哪些功能?

如果有遗漏,评论区告诉我进行补充 面试官: Redis有哪些功能? 我回答: Redis 是一个开源的、基于键值对的 NoSQL 数据库&#xff0c;以其高性能、丰富的数据结构和多种功能而闻名。在高级 Java 面试中&#xff0c;了解 Redis 的核心功能和高级特性是非常重要的。以下是 Redi…

java中设计模式的使用(持续更新中)

概述 设计模式的目的&#xff1a;编写软件过程中&#xff0c;程序员面临着来自耦合性&#xff0c;内聚性以及可维护性&#xff0c;可扩展性&#xff0c;重用性&#xff0c;灵活性等多方面的挑战&#xff0c;设计模式是为了让程序&#xff08;软件&#xff09;&#xff0c;具有…