openai agent第二弹:deepresearch原理介绍

news/2025/2/6 12:57:34/

文章目录

  • 技术原理
  • 类似开源项目
    • OpenDeepResearcher
    • open-deep-research
    • ollama-deep-researcher
    • smolagents的open_deep_research
  • 参考资料

2月2日openai上线了第二个agent: deep research,具体功能类似24年11月google gemini发布的deep research。

技术原理

deep research 使用端到端强化学习,训练模型在不同领域推理和复杂浏览任务的能力;这种方法的核心原则是,模型学会自主规划和执行多步骤过程以找到相关数据,包括基于实时信息进行回溯和适应的能力。此过程允许模型处理诸如浏览用户上传的文件、生成和细化图形以及引用网络来源等任务。

类似开源项目

OpenDeepResearcher

开源地址:https://github.com/mshumer/OpenDeepResearcher

该项目侧重于用asyncio和aiohttp进行异步编程和请求响应,以此项目为例,具体进行深度研究的流程如下:

  1. 根据用户输入的研究主题,生成多个相关的query:
async def generate_search_queries_async(session, user_query):"""Ask the LLM to produce up to four precise search queries (in Python list format)based on the user’s query."""prompt = ("You are an expert research assistant. Given the user's query, generate up to four distinct, ""precise search queries that would help gather comprehensive information on the topic. ""Return only a Python list of strings, for example: ['query1', 'query2', 'query3'].")messages = [{"role": "system", "content": "You are a helpful and precise research assistant."},{"role": "user", "content": f"User Query: {user_query}\n\n{prompt}"}]response = await call_openrouter_async(session, messages)if response:try:# Expect exactly a Python list (e.g., "['query1', 'query2']")search_queries = eval(response)if isinstance(search_queries, list):return search_querieselse:print("LLM did not return a list. Response:", response)return []except Exception as e:print("Error parsing search queries:", e, "\nResponse:", response)return []return []
  1. 根据多个query,异步式调用搜索引擎API,获取相关网页的url或文本text;

async def perform_search_async(session, query):"""Asynchronously perform a Google search using SERPAPI for the given query.Returns a list of result URLs."""params = {"q": query,"api_key": SERPAPI_API_KEY,"engine": "google"}try:async with session.get(SERPAPI_URL, params=params) as resp:if resp.status == 200:results = await resp.json()if "organic_results" in results:links = [item.get("link") for item in results["organic_results"] if "link" in item]return linkselse:print("No organic results in SERPAPI response.")return []else:text = await resp.text()print(f"SERPAPI error: {resp.status} - {text}")return []except Exception as e:print("Error performing SERPAPI search:", e)return []
  1. 处理网页链接link:
async def process_link(session, link, user_query, search_query):"""Process a single link: fetch its content, judge its usefulness, and if useful, extract the relevant context."""print(f"Fetching content from: {link}")page_text = await fetch_webpage_text_async(session, link)if not page_text:return Noneusefulness = await is_page_useful_async(session, user_query, page_text)print(f"Page usefulness for {link}: {usefulness}")if usefulness == "Yes":context = await extract_relevant_context_async(session, user_query, search_query, page_text)if context:print(f"Extracted context from {link} (first 200 chars): {context[:200]}")return contextreturn None
  1. 使用llm as a judge,根据之前获取的内容,判断是否还需要补充新的query来查询内容;

async def get_new_search_queries_async(session, user_query, previous_search_queries, all_contexts):"""Based on the original query, the previously used search queries, and all the extracted contexts,ask the LLM whether additional search queries are needed. If yes, return a Python list of up to four queries;if the LLM thinks research is complete, it should return ""."""context_combined = "\n".join(all_contexts)prompt = ("You are an analytical research assistant. Based on the original query, the search queries performed so far, ""and the extracted contexts from webpages, determine if further research is needed. ""If further research is needed, provide up to four new search queries as a Python list (for example, ""['new query1', 'new query2']). If you believe no further research is needed, respond with exactly .""\nOutput only a Python list or the token  without any additional text.")messages = [{"role": "system", "content": "You are a systematic research planner."},{"role": "user", "content": f"User Query: {user_query}\nPrevious Search Queries: {previous_search_queries}\n\nExtracted Relevant Contexts:\n{context_combined}\n\n{prompt}"}]response = await call_openrouter_async(session, messages)if response:cleaned = response.strip()if cleaned == "":return ""try:new_queries = eval(cleaned)if isinstance(new_queries, list):return new_querieselse:print("LLM did not return a list for new search queries. Response:", response)return []except Exception as e:print("Error parsing new search queries:", e, "\nResponse:", response)return []return []
  1. 让llm根据之前搜集的资料,编写report:
async def generate_final_report_async(session, user_query, all_contexts):"""Generate the final comprehensive report using all gathered contexts."""context_combined = "\n".join(all_contexts)prompt = ("You are an expert researcher and report writer. Based on the gathered contexts below and the original query, ""write a comprehensive, well-structured, and detailed report that addresses the query thoroughly. ""Include all relevant insights and conclusions without extraneous commentary.")messages = [{"role": "system", "content": "You are a skilled report writer."},{"role": "user", "content": f"User Query: {user_query}\n\nGathered Relevant Contexts:\n{context_combined}\n\n{prompt}"}]report = await call_openrouter_async(session, messages)return report

open-deep-research

开源地址: https://github.com/nickscamara/open-deep-research

使用ts开发的一款AI应用,采用firecrawl提取和搜索网页,用微调的o3模型进行深度推理;

ollama-deep-researcher

开源地址:https://github.com/langchain-ai/ollama-deep-researcher

基于langgraph开发,使用ollama本地部署deepseek-r1的8b版本作为深度推理模型,用TAVILY或PERPLEXITY的搜索服务API;

agentsopen_deep_research_158">smolagents的open_deep_research

开源地址:https://github.com/huggingface/smolagents/tree/main/examples/open_deep_research

基于huggingface的smolagents开发的deep research agent;

参考资料

https://openai.com/index/introducing-deep-research/
https://jina.ai/
https://openrouter.ai/


http://www.ppmy.cn/news/1569800.html

相关文章

验证工具:SVN版本控制

1-SVN概念 SVN(Subversion)是一种集中式版本控制系统,它用于文件和目录的版本管理,允许多个用户协同工作,同时追踪每个文件和目录的历史修改记录。以下是关于SVN版本控制的详细介绍: 一、SVN的基本概念 仓库(Repository):SVN的仓库是一个集中存储所有文件和目录的地…

出口沙特|SASO清关程序2025年重要更新

沙特阿拉伯标准组织(SASO)于2024年10月发布通知,宣布自2025年1月1日起,所有出口至沙特的货物必须通过SABER系统提交相关文件,以获得符合性证书(PCOC和SCOC)。这意味着,未在SABER系统…

oracle 基础语法复习记录

Oracle SQL基础 学习范围 学习SQL基础语法 掌握SELECT、INSERT、UPDATE、DELETE等基本操作。 熟悉WHERE、GROUP BY、ORDER BY、HAVING等子句。 理解表连接: 学习INNER JOIN、LEFT JOIN、RIGHT JOIN、FULL OUTER JOIN等连接方式。 掌握聚合函数: 熟悉…

深度整理总结MySQL——Join的工作原理

Join的工作原理 前言Join到底能不能用连接基本概念连接的本质连接过程内连接AND外连接左(外)连接的语法右(外)连接的语法内连接的语法 连接的原理Join连接算法前置知识- OLTP && OLAPOLTPOLAP 嵌套循环连接(Nested-Loop Join)使用索引加快连接速度 基于块的嵌套循环连接…

人工智能|本地部署|ollama+chatbox快速Windows10下部署(初级篇)

一、 前言: 其实早一个月我已经使用过deepseek,并且也在自己的机器上通过ollama部署过,但一直没有太多动力,现在感觉还是的记录一下,省的自己给忘掉了 本文只是简单记录一下ollamaopen-webuichatbox部署通过网盘分享…

实际时钟(RTC)的介绍

一、RTC介绍 RTC:实时时钟(Real-Time Clock)的缩写,是一种集成电路,通常称为时钟芯片。 应用:广泛应用于消费类电子产品,提供精确的实时时间或为电子系统提供时间基准。 工作原理:采用精度较高…

解决threeJS加载obj gltf和glb模型后颜色太暗的方法

网上找到的部分解决方法 网上找到的部分解决方法 咱们有时候去glb官方下载glb或gltf模型时候,模型显示太黑 其实通过查找后不难发现网上给出了很多解决方法,但是大部分都无法从根本上解决问题。我之前看到有一篇文章对gltf的解决方法是让gltf增加自发光…

数据库开发常识(10.6)——SQL性能判断标准及索引误区(1)

10.6. 数据库开发常识 作为一名专业数据库开发人员,不但需要掌握数据库开发相关的语法和功能实现,还要掌握专业数据库开发的常识。这样,才能在保量完成工作任务的同时,也保质的完成工作任务,避免了为应用的日后维护埋…