说明
部分模型版本将于2024年11月30日下线停止服务,不再支持调用,为了避免影响您的业务运营,我们强烈建议您,核实模型版本使用情况,提前规划新模型版本升级/迁转执行计划,并于下线日之前完成业务迁移。模型处理情况如下:
类别 | 下线模型 | 处理方案 | 下线时间 |
---|---|---|---|
rerank_model | Doubao-pro-4k-rerank | 自动切换为m3-v2-rerank | 2024年11月30日 |
本节将说明如何基于一个已创建的知识库做在线检索。
说明
/api/knowledge/collection/search 接口用于对知识库进行检索,当前会默认对原始文本加工后的知识内容进行检索。
URI | /api/knowledge/collection/search | 统一资源标识符 |
---|---|---|
请求方法 | POST | 客户端对向量数据库服务器请求的操作类型 |
请求头 | Content-Type: application/json | 请求消息类型 |
Authorization: HMAC-SHA256 *** | 鉴权 |
参数 | 子参数 | 类型 | 是否必选 | 默认值 | 参数说明 |
---|---|---|---|---|---|
name | -- | string | 否 | -- | 知识库名称
|
project | -- | string | 否 | default | 知识库所属项目 |
resource_id | -- | string | 否 | -- | 知识库唯一 id |
query | -- | string | 是 | -- | 检索文本,最大可输入长度为 8000:
|
limit | -- | int | 否 | 10 | 检索结果数量
|
query_param | json | 否 | 检索的过滤和返回设置 | ||
doc_filter | map | 否 | -- | 检索过滤条件,支持对 doc 的 meta 信息过滤
| |
rerank_switch | -- | bool | 否 | False | 自动对结果做 rerank |
retrieve_count | -- | int | 否 | 25 | 进入重排的切片数量,默认为 25 |
dense_weight | -- | float | 否 | 0.5 | 混合检索中稠密向量的权重 |
endpoint_id | -- | string | 否 | -- | rerank 私有接入点 |
rerank_model | -- | string | 否 | "Doubao-pro-4k-rerank" | rerank 模型选择
Doubao-pro-4k-rerank模型将于2024年11月30日下线停止服务,无法再新建ep,原ep将自动切换为m3-v2-rerank
|
rerank_only_chunk | -- | bool | 否 | False | 是否仅根据 chunk 内容计算重排分数
|
get_attachment_link | -- | bool | 否 | False | 是否获取 attachment 的临时下载链接 |
参数 | 参数说明 |
---|---|
code | 状态码 |
message | 返回信息 |
request_id | 标识每个请求的唯一标识符 |
data | 检索返回内容 |
data 返回值
字段 | 子字段 | 字段类型 | 说明 |
---|---|---|---|
collection_name | -- | string | 检索知识库名字 |
count | -- | int | 检索返回的结果数量 |
result_list | list | 检索返回切片列表 | |
id | string | 索引的 primary_key | |
content | string | 切片内容,当文档类型为 faq 时,content为问题对应的答案 | |
score | float | 检索召回得分 | |
point_id | string | 知识点 id | |
chunk_title | string | 切片标题 | |
chunk_id | int | 切片 id,代表切片在对应文档里的位序,不同文档可能重复 | |
process_time | int | 知识处理完成时间 | |
rerank_score | float | rerank 得分,仅当 "rerank_switch" == True时出现 | |
doc_info | object |
| |
recall_position | int | 召回位次,整数,从 1 开始,仅当 "rerank_switch" == True时出现 | |
chunk_type | string | 切片所属类型, "text", "table", "image" 等 | |
table_chunk_fields | list | 结构化数据检索返回单行全量数据 | |
chunk_attachment | list | 检索召回附件(原始图片等)的临时下载链接,chunk_type 为 image 时有效 | |
generate_answer | -- | string | 生成回答,"stream" == True 时为空 |
usage | -- | string | 大模型输入输出消耗 token 量,"stream" == True 时为空 |
prompt | -- | string | 组装系统 prompt 和检索到的文本片后最终调用 LLM 时的完整 prompt |
chunk_attachment 返回值
字段 | 字段类型 | 说明 |
---|---|---|
link | string | type 为 image 时表示图片的临时下载链接,有效期 10 分钟 |
type | string | image 等 |
状态码 | http状态码 | 返回信息 | 状态码说明 |
---|---|---|---|
0 | 200 | success | 成功 |
1000001 | 401 | unauthorized | 缺乏鉴权信息 |
1000002 | 403 | no permission | 权限不足 |
1000003 | 400 | invalid request:%s | 非法参数 |
1000005 | 400 | collection not exist | collection不存在 |
curl -i -X POST \ -H 'Content-Type: application/json' \ -H 'Authorization: HMAC-SHA256 ***' \ https://api-knowledgebase.mlp.cn-beijing.volces.com/api/knowledge/collection/search \ -d '{ "name": "test_name", "project": ", "query": "introduce a new document level structure", "retrieve_count": 25, "limit": 2, "query_param": { "doc_filter": { "op": "must", "field": "doc_id", "conds": ["tos_doc_id_123", "tos_doc_id_456"] }, }, "rerank_switch": true, "dense_weight": 0.5 }'
执行成功返回:
HTTP/1.1 200 OK Content-Length: 43 Content-Type: application/json { "code":0, "data": { "collection_name": "test_name", "count": 2 "result_list": [ { "id": "tos_doc_id_123", "chunk_title": "Conclusion", "content": "In this paper, we discussed the task of document level structure parsing. This task is more intricate compared to the traditional page level scenario. This complexity arises because we need to consider connecting paragraphs across pages and linking paragraphs into sections. To address these challenges, we introduced a transition-based parser as a solution. Alongside this, we introduced a new dataset called DocTree to support this task.", "score":0.7119365930557251, "recall_position": 1 "point_id": "tos_doc_id_2_1-217-6834848478902922598", "process_time": 1709097567, "rerank_score": 0.877777 "rerank_position": 1 "doc_info": { "doc_id": "tos_doc_id_123", "doc_name": "DLSP: A Document Level Structure Parser for Multi-Page Digital Documents.pdf", "create_time": 1677561567, "doc_type": "pdf", "doc_meta": "[{"field_name": "author", "field_type": "string", "field_value": "Mike"}, {"field_name": "category", "field_type": "string", "field_value": "Mike"}]", "source": "tos", "title":"DLSP: A Document Level Structure Parser for Multi-Page Digital Documents" } }, { "id": "tos_doc_id_456", "chunk_title": "Conclusion", "content": "We also introduce a new document level structure parsing dataset called DocTree. It comprises 1,298 manually annotated documents with document level structural information. In contrast to previous datasets focusing on single page, the maximum page number in DocTree reaches 85 while the average is 7.2.", "score":0.711473822593689 "recall_position": 1 "point_id": "tos_doc_id_2_1-37-3242137170643999406", "process_time": 1709097567, "rerank_score": 0.5874546 "rerank_position": 2 "doc_info": { "doc_id": "tos_doc_id_123", "doc_name": "DLSP: A Document Level Structure Parser for Multi-Page Digital Documents.pdf", "create_time": 1677561593, "doc_type": "pdf", "doc_meta": "[{"field_name": "author", "field_type": "string", "field_value": "Mike"}, {"field_name": "category", "field_type": "string", "field_value": "Mike"}]", "source": "tos", "title":"DLSP: A Document Level Structure Parser for Multi-Page Digital Documents" } } ] }, "message": "success", "request_id": "02170910041086600000000000000000000ffff0a00609d26d25e" }
执行失败返回:
HTTP/1.1 400 OK Content-Length: 43 Content-Type: application/json {"code":1000003, "message":"invalid request:%s", "request_id": "021695029757920fd001de6666600000000000000000002569b8f"}