Function calling 是一种将大模型与外部工具和API相连的新功能，助力大模型向实际产业落地迈进。Function calling 允许开发者更可靠的从模型中获得结构化数据，无需用户输入复杂的Prompt。

在使用时，您可以用自然语言向模型描述一组 function 的功能和定义； 在对话过程中，当大模型觉得需要使用某函数时会智能地选择该 function，并返回调用所需参数，来满足用户的特定需求; 其他情况，大模型不会返回 function，而是继续对话。大模型不会直接调用 function，而是返回其对应的入参，您可自行调用该函数/API接口。

简单来说，function calling 是介于自然语言和信息接口的「翻译官」

• 把自然语言翻译成所需使用函数、参数，返回给大模型调用方
• 大模型调用方执行函数后把结果返回给大模型，大模型能总结为自然语言，或继续规划子任务

示例： 北京天气怎么样？把结果发给alan

========== round 1 ==========
user提问：北京天气怎么样？把结果发给alan
[发起请求1]
assistant 思考：根据FunctionList，先调用WeatherPlugin插件获得天气。 
assistant 返回：WeatherPlugin, {"city":"北京"}

========== round 2 ==========
user调用WeatherPlugin函数，传入参数，得到结果：6月12日，以晴为主，16°C~30°C， 降水概率：0%；湿度：27%；风速：21 公里/时
结果作为role=tool回填到req.messages中
[发起请求2]
assistant 思考：第1步已经获得北京天气，现在需要调用SendMessage插件发送消息给alan 
assistant 返回：SendMessage, {"receiver":"alan", "content": "北京晴，微风"}

========== round 3 ==========
user调用SendMessage函数，得到结果：发送成功
结果作为role=tool回填到req.messages中
[发起请求3]
assistant 总结：已告知alan今天北京晴，微风

其中，user调用xxxxx函数 由客户自行完成， 用户只需要把函数执行结果回填到messages内。
时序图如下：

这种能力带来了潜在风险。建议调用者在执行任何操作之前，严谨地确认和校验模型的生成结果。

拥有function calling能力的模型列表如下：

预先准备

Golang

可通过Github链接直接下载：https://github.com/volcengine/volcengine-go-sdk

Python

请按照如下方式安装最新SDK

pip install 'volcengine-python-sdk[ark]'

Java

请使用版本

<dependency>
  <groupId>com.volcengine</groupId>
  <artifactId>volcengine-java-sdk-ark-runtime</artifactId>
  <version>LATEST</version>
</dependency>

使用Function calling 的基本步骤如下：

1. 在模型请求基础上，新增 tools 字段调用模型，类似OpenAI。
   1. 参数类型支持json类型， 支持enum。 支持array、object等嵌套数据格式
2. 模型可以选择调用一个 function；这样的话，返回的内容是一个符合规格说明的 JSON 对象（注意：模型可能生成无效的 JSON 或虚构参数）。
3. 用户将返回的参数解析为 JSON，并调用对应 function。
4. 将 function 的响应(或报错）作为新的 message（role=tool) 告知模型，并让模型总结结果并回复给用户。

单轮function call代码示例

Golang sdk

package main
import (
        "context"
        "encoding/json"
        "fmt"
        "os"
        "strings"
        "github.com/volcengine/volcengine-go-sdk/service/arkruntime"
        "github.com/volcengine/volcengine-go-sdk/service/arkruntime/model"
        "github.com/volcengine/volcengine-go-sdk/volcengine"
)
func main() {
        client := arkruntime.NewClientWithApiKey(
                os.Getenv("ARK_API_KEY"),
                arkruntime.WithBaseUrl("https://ark.cn-beijing.volces.com/api/v3"),
                arkruntime.WithRegion("cn-beijing"),
        )
        ctx := context.Background()
        // Step 1: send the conversation and available functions to the model
        req := model.CreateChatCompletionRequest{
                Model: "ep-20240603XXXX-XXXX",
                Messages: []*model.ChatCompletionMessage{
                        {
                                Role: model.ChatMessageRoleSystem,
                                Content: &model.ChatCompletionMessageContent{
                                        StringValue: volcengine.String("你是豆包，是由字节跳动开发的 AI 人工智能助手"),
                                },
                        },
                        {
                                Role: model.ChatMessageRoleUser,
                                Content: &model.ChatCompletionMessageContent{
                                        StringValue: volcengine.String("上海的天气怎么样？"),
                                },
                        },
                },
                Tools: []*model.Tool{
                        {
                                Type: model.ToolTypeFunction,
                                Function: &model.FunctionDefinition{
                                        Name:        "get_current_weather",
                                        Description: "Get the current weather in a given location",
                                        Parameters: map[string]interface{}{
                                                "type": "object",
                                                "properties": map[string]interface{}{
                                                        "location": map[string]interface{}{
                                                                "type":        "string",
                                                                "description": "The city and state, e.g. Beijing",
                                                        },
                                                        "unit": map[string]interface{}{
                                                                "type": "string",
                                                                "description": "枚举值有celsius、fahrenheit",  
                                                        },
                                                },
                                                "required": []string{
                                                        "location",
                                                },
                                        },
                                },
                        },
                },
        }
        resp, err := client.CreateChatCompletion(ctx, req)
        if err != nil {
                fmt.Printf("chat error: %v\n", err)
                return
        }
        // extend conversation with assistant's reply
        req.Messages = append(req.Messages, &resp.Choices[0].Message)
        // Step 2: check if the model wanted to call a function.
        // The model can choose to call one or more functions; if so,
        // the content will be a stringified JSON object adhering to
        // your custom schema (note: the model may hallucinate parameters).
        for _, toolCall := range resp.Choices[0].Message.ToolCalls {
                fmt.Println("calling function")
                fmt.Println("    id:", toolCall.ID)
                fmt.Println("    name:", toolCall.Function.Name)
                fmt.Println("    argument:", toolCall.Function.Arguments)
                functionResponse, err := CallAvailableFunctions(toolCall.Function.Name, toolCall.Function.Arguments)
                if err != nil {
                        functionResponse = err.Error()
                }
                // extend conversation with function response
                req.Messages = append(req.Messages,
                        &model.ChatCompletionMessage{
                                Role:       model.ChatMessageRoleTool,
                                ToolCallID: toolCall.ID,
                                Content: &model.ChatCompletionMessageContent{
                                        StringValue: &functionResponse,
                                },
                        },
                )
        }
        // get a new response from the model where it can see the function response
        secondResp, err := client.CreateChatCompletion(ctx, req)
        if err != nil {
                fmt.Printf("second chat error: %v\n", err)
                return
        }
        fmt.Println("conversation", MustMarshal(req.Messages))
        fmt.Println("new message", MustMarshal(secondResp.Choices[0].Message))
}
func CallAvailableFunctions(name, arguments string) (string, error) {
        if name == "get_current_weather" {
                params := struct {
                        Location string `json:"location"`
                        Unit     string `json:"unit"`
                }{}
                if err := json.Unmarshal([]byte(arguments), &params); err != nil {
                        return "", fmt.Errorf("failed to parse function call name=%s arguments=%s", name, arguments)
                }
                return GetCurrentWeather(params.Location, params.Unit), nil
        } else {
                return "", fmt.Errorf("got unavailable function name=%s arguments=%s", name, arguments)
        }
}
// GetCurrentWeather get the current weather in a given location.
// Example dummy function hard coded to return the same weather.
// In production, this could be your backend API or an external API
func GetCurrentWeather(location, unit string) string {
        if unit == "" {
                unit = "celsius"
        }
        switch strings.ToLower(location) {
        case "beijing":
                return `{"location": "Beijing", "temperature": "10", "unit": unit}`
        case "北京":
                return `{"location": "Beijing", "temperature": "10", "unit": unit}`
        case "shanghai":
                return `{"location": "Shanghai", "temperature": "23", "unit": unit})`
        case "上海":
                return `{"location": "Shanghai", "temperature": "23", "unit": unit})`
        default:
                return fmt.Sprintf(`{"location": %s, "temperature": "unknown"}`, location)
        }
}
func MustMarshal(v interface{}) string {
        b, _ := json.Marshal(v)
        return string(b)
}

Python sdk

"""
Usage:
pip install 'volcengine-python-sdk[ark]'
"""
from volcenginesdkarkruntime import Ark
import time
client = Ark(api_key=apikey)

def test_function_call():
    messages = [
        {
            "role": "system",
            "content": "你是豆包，是由字节跳动开发的 AI 人工智能助手",
        },
        {
            "role": "user",
            "content": "北京今天的天气",
        },
    ]
    req = {
        "model": "ep-xxxx-xx",
        "messages": messages,
        "temperature": 0.8,
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "MusicPlayer",
                    "description": """歌曲查询Plugin，当用户需要搜索某个歌手或者歌曲时使用此plugin，给定歌手，歌名等特征返回相关音乐。\n 例子1：query=想听孙燕姿的遇见， 输出{"artist":"孙燕姿","song_name":"遇见","description":""}""",
                    "parameters": {
                        "properties": {
                            "artist": {"description": "表示歌手名字", "type": "string"},
                            "description": {
                                "description": "表示描述信息",
                                "type": "string",
                            },
                            "song_name": {
                                "description": "表示歌曲名字",
                                "type": "string",
                            },
                        },
                        "required": [],
                        "type": "object",
                    },
                },
            },
            {
                "type": "function",
                "function": {
                    "name": "get_current_weather",
                    "description": "",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "location": {
                                "type": "string",
                                "description": "地理位置，比如北京市",
                            },
                            "unit": {"type": "string", "description": "枚举值 [摄氏度,华氏度]"},
                        },
                        "required": ["location"],
                    },
                },
            },
        ],
    }

    ts = time.time()
    completion = client.chat.completions.create(**req)
    if completion.choices[0].message.tool_calls:
        print(
            f"Bot [{time.time() - ts:.3f} s][Use FC]: ",
            completion.choices[0].message.tool_calls[0],
        )
        # ========== 补充函数调用的结果 =========
        req["messages"].extend(
            [
                completion.choices[0].message.dict(),
                 {
                    "role": "tool",
                    "tool_call_id": completion.choices[0].message.tool_calls[0].id,
                    "content": "北京天气晴，24~30度",  # 根据实际调用函数结果填写，最好用自然语言。
                    "name": completion.choices[0].message.tool_calls[0].function.name,
                },
            ]
        )
        # 再请求一次模型，获得总结。 如不需要，也可以省略
        ts = time.time()
        completion = client.chat.completions.create(**req)
        print(
            f"Bot [{time.time() - ts:.3f} s][FC Summary]: ",
            completion.choices[0].message.content,
        )

Java sdk

package com.volcengine.ark.runtime;


import com.volcengine.ark.runtime.model.completion.chat.*;
import com.volcengine.ark.runtime.service.ArkService;

import java.util.*;

public class FunctionCallChatCompletionsExample {
    public static void main(String[] args) {
        String apiKey = System.getenv("ARK_API_KEY");
        ArkService service = ArkService.builder().apiKey(apiKey).baseUrl("${BASE_URL}").build();

        System.out.println("\n----- function call request -----");
        final List<ChatMessage> messages = new ArrayList<>();
        final ChatMessage userMessage = ChatMessage.builder().role(ChatMessageRole.USER).content("北京今天天气如何？").build();
        messages.add(userMessage);

        final List<ChatTool> tools = Arrays.asList(
                new ChatTool(
                        "function",
                        new ChatFunction.Builder()
                                .name("get_current_weather")
                                .description("获取给定地点的天气")
                                .parameters(new Weather(
                                        "object",
                                        new HashMap<String, Object>() {{
                                            put("location", new HashMap<String, String>() {{
                                                put("type", "string");
                                                put("description", "T地点的位置信息，比如北京");
                                            }});
                                            put("unit", new HashMap<String, Object>() {{
                                                put("type", "string");
                                                put("description", "枚举值有celsius、fahrenheit");
                                               }});
                                        }},
                                        Collections.singletonList("location")
                                ))
                                .build()
                        )
        );


        ChatCompletionRequest chatCompletionRequest = ChatCompletionRequest.builder()
                .model("${YOUR_ENDPOINT_ID}")
                .messages(messages)
                .tools(tools)
                .build();

        service.createChatCompletion(chatCompletionRequest).getChoices().forEach(System.out::println);

        // shutdown service
        service.shutdownExecutor();
    }

    public static class Weather {
        public String type;
        public Map<String, Object> properties;
        public List<String> required;

        public Weather(String type, Map<String, Object> properties, List<String> required) {
            this.type = type;
            this.properties = properties;
            this.required = required;
        }

        public String getType() {
            return type;
        }

        public void setType(String type) {
            this.type = type;
        }

        public Map<String, Object> getProperties() {
            return properties;
        }

        public void setProperties(Map<String, Object> properties) {
            this.properties = properties;
        }

        public List<String> getRequired() {
            return required;
        }

        public void setRequired(List<String> required) {
            this.required = required;
        }
    }

}

cURL示例

• 请求示例

curl https://ark.cn-beijing.volces.com/api/v3/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {YOUR_API_KEY}" \
  -d '{
    "model": "ep-xxxxxxxxxx",
    "messages": [
        {
            "role": "system",
            "content": "你是豆包AI助手"
        },
        {
            "role": "user",
            "content": "上海天气怎么样？"
        }
    ],   
    "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "查询天气",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "需要查询的城市"
            },
            "unit": {
              "type": "string",
              "enum": ["摄氏度", "华氏度"]
            }
          },
          "required": ["location"]
        }
      }
    }
  ]
}'

• 响应示例

{"choices":[{"finish_reason":"tool_calls","index":0,"logprobs":null,"message":{"content":"好的，正在为您查询上海天气","role":"assistant","tool_calls":[{"function":{"arguments":"{\"location\": \"上海\", \"unit\": \"celsius\"}","name":"get_current_weather"},"id":"call_2d13sqcanleeezy62as2cshm","type":"function"}]}}],"created":1734349923,"id":"021734349921275679fc877cfa440c1efdd7643d16d14ef2bd444","model":"doubao-pro-4k-functioncall-240615","object":"chat.completion","usage":{"completion_tokens":67,"prompt_tokens":106,"total_tokens":173,"prompt_tokens_details":{"cached_tokens":0}}}

多轮function call串行编排

调用Response细节图：

代码示例

• 请求示例

"""
Usage:

1. python3 -m pip install --user volcengine
2. VOLC_ACCESSKEY=XXXXX VOLC_SECRETKEY=YYYYY python main.py
"""

import os
import re, time, json
from volcenginesdkarkruntime import Ark

client = Ark(api_key="")
ep = "ep-2024xxxx"
tool_list = [
    {
        "type": "function",
        "function": {
            "name": "MusicPlayer",
            "description": "歌曲查询Plugin，当用户需要搜索某个歌手或者歌曲时使用此plugin，给定歌手，歌名等特征返回相关音乐",
            "parameters": {
                "properties": {
                    "artist": {"description": "表示歌手名字", "type": "string"},
                    "description": {
                        "description": "表示描述信息",
                        "type": "string",
                    },
                    "song_name": {
                        "description": "表示歌曲名字",
                        "type": "string",
                    },
                },
                "required": [],
                "type": "object",
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "GetCurrentWeather",
            "description": "查询当前的天气",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "地理位置，比如北京市",
                    },
                    "unit": {"type": "string", "description":"单位类型",
                    "enum":["celsius","fahrenheit"]}, 
                },
                "required": ["location"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "LinkReaderPlugin",
            "description": "当需要解析网页内容时，调用本插件",
            "parameters": {
                "type": "object",
                "properties": {
                    "url": {
                        "type": "string",
                        "description": "需要解析网页链接,最多3个；",
                    },
                },
                "required": ["url"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "WebSearchPlugin",
            "description": "当需要搜索互联网内容时，调用本插件",
            "parameters": {
                "type": "object",
                "properties": {
                    "keywords": {
                        "type": "string",
                        "description": "需要搜索的内容",
                    },
                },
                "required": ["keywords"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "SendMessage",
            "description": "需要发送微信消息时，使用本函数",
            "parameters": {
                "type": "object",
                "properties": {
                    "receiver": {
                        "type": "string",
                        "description": "接受对象",
                    },
                    "content": {
                        "type": "string",
                        "description": "需要发送的内容",
                    },
                },
                "required": ["receiver"],
            },
        },
    },
]

def LinkReaderPlugin(argument: dict):
    import requests

    urls = argument["url"]
    resp_text = []
    # if type(urls)==list:
    for url in urls:
        resp = requests.get(f"https://r.jina.ai/{url}")
        resp_text.append(resp.text.partition("Markdown Content:")[2])
    print("===============\n".join(resp_text))
    return "===============\n".join(resp_text)

TOOL_RESPONSE = {
    "GetCurrentWeather": lambda argument: f"{argument['location'] }今天20~24度，天气：阵雨。",
    "SendMessage": lambda argument: f"成功发送微信消息至{argument['receiver']}",
    "LinkReaderPlugin": LinkReaderPlugin,
    "WebSearchPlugin": lambda argument: f"{argument['keywords'].split(' ')[0]} 是一个开源项目，创立于2022年12日7日，主要目的是方便大家使用Transformer并加速人工智能技术的发展",
}

def request(req):
    completion = client.chat.completions.create(**req)
    if completion.choices[0].message.tool_calls:
        #  FunctionCall response
        req["messages"].append(completion.choices[0].message.dict())
    else:
        # Normal response (no FunctionCall)
        req["messages"].append(
            {
                "role": completion.choices[0].message.role,
                "content": completion.choices[0].message.content,
            }
        )
    return req

def test_function_call():
    round_num = 1
    req = {
        "model": ep,
        "messages": [
            {
                "role": "user",
                "content": "先查询北京的天气，如果是晴天微信发给Alan，否则发给Peter",
            },
        ],
        "temperature": 0.8,
        "tools": tool_list,
    }
    while True:
        if req["messages"][-1]["role"] == "assistant":
            if "tool_calls" in req["messages"][-1]:
                tool_name = req["messages"][-1]["tool_calls"][0]["function"]["name"]
                req["messages"].append(
                    {
                        "role": "tool",
                        "tool_call_id": req["messages"][-1]["tool_calls"][0]["id"],
                        "content": TOOL_RESPONSE[tool_name](
                            json.loads(
                                req["messages"][-1]["tool_calls"][0]["function"][
                                    "arguments"
                                ]
                            )
                        ),  # 根据实际调用函数结果填写，最好用自然语言。
                        "name": tool_name,
                    }
                )
                pass
            else:
                query = input("human:").strip()
                req["messages"].append(
                    {
                        "role": "user",
                        "content": f"""{query}""",
                    }
                )
                
        ts = time.time()
        req = request(req)
        print("=" * 10 + f" Round {round_num} " + "=" * 10)
        print(
            f"\033[31m{req['messages'][-2]['role']}\033[0m: {req['messages'][-2]['content'][:50]+'...'}\n"
        )
        if "tool_calls" in req["messages"][-1]:
            # FC
            _flag = "[FC Response]"
            _resp = f"name={req['messages'][-1]['tool_calls'][0]['function']['name']}, args={req['messages'][-1]['tool_calls'][0]['function']['arguments']}"
        else:
            # No FunctionCall
            if len(req["messages"]) >= 3 and req["messages"][-2]["role"] == "tool":
                _flag = "[Final Answer]"
            else:
                _flag = "[Normal Response]"
            _resp = req["messages"][-1]["content"]
        round_num += 1
        print(
            f"\033[31massistant\033[0m \033[34m{_flag}\033[0m:\n{_resp} \n[elpase={time.time() - ts:.3f} s]"
        )

test_function_call()

• 响应示例

========== Round 1 ==========
user: 先查询北京的天气，如果是晴天微信发给Alan，否则发给Peter...

assistant [FC Response]:
name=GetCurrentWeather, args={"location": "\u5317\u4eac"} 
[elpase=2.607 s]
========== Round 2 ==========
tool: 北京今天20~24度，天气：阵雨。...

assistant [FC Response]:
name=SendMessage, args={"content": "\u4eca\u5929\u5317\u4eac\u7684\u5929\u6c14", "receiver": "Peter"} 
[elpase=3.492 s]
========== Round 3 ==========
tool: 成功发送微信消息至Peter...

assistant [Final Answer]:
好的，请问还有什么可以帮助您？ 
[elpase=0.659 s]

参数类型说明

"person": {
    "type": "object",
    "description": "个人特征",
    "items": {
        "age": {"type": "integer", "description": "年龄"},
        "gender": {"type": "string", "description": "性别"},
        "married": {"type": "boolean", "description": "是否已婚"}
    },
    "required": ["age"],
}

"url": {
    "type": "array",
    "description": "需要解析网页链接,最多3个",
    "items": {"type": "string"}

"matrix": {
    "type": "array",
    "description": "需要计算的二维矩阵",
    "items": {"type": "array", "items": {"type": "number"}},
},

{
    "type": "object",
    "properties": {
        "grade": {
            "description": "年级, 支持多选",
            "type": "array",
            "items": {
                "type": "string",
                "description": """枚举值有
                    "一年级",
                    "二年级",
                    "三年级",
                    "四年级",
                    "五年级",
                    "六年级"。 """,
            },
        },
        "task": {
            "description": "任务",
            "type": "string",
            "description": """枚举值有["卫生", "黑板报", "才艺", "升旗"]""",
        },
    },
    "required": ["grade", "task"],
}

如果FunctionCall效果达不到目标，可以考虑精调（SFT）。以下需求建议精调：

• 提高函数准确性（是否调用函数、调用哪个函数）
• 提高参数准确性
• 优化模型总结函数执行结果

为了最好的效果，建议精调样本与推理请求时的function list（参数、介绍）一致

多轮精调

样本格式参考：模型精调数据集格式说明
针对functioncall样本，有以下新增内容：

1. 必填tools字段，格式与推理一致
2. 可选parallel_tool_calls 字段，布尔型，默认值为True
   1. 含义与推理API参数一致，表达该样本是否为并行tool_calls样本。
   2. 如果设为false，但当前轮次label又出现了超过1个tool_calls，则会报错
3. messages内，如果assistant触发fc，会有tool_calls字段
   1. 每个function内必须包含name、arguments
   2. arguments 是合法的json dump字符串
4. messages内，会有role=tool的msg， loss_weight为0.0且不可修改

一个完整的（多轮、含并行）的FC精调样例：

{
  "messages": [
    {
      "role": "system",
      "content": "你是豆包"
    },
    {
      "role": "user",
      "content": "把北京上海的天气发给Alan"
    },
    {
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "type": "function",
          "function": {
            "name": "GetCurrentWeather",
            "arguments": "{\"location\": \"北京\"}"
          }
        },
        {
          "type": "function",
          "function": {
            "name": "GetCurrentWeather",
            "arguments": "{\"location\": \"上海\"}"
          }
        }
      ],
      "loss_weight": 1.0
    },
    {
      "role": "tool",
      "content": "北京天气晴，18~25°，微风"
    },
    {
      "role": "tool",
      "content": "上海天气阴，20~30°，大风"
    },
    {
      "role": "assistant",
      "content": "已经查到北京上海天气，需要发送给alan",
      "tool_calls": [
        {
          "type": "function",
          "function": {
            "name": "SendMessage",
            "arguments": "{\"receiver\": \"Alan\", \"content\": \"今天北京天气晴，18~25°，微风；上海天气阴，20~30°，大风\"}"
          }
        }
      ],
      "loss_weight": 1.0
    },
    {
      "role": "tool",
      "content": "发送成功"
    },
    {
      "role": "assistant",
      "content": "已经查到北京上海今日天气，发送给了Alan",
      "loss_weight": 1.0
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "MusicPlayer",
        "description": "歌曲查询Plugin，当用户需要搜索某个歌手或者歌曲时使用此plugin，给定歌手，歌名等特征返回相关音乐",
        "parameters": {
          "properties": {
            "artist": {
              "description": "表示歌手名字",
              "type": "string"
            },
            "description": {
              "description": "表示描述信息",
              "type": "string"
            },
            "song_name": {
              "description": "表示歌曲名字",
              "type": "string"
            }
          },
          "required": [],
          "type": "object"
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "GetCurrentWeather",
        "description": "查询当前的天气",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "地理位置，比如北京市"
            },
            "unit": {
              "type": "string",
              "description": "温度单位",
              "enum": [
                "celsius",
                "fahrenheit"
              ]
            }
          },
          "required": [
            "location"
          ]
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "LinkReaderPlugin",
        "description": "当需要解析网页内容时，调用本插件",
        "parameters": {
          "type": "object",
          "properties": {
            "url": {
              "type": "string",
              "description": "需要解析网页链接,最多3个；"
            }
          },
          "required": [
            "url"
          ]
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "WebSearchPlugin",
        "description": "当需要搜索互联网内容时，调用本插件",
        "parameters": {
          "type": "object",
          "properties": {
            "keywords": {
              "type": "string",
              "description": "需要搜索的内容"
            }
          },
          "required": [
            "keywords"
          ]
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "SendMessage",
        "description": "需要发送微信消息时，使用本函数",
        "parameters": {
          "type": "object",
          "properties": {
            "receiver": {
              "type": "string",
              "description": "接受对象"
            },
            "content": {
              "type": "string",
              "description": "需要发送的内容"
            }
          },
          "required": [
            "receiver"
          ]
        }
      }
    }
  ]
}

实际在jsonl文件里需要写为一行：

{"messages":[{"role":"system","content":"你是豆包"},{"role":"user","content":"把北京的天气发给Alan"},{"role":"assistant","content":"","tool_calls":[{"type":"function","function":{"name":"GetCurrentWeather","arguments":"{\"location\": \"北京\"}"}}],"loss_weight":1.0},{"role":"tool","content":"北京天气晴，18~25°，微风"},{"role":"assistant","content":"已经查到北京天气，需要发送给alan","tool_calls":[{"type":"function","function":{"name":"SendMessage","arguments":"{\"receiver\": \"Alan\", \"content\": \"今天北京天气晴，18~25°，微风\"}"}}],"loss_weight":1.0},{"role":"tool","content":"发送成功"},{"role":"assistant","content":"已经查到北京今日天气，发送给了Alan","loss_weight":1.0}],"tools":[{"type":"function","function":{"name":"MusicPlayer","description":"歌曲查询Plugin，当用户需要搜索某个歌手或者歌曲时使用此plugin，给定歌手，歌名等特征返回相关音乐","parameters":{"properties":{"artist":{"description":"表示歌手名字","type":"string"},"description":{"description":"表示描述信息","type":"string"},"song_name":{"description":"表示歌曲名字","type":"string"}},"required":[],"type":"object"}}},{"type":"function","function":{"name":"GetCurrentWeather","description":"查询当前的天气","parameters":{"type":"object","properties":{"location":{"type":"string","description":"地理位置，比如北京市"},"unit":{"type":"string","description":"温度单位","enum":["celsius","fahrenheit"]}},"required":["location"]}}},{"type":"function","function":{"name":"LinkReaderPlugin","description":"当需要解析网页内容时，调用本插件","parameters":{"type":"object","properties":{"url":{"type":"string","description":"需要解析网页链接,最多3个；"}},"required":["url"]}}},{"type":"function","function":{"name":"WebSearchPlugin","description":"当需要搜索互联网内容时，调用本插件","parameters":{"type":"object","properties":{"keywords":{"type":"string","description":"需要搜索的内容"}},"required":["keywords"]}}},{"type":"function","function":{"name":"SendMessage","description":"需要发送微信消息时，使用本函数","parameters":{"type":"object","properties":{"receiver":{"type":"string","description":"接受对象"},"content":{"type":"string","description":"需要发送的内容"}},"required":["receiver"]}}}]}

Function calling适用场景

Tool如何填写

1. Function简单清晰、互相之间无歧义无重叠，如查询天气、设置闹钟、发送邮件等
2. Function参数含义清晰、无歧义，有足够的例子（prompt）约束模型输出
   1. 歧义，比如房产业务下用户说“100”可能代表100或100万，需要prompt告知模型或sft
   2. 时间相关需要明确格式，用户说10点， 模型可能返回 “10 AM”、“10点”、“10:00” 等
   3. 参数之间有重叠混淆，则模型有可能出错。

新业务接入

1. 明确业务需求。
   1. 如果对function calling依赖较重（如流水线、插件较多）、需要串行调用等场景，建议用function calling分支模型（doubao-pro 版本中带有functioncall字样的版本）
   2. 对function calling依赖较轻、无串行调用，对模型通用能力更为依赖的业务，建议用pro即可
2. 准备评测集，在FC模型测试，看看准确率，以及业务预期上线准确率。
3. 常规调优手段：
   1. Functions的params、description、examples等字段准确填写。 System prompt中不用再重复介绍函数，（可选）可以描述在何种情况下调用某函数
   • 优化函数、参数的描述， 明确不同函数的边界情况； 避免歧义；增加示例等
   • 对模型进行sft（建议至少50条数据，模型越多、参数越多、情况越多，则所需要的数据越多），见上文【精调功能】
4. 速度优化手段：
   • 对于简单无歧义的函数或参数，适当精简输入输出内容
   • 对于system prompt、fc list固定的流量，可购买模型单元，并联系我们开启Cache
   • 对于实时性要求较高的场景，可以人工拆分为2个阶段，1阶段调用LLM做函数选择，2阶段调用LLM做该函数内的参数提取。 可测试2个阶段分别用pro/lite的表现； 该方法可能会降低效果

›

prompt最佳实践

原则： Treat LLM as a kid

1. 能不用大模型完成的任务，就不要调用大模型，尽量代码完成
2. 和任务无关的信息，避免输入，避免信息干扰；

{
   "type": "function",
    "function": {
        "name": "GPT1",
        "description": "新建日程",
     }
}

{
   "type": "function",
    "function": {
        "name": "CreateEvent",
        "description": "当需要为用户新建日程时，此工具将创建日程，并返回日程ID",
     }
}

{
    "time": {
        "type": "object",
        "description": "事件时间",
        "properties": {
            "timestamp": {
                "description": "事件时间"
            }
        }
    }
}

{
    "time": {
        "type": "string",
        "description": "事件时间",
    }
}

{
    "time": {
        "type": "object",
        "description": "事件时间",
        "properties": {
            "timestamp": {
                "description": "固定传2024-01-01即可"
            }
        }
    }
}

你正在与用户Alan沟通，你需要先查询用户ID，再通过ID创建日程……

你正在与用户Alan（ID=abc123）沟通，你可以通过ID创建日程……

可以通过ID查找用户，并获得用户的日程ID

每个用户具有唯一的用户ID；每个日程有对应的日常ID，两者独立的ID。
可以通过用户ID查找用户，并获得用户的所有日程ID

需求澄清（追问）

需求澄清（确认需求），不依赖与FC，可独立使用
System promp加入

如果用户没有提供足够的信息来调用函数，请继续提问以确保收集到了足够的信息。
在调用函数之前，你必须总结用户的描述并向用户提供总结，询问他们是否需要进行任何修改。
......

在函数的description中加入

函数参数除了提取a和b， 还应要求用户提供c、d、e、f和其他相关细节。

或

如果用户提供的信息缺少工具所需要的必填参数，你需要进一步追问让用户提供更多信息。

Function calling流式输出适配

Doubao 1.5 代模型相比之前的模型，除了性能上提升之外，还升级了Function Calling的能力。

多轮FC不符合预期排查

1. 先保留最后一轮（单轮），进行验证，是否触发fc
   1. 如果不触发，尝试sft，或联系我们解决
2. 更换更好的模型，如doubao-pro4k-0515版本
3. 删除历史轮次中的中间过程，并保证历史轮次内容无特殊符号

chat v3接口 与open AI推理请求有何差异？

1. 不支持message.name 字段，填入无效
2. 如需使用 tool_choice beta版本，可以联系我们申请
3. Parallel function calling开发中
4. 暂不支持流式返回fc，如有需要，可以联系我们申请

支持Langchain吗

请检查langchain调用细节，已知的差异见前一小节

流式输出

从Doubao-1.5系列开始支持。 历史模型不支持。

与coze的关系

合作关系，在coze专业版里，可以使用账户内方舟里的模型（含精调）

三方模型支持FC列表

name=unknown

在大模型生成的fc json的时候，有可能出现json不合法的情况，此时，模型会返回一个tool_calls

• name=unknown ，表示解析失败
• arguments=[{"name":"函数名", "parameters":{xxxx}}] 表示函数的原始输出

解法一：
尝试使用一些json修复库去解析，如 json_repair

# pip install json_repair
import json_repair
completion = completion = client.chat.completions.create(**req)
fc_dict = json_repair.loads(completion.choices[0].message.tool_calls[0].function.arguments)

解法二：
可以根据arguments内的内容，判断json在哪出错，进行PE，尝试解决该问题。

# 示例
[
{"name":"ABC","parameters":{"data":{"Column1":[1,2,3,4],"Column2":["A","B","C","D"],"Column3":[10.1,20.2,30.3,40.4]}}}]
]

排查发现是最后多了一个] 。因此，尝试在函数ABC的description 里加上注意，在}}}结尾之后只有一个] ，未解决。
继续尝试在System prompt里加上注意，在}}}结尾之后只有一个] ，解决。

{
    "role": resp.choices[0].message.role,
    "content": resp.choices[0].message.content,
    "name": resp.choices[0].message.name,
    "tool_calls": [{
        "name": resp.choices[0].message.tool_calls[0].name,
        "arguments": resp.choices[0]
        .message.tool_calls[0]
        .arguments,
    }],
},