API Documentation

Chat Completion

Request URL

POST https://api.infly.cn/v1/chat/completions

Request Header

{
    "Authorization": "Bearer <API_KEY>"
}

Request Body

Example

{
    "model": "inf-chat-v1",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant"
        },
        { "role": "user", "content": "hi!" }
    ],
    "temperature": 0.3
}

Parameter Description

Parameter	Required	Description	Type	Default Value
model	required	Model name	string	Currently supports inf-chat-v1
messages	required	List of conversation messages	List[Dict]	This is a list of structures, each element is similar to: {"role": "user", "content": "Hello"} role only supports system,user,assistant, content cannot be empty
max_tokens	optional	Maximum number of tokens generated when the conversation is completed. If the content is still not finished after generating max_tokens, the value of finish_reason is 'length', otherwise it is 'stop'	int	4096
temperature	optional	The higher the sampling temperature, the more random the output, the lower the output, the more certain, the value range is [0, 1]	float	1.0
top_p	optional	Another sampling mechanism, the smaller the topp value, the more certain the generated result, the larger the top_p value, the more diverse the generated result, the value range is [0, 1]	float	0.9
n	optional	Number of results generated per prompt, a positive integer not greater than 4	int	1
presence_penalty	optional	Presence penalty, a positive value will penalize based on whether the newly generated words appear in the text, increasing the possibility of the model discussing new topics, the value range is [-2, 2]	float	0.0
frequency_penalty	optional	Frequency penalty, a positive value will penalize based on the frequency of newly generated words in the text, reducing the possibility of the model repeating the same words verbatim, the value range is [-2, 2]	float	0.0
repetition_penalty	optional	Repetition penalty, higher values will make the model more strongly avoid repetition, the value range is (0, 2]	float	1.0
stop	optional	Stop words, when this (group) word is matched, the output will stop, this (group) word itself will not be output.	List[string]	null
stream	optional	Whether to stream the output result	bool	false
tools	optional	List of tools, for details please refer to function calling	array	null
response_format	optional	Structured output, for details please refer to structured output	object	null

Response Body

Non-Streaming

{
  "id": "chatcmpl-9mc4YqcON8S6s9oN0M2hFAHRyiMAz",
  "object": "chat.completion",
  "created": 1721373122,
  "model": "inf-chat-v1",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I assist you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 8,
    "completion_tokens": 9,
    "total_tokens": 17
  }
}

Streaming

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"role": "assistant"}}], "usage": {"completion_tokens": 0, "prompt_tokens": 0, "total_tokens": 0}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": "Hello"}}], "usage": {"completion_tokens": 1, "prompt_tokens": 10, "total_tokens": 11}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": "!"}}], "usage": {"completion_tokens": 2, "prompt_tokens": 10, "total_tokens": 12}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": " I"}}], "usage": {"completion_tokens": 3, "prompt_tokens": 10, "total_tokens": 13}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": " am"}}], "usage": {"completion_tokens": 4, "prompt_tokens": 10, "total_tokens": 14}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": " 光"}}], "usage": {"completion_tokens": 5, "prompt_tokens": 10, "total_tokens": 15}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": "语"}}], "usage": {"completion_tokens": 6, "prompt_tokens": 10, "total_tokens": 16}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": "慧"}}], "usage": {"completion_tokens": 7, "prompt_tokens": 10, "total_tokens": 17}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": "言"}}], "usage": {"completion_tokens": 8, "prompt_tokens": 10, "total_tokens": 18}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": "!"}}], "usage": {"completion_tokens": 9, "prompt_tokens": 10, "total_tokens": 19}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": " How"}}], "usage": {"completion_tokens": 10, "prompt_tokens": 10, "total_tokens": 20}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": " can"}}], "usage": {"completion_tokens": 11, "prompt_tokens": 10, "total_tokens": 21}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": " I"}}], "usage": {"completion_tokens": 12, "prompt_tokens": 10, "total_tokens": 22}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": " assist"}}], "usage": {"completion_tokens": 13, "prompt_tokens": 10, "total_tokens": 23}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": " you"}}], "usage": {"completion_tokens": 14, "prompt_tokens": 10, "total_tokens": 24}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": " today"}}], "usage": {"completion_tokens": 15, "prompt_tokens": 10, "total_tokens": 25}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": "?"}}], "usage": {"completion_tokens": 16, "prompt_tokens": 10, "total_tokens": 26}}

data:{"id": "0218957931dd46118e0c6705d5afc352", "object": "chat.completion.chunk", "created": 1724907803, "model": "inf-chat-v1", "choices": [{"index": 0, "delta": {"content": ""}, "finish_reason": "stop"}], "usage": {"completion_tokens": 17, "prompt_tokens": 10, "total_tokens": 27}}

data: [DONE]

List Models

Request URL

GET https://api.infly.cn/v1/models

Request Header

{
    "Authorization": "Bearer <API_KEY>"
}

Response Body

[
  {
    "id": "cf4fd88b-452d-417b-b161-51806a8712df",
    "name": "inf-chat-v1",
    "description": "34b大模型",
    "api_version": "v1",
    "source": "inf",
    "created_at": "2024-08-05 14:55:41"
  }
]