Skip to main content

User Guide

Basic Concepts​

Large Language Model​

INF's large language model is trained on a vast amount of data to understand natural and written language, and can generate text output based on input. The input to the model is also known as a "prompt". You can provide clear instructions and examples to enable the model to complete specific tasks. Designing prompts is essentially learning how to "train" the model. The inf-chat-v1 model can be used for various tasks including content or code generation, summarization, conversation, creative writing, etc.

Token​

The large language model processes text in units called Tokens. Tokens represent common sequences of characters. For example, a single Chinese character "龖" might be broken down into several Tokens, while a short and common phrase like "你好" might use a single Token. Roughly speaking, for a typical Chinese text, 1 Token is approximately equivalent to 1.5-2 Chinese characters. Note that for our model, the total length of input and output combined cannot exceed the model's maximum context length.

API Key​

The key used for authentication, which is your unique credential for interacting with the INF OpenAPI, can be created in our console.

Quick Start​

Prerequisites​

  • Please register an account on the INF Large Model Open Platform and complete identity verification.
  • Go to the console to create or view your API Key.

Sending Requests​

HTTP Interface Call​

curl --location 'https://api.infly.cn/v1/chat/completions' \
--header 'Authorization: Bearer $API_KEY' \
--header 'Content-Type: application/json' \
--data '{
    "messages": [
        {
            "role": "user",
            "content": "hi"
        }
    ],
    "model": "inf-chat-v1"
}'

SDK​

Our API is fully compatible with OpenAI's Python SDK with minimal configuration required.

  • Install OpenAI SDK
pip install openai
  • Example Code
from openai import OpenAI
API_BASE = "https://api.infly.cn/v1"
API_KEY = "your key"
client = OpenAI(
    api_key=API_KEY,
    base_url=API_BASE
)
completion = client.chat.completions.create(
    model="inf-chat-v1",
    messages=[{"role": "user", "content": "hi"}]
)
print(completion)