> ## Documentation Index
> Fetch the complete documentation index at: https://java.agentscope.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Model Integration

This guide introduces the LLM models supported by AgentScope Java and how to configure them.

## Supported Models

| Provider  | Class                | Streaming | Tools | Vision | Reasoning |
| --------- | -------------------- | --------- | ----- | ------ | --------- |
| DashScope | `DashScopeChatModel` | ✅         | ✅     | ✅      | ✅         |
| OpenAI    | `OpenAIChatModel`    | ✅         | ✅     | ✅      | ✅         |
| Anthropic | `AnthropicChatModel` | ✅         | ✅     | ✅      | ✅         |
| Gemini    | `GeminiChatModel`    | ✅         | ✅     | ✅      | ✅         |
| Ollama    | `OllamaChatModel`    | ✅         | ✅     | ✅      | ✅         |

> **Note**:
>
> * `OpenAIChatModel` is compatible with OpenAI API specification, works with vLLM, DeepSeek, etc.
> * `GeminiChatModel` supports both Gemini API and Vertex AI

## Getting API Keys

| Provider  | URL                                                                  | Environment Variable |
| --------- | -------------------------------------------------------------------- | -------------------- |
| DashScope | [Alibaba Cloud Bailian Console](https://bailian.console.aliyun.com/) | `DASHSCOPE_API_KEY`  |
| OpenAI    | [OpenAI Platform](https://platform.openai.com/api-keys)              | `OPENAI_API_KEY`     |
| Anthropic | [Anthropic Console](https://console.anthropic.com/settings/keys)     | `ANTHROPIC_API_KEY`  |
| Gemini    | [Google AI Studio](https://aistudio.google.com/apikey)               | `GEMINI_API_KEY`     |
| DeepSeek  | [DeepSeek Platform](https://platform.deepseek.com/api_keys)          | -                    |

## ModelRegistry

[`ModelRegistry`](https://github.com/agentscope-ai/agentscope-java/blob/main/agentscope-core/src/main/java/io/agentscope/core/model/ModelRegistry.java) (`io.agentscope.core.model.ModelRegistry`) resolves a `Model` from a **string id**, so you do not have to call each vendor’s `*ChatModel.builder()` for simple setups. With Harness, use `HarnessAgent.builder().model(String)`; anywhere else that needs a `Model`, call `ModelRegistry.resolve(...)` and pass the result into `ReActAgent` or other builders.

### API summary

| Method                                                | Description                                                                                                                                         |
| ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `register(String name, Model model)`                  | Registers a **named** model; `resolve(name)` returns that instance.                                                                                 |
| `registerFactory(String regex, ModelFactory factory)` | Registers a custom factory for ids matching the regex; **later** registrations take precedence over earlier user factories and over built-in rules. |
| `resolve(String modelId)`                             | Returns a `Model`; throws `IllegalArgumentException` if the id cannot be resolved or creation fails.                                                |
| `canResolve(String modelId)`                          | Returns whether the id can be resolved (does not create a model).                                                                                   |
| `reset()`                                             | Clears named registrations, user factories, and the resolve cache; built-in rules stay. Intended for tests or in-process reset.                     |

`ModelFactory` is a functional interface: `Model create(String modelId)` with the full id string.

### Built-in id formats and environment variables

With the right environment variables set, you can use these id forms (with `resolve` or `HarnessAgent.Builder.model(String)`, for example):

| Example id                                    | Environment variable                                              | Notes                                              |
| --------------------------------------------- | ----------------------------------------------------------------- | -------------------------------------------------- |
| `openai:gpt-4o-mini`                          | `OPENAI_API_KEY`                                                  | OpenAI-compatible HTTP model                       |
| `dashscope:qwen-max`                          | `DASHSCOPE_API_KEY`                                               | Alibaba DashScope / Bailian                        |
| Any id starting with `qwen-`, e.g. `qwen-max` | `DASHSCOPE_API_KEY`                                               | Uses the whole string as the DashScope `modelName` |
| `anthropic:claude-sonnet-4-5-20250929`        | `ANTHROPIC_API_KEY` (optional; SDK may read from the environment) | Anthropic Claude                                   |
| `gemini:gemini-2.5-flash`                     | `GEMINI_API_KEY`                                                  | Google Gemini API                                  |
| `ollama:llama3`                               | `OLLAMA_BASE_URL` (optional, default `http://localhost:11434`)    | Local Ollama                                       |

Within one process, repeated `resolve` of the **same** factory-based id returns a **cached** `Model` instance. **Named** registrations are not cached that way.

### Example: named registration (reuse a tuned model)

Build once with full control, then register under a name:

```java theme={null}
import io.agentscope.core.model.GenerateOptions;
import io.agentscope.core.model.ModelRegistry;
import io.agentscope.extensions.model.openai.OpenAIChatModel;
import io.agentscope.harness.agent.HarnessAgent;

Model tuned = OpenAIChatModel.builder()
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("gpt-4o")
        .generateOptions(GenerateOptions.builder().temperature(0.2).build())
        .build();
ModelRegistry.register("my-gpt4o", tuned);

HarnessAgent agent = HarnessAgent.builder()
        .name("demo")
        .model("my-gpt4o")
        .workspace(workspace)
        .build();
```

### Example: built-in prefix (default connection settings)

```java theme={null}
import io.agentscope.harness.agent.HarnessAgent;

HarnessAgent agent = HarnessAgent.builder()
        .name("demo")
        .model("dashscope:qwen-max")
        .workspace(workspace)
        .build();
```

### Example: custom factory

```java theme={null}
import io.agentscope.core.model.Model;
import io.agentscope.core.model.ModelRegistry;

ModelRegistry.registerFactory(
        "my-llm:.+",
        id -> myModelFactory(id.substring("my-llm:".length())));

Model m = ModelRegistry.resolve("my-llm:prod");
```

## DashScope

Alibaba Cloud LLM platform, providing Qwen series models.

```java theme={null}
DashScopeChatModel model = DashScopeChatModel.builder()
        .apiKey(System.getenv("DASHSCOPE_API_KEY"))
        .modelName("qwen3-max")
        .build();
```

### Configuration

| Option                            | Description                                                                                                             |
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| `apiKey`                          | DashScope API key                                                                                                       |
| `modelName`                       | Model name, e.g., `qwen3-max`, `qwen-vl-max`                                                                            |
| `baseUrl`                         | Custom API endpoint (optional)                                                                                          |
| `stream`                          | Enable streaming, default `true`                                                                                        |
| `enableThinking`                  | Enable thinking mode to show reasoning process                                                                          |
| `enableSearch`                    | Enable web search for real-time information                                                                             |
| `endpointType`                    | API endpoint type (default `AUTO` auto-detect), options: `TEXT` (force text API) or `MULTIMODAL` (force multimodal API) |
| `defaultOptions`                  | Default generation options (temperature, maxTokens, etc.)                                                               |
| `formatter`                       | Message formatter (default `DashScopeChatFormatter`)                                                                    |
| `nativeStructuredOutput`          | Enable native `response_format` structured output, default `false`                                                      |
| `nativeStructuredOutputWithTools` | Enable native structured output when tools are present, defaults to `nativeStructuredOutput`                            |

### Endpoint Type (endpointType)

DashScope models support both text and multimodal API endpoints. By default, the framework automatically detects the appropriate endpoint type based on the model name (e.g., `qwen-vl-*` and `qwen3.5` series automatically use the multimodal endpoint).

When auto-detection is inaccurate (e.g., using custom model names or compatible APIs), you can manually specify the endpoint type:

```java theme={null}
// Force multimodal API (suitable for scenarios with images, audio, etc.)
DashScopeChatModel model = DashScopeChatModel.builder()
        .apiKey(System.getenv("DASHSCOPE_API_KEY"))
        .modelName("custom-model")
        .endpointType(EndpointType.MULTIMODAL)
        .build();

// Force text API
DashScopeChatModel model = DashScopeChatModel.builder()
        .apiKey(System.getenv("DASHSCOPE_API_KEY"))
        .modelName("custom-model")
        .endpointType(EndpointType.TEXT)
        .build();
```

### Thinking Mode

```java theme={null}
DashScopeChatModel model = DashScopeChatModel.builder()
        .apiKey(System.getenv("DASHSCOPE_API_KEY"))
        .modelName("qwen3-max")
        .enableThinking(true)  // Enables thinking mode and automatically enables streaming
        .defaultOptions(GenerateOptions.builder()
                .thinkingBudget(5000)  // Token budget for thinking
                .build())
        .build();


OllamaChatModel model =
        OllamaChatModel.builder()
                .modelName("qwen3-max")
                .baseUrl("http://localhost:11434")
                .defaultOptions(OllamaOptions.builder()
                        .thinkOption(ThinkOption.ThinkBoolean.ENABLED)
                        .temperature(0.8)
                        .build())
                .build();

```

## OpenAI

OpenAI models and compatible APIs.

```java theme={null}
OpenAIChatModel model = OpenAIChatModel.builder()
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("gpt-4o")
        .build();
```

### Compatible APIs

For DeepSeek, vLLM, and other compatible providers. Note that you need to configure the appropriate Formatter and structured output capabilities:

```java theme={null}
OpenAIChatModel model = OpenAIChatModel.builder()
        .apiKey("your-api-key")
        .modelName("deepseek-chat")
        .baseUrl("https://api.deepseek.com")
        .formatter(new DeepSeekFormatter())
        .nativeStructuredOutput(false)
        .nativeStructuredOutputWithTools(false)
        .build();
```

### Configuration

| Option                            | Description                                                                                                                                                |
| --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `apiKey`                          | API key                                                                                                                                                    |
| `modelName`                       | Model name, e.g., `gpt-4o`, `gpt-4o-mini`                                                                                                                  |
| `baseUrl`                         | Custom API endpoint (optional)                                                                                                                             |
| `endpointPath`                    | Custom request path (optional), e.g., `/v4/chat/completions`                                                                                               |
| `stream`                          | Enable streaming, default `true`                                                                                                                           |
| `generateOptions`                 | Default generation options (note: OpenAI uses `.generateOptions()` instead of `.defaultOptions()`)                                                         |
| `formatter`                       | Message formatter (default `OpenAIChatFormatter`). Compatible providers need their own Formatter (e.g., `DeepSeekFormatter`, `GLMFormatter`)               |
| `nativeStructuredOutput`          | Enable native `response_format` structured output, default `true`. Unsupported providers (DeepSeek, vLLM, etc.) should set to `false`                      |
| `nativeStructuredOutputWithTools` | Enable native structured output when tools are present, default `true`. Some providers prioritize `response_format` over tool calls, should set to `false` |
| `contextWindowSize`               | Override context window size (optional, auto-inferred from model name by default)                                                                          |
| `proxy`                           | Proxy configuration (optional), e.g., `ProxyConfig.http("localhost", 8080)`                                                                                |

## Anthropic

Anthropic's Claude series models.

```java theme={null}
AnthropicChatModel model = AnthropicChatModel.builder()
        .apiKey(System.getenv("ANTHROPIC_API_KEY"))
        .modelName("claude-sonnet-4-5-20250929")  // Default
        .build();
```

### Configuration

| Option      | Description                                      |
| ----------- | ------------------------------------------------ |
| `apiKey`    | Anthropic API key                                |
| `modelName` | Model name, default `claude-sonnet-4-5-20250929` |
| `baseUrl`   | Custom API endpoint (optional)                   |
| `stream`    | Enable streaming, default `true`                 |

## Gemini

Google's Gemini series models, supporting both Gemini API and Vertex AI.

### Gemini API

```java theme={null}
GeminiChatModel model = GeminiChatModel.builder()
        .apiKey(System.getenv("GEMINI_API_KEY"))
        .modelName("gemini-2.5-flash")  // Default
        .baseUrl("https://your-gateway.example")  // Optional
        .build();
```

### Vertex AI

```java theme={null}
GeminiChatModel model = GeminiChatModel.builder()
        .modelName("gemini-2.0-flash")
        .project("your-gcp-project")
        .location("us-central1")
        .vertexAI(true)
        .credentials(GoogleCredentials.getApplicationDefault())
        .build();
```

### Configuration

| Option          | Description                            |
| --------------- | -------------------------------------- |
| `apiKey`        | Gemini API key                         |
| `baseUrl`       | Custom Gemini API endpoint (optional)  |
| `modelName`     | Model name, default `gemini-2.5-flash` |
| `project`       | GCP project ID (Vertex AI)             |
| `location`      | GCP region (Vertex AI)                 |
| `vertexAI`      | Whether to use Vertex AI               |
| `credentials`   | GCP credentials (Vertex AI)            |
| `streamEnabled` | Enable streaming, default `true`       |

For endpoint override, use `baseUrl(...)`. For more advanced transport or proxy setup, continue to use `httpOptions(...)` or `clientOptions(...)`.

## Ollama

Self-hosted open-source LLM platform supporting various models.

```java theme={null}
OllamaChatModel model = OllamaChatModel.builder()
        .modelName("qwen3-max")
        .baseUrl("http://localhost:11434")  // Default
        .build();
```

### Configuration

| Option           | Description                                                         |
| ---------------- | ------------------------------------------------------------------- |
| `modelName`      | Model name, e.g., `qwen3-max`,`llama3.2`, `mistral`, `phi3`         |
| `baseUrl`        | Ollama server endpoint (optional, default `http://localhost:11434`) |
| `defaultOptions` | Default generation options                                          |
| `formatter`      | Message formatter (optional)                                        |
| `httpTransport`  | HTTP transport configuration (optional)                             |

### Advanced Configuration

For advanced model loading and generation parameters:

```java theme={null}
OllamaOptions options = OllamaOptions.builder()
        .numCtx(4096)           // Context window size
        .temperature(0.7)       // Generation randomness
        .topK(40)               // Top-K sampling
        .topP(0.9)              // Nucleus sampling
        .repeatPenalty(1.1)     // Repetition penalty
        .build();

OllamaChatModel model = OllamaChatModel.builder()
        .modelName("qwen3-max")
        .baseUrl("http://localhost:11434")
        .defaultOptions(options)
        .build();
```

### GenerateOptions Support

Ollama also supports `GenerateOptions` for standard configuration:

```java theme={null}
GenerateOptions options = GenerateOptions.builder()
        .temperature(0.7)           // Maps to Ollama's temperature
        .topP(0.9)                  // Maps to Ollama's top_p
        .topK(40)                   // Maps to Ollama's top_k
        .maxTokens(2000)            // Maps to Ollama's num_predict
        .seed(42L)                  // Maps to Ollama's seed
        .frequencyPenalty(0.5)      // Maps to Ollama's frequency_penalty
        .presencePenalty(0.5)       // Maps to Ollama's presence_penalty
        .additionalBodyParam(OllamaOptions.ParamKey.NUM_CTX.getKey(), 4096)      // Context window size
        .additionalBodyParam(OllamaOptions.ParamKey.NUM_GPU.getKey(), -1)        // Offload all layers to GPU
        .additionalBodyParam(OllamaOptions.ParamKey.REPEAT_PENALTY.getKey(), 1.1) // Repetition penalty
        .additionalBodyParam(OllamaOptions.ParamKey.MAIN_GPU.getKey(), 0)        // Main GPU index
        .additionalBodyParam(OllamaOptions.ParamKey.LOW_VRAM.getKey(), false)    // Low VRAM mode
        .additionalBodyParam(OllamaOptions.ParamKey.F16_KV.getKey(), true)       // 16-bit KV cache
        .additionalBodyParam(OllamaOptions.ParamKey.NUM_THREAD.getKey(), 8)      // Number of CPU threads
        .build();

OllamaChatModel model = OllamaChatModel.builder()
        .modelName("qwen3-max")
        .baseUrl("http://localhost:11434")
        .defaultOptions(OllamaOptions.fromGenerateOptions(options))  // Will be converted to OllamaOptions internally
        .build();
```

### Available Parameters

Ollama supports over 40 parameters for fine-tuning:

#### Model Loading Parameters

* `numCtx`: Context window size (default: 2048)
* `numBatch`: Batch size for prompt processing (default: 512)
* `numGPU`: Number of layers to offload to GPU (-1 for all)
* `lowVRAM`: Enable low VRAM mode for limited GPU memory
* `useMMap`: Use memory mapping for model loading
* `useMLock`: Lock model in memory to prevent swapping

#### Generation Parameters

* `temperature`: Generation randomness (0.0-2.0)
* `topK`: Top-K sampling (standard: 40)
* `topP`: Nucleus sampling (standard: 0.9)
* `minP`: Minimum probability threshold (default: 0.0)
* `numPredict`: Max tokens to generate (-1 for infinite)
* `repeatPenalty`: Penalty for repetitions (default: 1.1)
* `presencePenalty`: Penalty based on token presence
* `frequencyPenalty`: Penalty based on token frequency
* `seed`: Random seed for reproducible results
* `stop`: Strings that stop generation immediately

#### Sampling Strategies

* `mirostat`: Mirostat sampling (0=disabled, 1=Mirostat v1, 2=Mirostat v2)
* `mirostatTau`: Target entropy for Mirostat (default: 5.0)
* `mirostatEta`: Learning rate for Mirostat (default: 0.1)
* `tfsZ`: Tail-free sampling (default: 1.0 disables)
* `typicalP`: Typical probability sampling (default: 1.0)

## Generation Options

Configure generation parameters with `GenerateOptions`:

```java theme={null}
GenerateOptions options = GenerateOptions.builder()
        .temperature(0.7)           // Randomness (0.0-2.0)
        .topP(0.9)                  // Nucleus sampling
        .topK(40)                   // Top-K sampling
        .maxTokens(2000)            // Maximum output tokens
        .seed(42L)                  // Random seed
        .toolChoice(new ToolChoice.Auto())  // Tool choice strategy
        .build();

DashScopeChatModel model = DashScopeChatModel.builder()
        .apiKey(System.getenv("DASHSCOPE_API_KEY"))
        .modelName("qwen3-max")
        .defaultOptions(options)
        .build();

OllamaChatModel model = OllamaChatModel.builder()
        .modelName("qwen3-max")
        .baseUrl("http://localhost:11434")
        .defaultOptions(OllamaOptions.fromGenerateOptions(options))
        .build();
```

### Parameters

| Parameter             | Type       | Description                                            |
| --------------------- | ---------- | ------------------------------------------------------ |
| `temperature`         | Double     | Controls randomness, 0.0-2.0                           |
| `topP`                | Double     | Nucleus sampling threshold, 0.0-1.0                    |
| `topK`                | Integer    | Limits candidate tokens                                |
| `maxTokens`           | Integer    | Maximum tokens to generate                             |
| `maxCompletionTokens` | Integer    | Maximum completion tokens                              |
| `thinkingBudget`      | Integer    | Token budget for thinking                              |
| `reasoningEffort`     | String     | Reasoning effort level (e.g., `low`, `medium`, `high`) |
| `frequencyPenalty`    | Double     | Frequency penalty, -2.0-2.0                            |
| `presencePenalty`     | Double     | Presence penalty, -2.0-2.0                             |
| `seed`                | Long       | Random seed                                            |
| `toolChoice`          | ToolChoice | Tool choice strategy                                   |

### Tool Choice Strategy

```java theme={null}
new ToolChoice.Auto()              // Model decides (default)
new ToolChoice.None()              // Disable tool calling
new ToolChoice.Required()          // Force tool calling
new ToolChoice.Specific("tool_name")  // Force specific tool
```

### Additional Parameters

Support for provider-specific parameters:

```java theme={null}
GenerateOptions options = GenerateOptions.builder()
        .additionalHeader("X-Custom-Header", "value")
        .additionalBodyParam("custom_param", "value")
        .additionalQueryParam("version", "v2")
        .build();
```

## Timeout and Retry

```java theme={null}
ExecutionConfig execConfig = ExecutionConfig.builder()
        .timeout(Duration.ofMinutes(2))
        .maxAttempts(3)
        .initialBackoff(Duration.ofSeconds(1))
        .maxBackoff(Duration.ofSeconds(10))
        .backoffMultiplier(2.0)
        .build();

GenerateOptions options = GenerateOptions.builder()
        .executionConfig(execConfig)
        .build();
```

## Formatter

Formatter converts AgentScope's unified message format to each LLM provider's API format. Each provider has two types of Formatter:

| Provider  | Single-Agent             | Multi-Agent                    |
| --------- | ------------------------ | ------------------------------ |
| DashScope | `DashScopeChatFormatter` | `DashScopeMultiAgentFormatter` |
| OpenAI    | `OpenAIChatFormatter`    | `OpenAIMultiAgentFormatter`    |
| Anthropic | `AnthropicChatFormatter` | `AnthropicMultiAgentFormatter` |
| Gemini    | `GeminiChatFormatter`    | `GeminiMultiAgentFormatter`    |
| Ollama    | `OllamaChatFormatter`    | `OllamaMultiAgentFormatter`    |

### Default Behavior

When no Formatter is specified, the model uses the corresponding `ChatFormatter`, suitable for single-agent scenarios.

### Multi-Agent Scenarios

In multi-agent collaboration (such as Pipeline, MsgHub), use `MultiAgentFormatter`. It will:

* Merge messages from multiple agents into conversation history
* Use `<history></history>` tags to structure historical messages
* Distinguish between current agent and other agents' messages

```java theme={null}
// DashScope multi-agent
DashScopeChatModel model = DashScopeChatModel.builder()
        .apiKey(System.getenv("DASHSCOPE_API_KEY"))
        .modelName("qwen3-max")
        .formatter(new DashScopeMultiAgentFormatter())
        .build();

// OpenAI multi-agent
OpenAIChatModel model = OpenAIChatModel.builder()
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("gpt-4o")
        .formatter(new OpenAIMultiAgentFormatter())
        .build();

// Anthropic multi-agent
AnthropicChatModel model = AnthropicChatModel.builder()
        .apiKey(System.getenv("ANTHROPIC_API_KEY"))
        .formatter(new AnthropicMultiAgentFormatter())
        .build();

// Gemini multi-agent
GeminiChatModel model = GeminiChatModel.builder()
        .apiKey(System.getenv("GEMINI_API_KEY"))
        .formatter(new GeminiMultiAgentFormatter())
        .build();

// Ollama multi-agent
OllamaChatModel model = OllamaChatModel.builder()
        .modelName("qwen3-max")
        .formatter(new OllamaMultiAgentFormatter())
        .build();
```

### Custom History Prompt

You can customize the conversation history prompt:

```java theme={null}
String customPrompt = "# Conversation Record\nBelow is the previous conversation:\n";

DashScopeChatModel model = DashScopeChatModel.builder()
        .apiKey(System.getenv("DASHSCOPE_API_KEY"))
        .modelName("qwen3-max")
        .formatter(new DashScopeMultiAgentFormatter(customPrompt))
        .build();
```

### When to Use MultiAgentFormatter

| Scenario                      | Recommended Formatter     |
| ----------------------------- | ------------------------- |
| Single-agent conversation     | `ChatFormatter` (default) |
| Pipeline sequential execution | `MultiAgentFormatter`     |
| MsgHub group chat             | `MultiAgentFormatter`     |
| Multi-agent debate            | `MultiAgentFormatter`     |