Model

Overview

The model layer separates shared contracts from provider implementations. agentscope-core keeps the common APIs (Model, ChatModelBase, Formatter, ModelRegistry, and the ModelProvider SPI). OpenAI, DashScope, Gemini, Anthropic, and Ollama implementations live in their own model extension modules. At runtime, the model layer is two-tiered: at the top sit Credentials (based on io.agentscope.core.credential), which carry a provider’s API auth fields; below them sit Chat Models, the concrete inference implementations attached to a credential.

CredentialBase/
└── ChatModelBase/
    ├── OpenAIChatModel
    ├── AnthropicChatModel
    ├── DashScopeChatModel
    ├── GeminiChatModel
    └── OllamaChatModel

A Credential carries a provider’s API auth fields (apiKey, baseUrl, …). Starting from a credential, you can call listModels() to enumerate the models available under that provider (returns Mono<List<ModelCard>>). This layering matches the natural UX in a frontend — register the credential first, then pick a model under it — so the UI authenticates once and shows everything that provider supports.

Model extension modules

Provider-specific model implementations have been moved out of agentscope-core into independent extension modules. Each provider module owns its chat model, credential, formatter, DTO, exception, and SDK/API client, etc.

Provider	Maven artifact	Main package
OpenAI	`agentscope-extensions-model-openai`	`io.agentscope.extensions.model.openai`
DashScope	`agentscope-extensions-model-dashscope`	`io.agentscope.extensions.model.dashscope`
Gemini	`agentscope-extensions-model-gemini`	`io.agentscope.extensions.model.gemini`
Anthropic	`agentscope-extensions-model-anthropic`	`io.agentscope.extensions.model.anthropic`
Ollama	`agentscope-extensions-model-ollama`	`io.agentscope.extensions.model.ollama`

Migration checklist

Add the provider extension module dependency. For example, DashScope:

<dependency>
    <groupId>io.agentscope</groupId>
    <artifactId>agentscope-extensions-model-dashscope</artifactId>
</dependency>

Other provider artifacts follow the same pattern: agentscope-extensions-model-openai, agentscope-extensions-model-gemini, agentscope-extensions-model-anthropic, and agentscope-extensions-model-ollama.

Replace provider imports from io.agentscope.core.model.* with io.agentscope.extensions.model.<provider>.*.
Replace provider formatter imports from io.agentscope.core.formatter.<provider>.* with io.agentscope.extensions.model.<provider>.formatter.*.
For Spring Boot applications, replace the generic model creation path with the matching provider-specific starter and its agentscope.<provider>.* properties.

<dependency>
    <groupId>io.agentscope</groupId>
    <artifactId>agentscope-dashscope-spring-boot-starter</artifactId>
</dependency>

Choose a creation path

String model id

For simple non-Spring applications, use a ModelRegistry string id such as dashscope:qwen-plus, openai:gpt-4.1-mini, or deepseek:deepseek-v4-flash. Add the matching model extension module, set the provider’s standard environment variable such as DASHSCOPE_API_KEY, OPENAI_API_KEY, or DEEPSEEK_API_KEY, and pass the id directly to the agent:

ReActAgent agent =
        ReActAgent.builder()
                .name("assistant")
                .model("dashscope:qwen-plus") // resolved internally by ModelRegistry.resolve(modelId)
                .build();

The extension module is discovered through Java SPI. The model provider reads its standard environment variables such as DASHSCOPE_API_KEY, OPENAI_API_KEY, DEEPSEEK_API_KEY, GLM_API_KEY, ANTHROPIC_API_KEY, or GEMINI_API_KEY. Ollama reads OLLAMA_BASE_URL when present and otherwise defaults to the local Ollama endpoint.

Explicit model builder

When you need a custom API key, base URL, formatter, transport, timeout, generation options, or other provider-specific configuration, build the model explicitly and pass the Model instance to the agent:

import io.agentscope.extensions.model.dashscope.DashScopeChatModel;
import io.agentscope.extensions.model.dashscope.formatter.DashScopeChatFormatter;

DashScopeChatModel model =
        DashScopeChatModel.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .modelName("qwen-plus")
                .stream(true)
                .formatter(new DashScopeChatFormatter())
                .build();

ReActAgent agent =
        ReActAgent.builder()
                .name("assistant")
                .model(model)
                .build();

Spring Boot applications

For Spring Boot, prefer provider-specific starters such as agentscope-openai-spring-boot-starter, agentscope-dashscope-spring-boot-starter, agentscope-gemini-spring-boot-starter, agentscope-anthropic-spring-boot-starter, and agentscope-ollama-spring-boot-starter. These starters directly depend on the matching model extension, create Spring-managed Model beans, and leave the generic starter focused on common AgentScope infrastructure. They do not create models through the static ModelRegistry; advanced users can always provide their own Model bean. OpenAI example:

agentscope:
  model:
    provider: openai
  openai:
    api-key: ${OPENAI_API_KEY}
    model-name: gpt-4.1-mini
    stream: true

Builder customizers

Provider-specific starters also expose ordered Spring bean customizers for the auto-configured chat model builders. Use them when property binding covers the common settings but you still need to tune builder-only options such as custom formatters, default generation options, proxy/client settings, or provider-specific flags.

Starter	Customizer type
`agentscope-openai-spring-boot-starter`	`OpenAIChatModelBuilderCustomizer`
`agentscope-dashscope-spring-boot-starter`	`DashScopeChatModelBuilderCustomizer`
`agentscope-gemini-spring-boot-starter`	`GeminiChatModelBuilderCustomizer`
`agentscope-anthropic-spring-boot-starter`	`AnthropicChatModelBuilderCustomizer`
`agentscope-ollama-spring-boot-starter`	`OllamaChatModelBuilderCustomizer`

Customizer beans are applied after starter properties are bound and before builder.build() is called. Multiple customizers are supported and follow Spring’s @Order / Ordered ordering.

import io.agentscope.core.model.GenerateOptions;
import io.agentscope.spring.boot.openai.OpenAIChatModelBuilderCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.annotation.Order;

@Configuration(proxyBeanMethods = false)
class ModelCustomizerConfiguration {

    @Bean
    @Order(0)
    OpenAIChatModelBuilderCustomizer openAIModelDefaults() {
        return builder ->
                builder.defaultOptions(
                        GenerateOptions.builder()
                                .temperature(0.2)
                                .parallelToolCalls(false)
                                .build());
    }
}

ModelRegistry and ModelCreationContext

ModelRegistry is a global registry for model instance creation and lookup, supporting multiple resolution strategies. During resolution, it tries in priority order: named model instances directly registered via ModelRegistry.register(name, model), custom factories registered via registerFactory(regex, factory), and ModelProvider implementations automatically discovered from extension modules through the Java SPI mechanism. For simple scenarios, prefer a string id in the provider:model format together with the provider’s standard environment variable; for fine-grained control, use explicit model builders. ModelCreationContext is mainly for integration-layer code that must resolve models dynamically.

Advanced integration context

ModelCreationContext is for integration layers that must create models dynamically without importing a concrete provider builder, such as multi-tenant gateways, plugin systems, or framework adapters. It can pass common values such as API key, base URL, endpoint path, stream mode, and extension-defined options/components to the SPI provider:

import io.agentscope.core.model.GenerateOptions;
import io.agentscope.core.model.Model;
import io.agentscope.core.model.ModelCreationContext;
import io.agentscope.core.model.ModelRegistry;

ModelCreationContext context =
        ModelCreationContext.builder()
                .apiKey(tenantApiKey)
                .baseUrl(tenantBaseUrl)
                .stream(false)
                // Extension-defined scalar options, keyed by names the provider documents.
                .option("contextWindowSize", 128000)
                // Type-keyed components for richer provider settings, transports, or formatters.
                .component(
                        GenerateOptions.class,
                        GenerateOptions.builder()
                                .parallelToolCalls(false)
                                .build())
                .build();

Model model = ModelRegistry.resolve("openai:gpt-4.1-mini", context);

Cache policy

ModelRegistry caches models resolved from simple provider:model strings. Context-aware creation is not cached by default to avoid reusing a model instance with a different tenant’s API key, base URL, or stream setting.

Policy	Behavior
`DEFAULT`	`resolve(String)` keeps legacy model-id caching. `resolve(String, nonEmptyContext)` is not cached.
`DISABLED`	Never cache; every resolution creates a new model instance.
`ENABLED`	Cache only when the caller explicitly opts in. Use `cacheId(...)` for tenant- or configuration-specific identity.

If CachePolicy.ENABLED is used with option(...) or component(...), the user must provide a cacheId.

ModelProvider SPI

Provider extension modules are discovered with Java SPI through META-INF/services/io.agentscope.core.model.spi.ModelProvider. A provider can implement supports(String, ModelCreationContext) and create(String, ModelCreationContext) to consume context values. Simple providers can keep implementing the original supports(String) and create(String) methods because the context-aware methods have compatible defaults.

Chat model

A Chat Model is the LLM driving conversation and tool calling, with input and output potentially spanning multiple modalities. AgentScope Java currently ships:

Provider	Class	Notes
OpenAI	`OpenAIChatModel`	Chat Completions API; works with vLLM and OpenAI-compatible endpoints (DeepSeek, Kimi, …)
Anthropic	`AnthropicChatModel`	Claude models; prompt caching and thinking
DashScope	`DashScopeChatModel`	Qwen models; multi-modal (vision/audio/video), reasoning
Gemini	`GeminiChatModel`	Google Gemini; multi-modal
Ollama	`OllamaChatModel`	Locally hosted LLMs; credential optional

Provider credential classes live with their model extension modules, for example OpenAICredential, AnthropicCredential, DashScopeCredential, GeminiCredential, and OllamaCredential. OpenAI-compatible credentials such as DeepSeekCredential, KimiCredential, and XAICredential remain available from core.

Creating a chat model

Each chat model is built with a builder. The most common fields are apiKey, modelName, stream, formatter, defaultOptions. Three typical setups:

Streaming
Tools
Reasoning

import io.agentscope.extensions.model.dashscope.formatter.DashScopeChatFormatter;
import io.agentscope.extensions.model.dashscope.DashScopeChatModel;

DashScopeChatModel model =
        DashScopeChatModel.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .modelName("qwen-plus")
                .stream(true)
                .formatter(new DashScopeChatFormatter())
                .build();

import io.agentscope.extensions.model.dashscope.formatter.DashScopeChatFormatter;
import io.agentscope.extensions.model.dashscope.DashScopeChatModel;
import io.agentscope.core.model.GenerateOptions;

DashScopeChatModel model =
        DashScopeChatModel.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .modelName("qwen-plus")
                .stream(false)
                .formatter(new DashScopeChatFormatter())
                .defaultOptions(
                        GenerateOptions.builder()
                                .parallelToolCalls(false)
                                .build())
                .build();

import io.agentscope.extensions.model.dashscope.formatter.DashScopeChatFormatter;
import io.agentscope.extensions.model.dashscope.DashScopeChatModel;
import io.agentscope.core.model.GenerateOptions;

DashScopeChatModel model =
        DashScopeChatModel.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .modelName("qwen3-235b-a22b-thinking-2507")
                .stream(true)
                .enableThinking(true)
                .formatter(new DashScopeChatFormatter())
                .defaultOptions(
                        GenerateOptions.builder()
                                .thinkingBudget(2048)
                                .build())
                .build();

Common builder fields:

Field	Type	Description
`apiKey`	`String`	API key (some providers also accept `credential(...)`)
`modelName`	`String`	Model identifier (e.g. `"qwen-plus"`)
`stream`	`boolean`	Whether to stream output
`defaultOptions`	`GenerateOptions`	Provider-specific options (`temperature`, `maxTokens`, `thinkingBudget`, `parallelToolCalls`, …)
`formatter`	`Formatter`	Override the default message formatter
`baseUrl`	`String`	Custom service endpoint (e.g. an OpenAI-compatible proxy)

Calling a chat model

The Model interface exposes a unified stream(messages, tools, options) returning Flux<ChatResponse>:

import io.agentscope.core.message.UserMessage;
import io.agentscope.core.model.ChatResponse;
import io.agentscope.extensions.model.dashscope.DashScopeChatModel;
import io.agentscope.core.model.GenerateOptions;
import io.agentscope.extensions.model.dashscope.formatter.DashScopeChatFormatter;
import java.util.List;

DashScopeChatModel model =
        DashScopeChatModel.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .modelName("qwen-plus")
                .stream(true)
                .formatter(new DashScopeChatFormatter())
                .build();

model.stream(
                List.of(new UserMessage("Count from 1 to 5.")),
                /* tools = */ List.of(),
                GenerateOptions.builder().build())
        .doOnNext(chunk -> System.out.println("Chunk: " + chunk.getContent()))
        .doOnComplete(() -> System.out.println("Stream completed"))
        .blockLast();

A ChatResponse carries a list of content blocks (TextBlock, ThinkingBlock, ToolUseBlock, DataBlock) and a ChatUsage recording token counts and timing. In practice you usually call models indirectly via ReActAgent. For lightweight direct invocation, see agentscope-examples/documentation/.../model/ModelRegistryExample.java.

Generating structured output

The agent layer offers a convenience overload for binding the model output to a Java POJO via ReActAgent.call(msgs, structuredOutputClass, runtimeContext):

import io.agentscope.core.ReActAgent;
import io.agentscope.core.agent.RuntimeContext;
import io.agentscope.core.message.Msg;
import io.agentscope.core.message.UserMessage;
import java.util.List;

public class WeatherInfo {
    public String city;
    public double temperature;
    public String unit;
}

Msg msg =
        agent.call(
                        List.of(new UserMessage("What's the weather in Shanghai?")),
                        WeatherInfo.class,
                        RuntimeContext.empty())
                .block();

WeatherInfo info = msg.getStructuredData(WeatherInfo.class);

How it works: the framework synthesizes a forced structured tool call from the target class, validates and repairs the model output, and writes the result into Msg.metadata under the structured_output key, so getStructuredData(Class) can deserialize it directly. Complete example: agentscope-examples/documentation/.../structuredoutput/StructuredOutputExample.java.

Structured output path selection

The framework provides two structured output paths:

Path	Condition	Mechanism
Native	`supportsNativeStructuredOutput() = true`	Uses `response_format` + `json_schema` for direct JSON output
Fallback (default)	`supportsNativeStructuredOutput() = false`	Injects a `generate_response` synthetic tool; model returns structured data via tool call

If the native path fails (e.g. model returns HTTP 400), the framework automatically falls back to the synthetic tool path — no user intervention needed.

Default behavior per provider

Provider	`supportsNativeStructuredOutput`	Notes
OpenAI (GPT-4o, etc.)	`true`	Native `json_schema` support
OpenAI (DeepSeek/GLM formatter)	`false`	Not supported; auto-fallback
DashScope	`false`	Native endpoint only supports `json_object`, not `json_schema`; fallback by default
Anthropic	`false` (default)	—

DashScope users: Thinking mode (enableThinking(true)) does not support structured output at all — the framework forces the fallback path.

Explicit configuration

If you confirm your model/endpoint supports json_schema, enable the native path via builder:

DashScopeChatModel model = DashScopeChatModel.builder()
        .apiKey(System.getenv("DASHSCOPE_API_KEY"))
        .modelName("qwen-plus")
        .nativeStructuredOutput(true)  // explicitly enable native json_schema path
        .build();

Structured output with tool calling

When an agent has both tools and structured output, some OpenAI-compatible providers (e.g. Kimi, Deepseek) prioritise the response_format constraint and skip tool calling entirely. Set nativeStructuredOutputWithTools(false) to resolve this:

OpenAIChatModel model = OpenAIChatModel.builder()
        .apiKey("...")
        .baseUrl("https://api.moonshot.cn/v1")
        .modelName("moonshot-v1-8k")
        .nativeStructuredOutputWithTools(false)
        .build();

DashScopeChatModel supports this option as well. For native OpenAI models (GPT-4o, etc.) the default behavior handles both correctly — no configuration needed.

Formatter

A Formatter converts AgentScope Msg objects into the request payload each provider’s API expects. It is configured via the chat model builder’s formatter(...). Each provider ships two formatters:

Type	Use case
ChatFormatter (default)	Standard single-agent chat. Each `Msg` maps 1:1 to one API message, preserving the role (`USER`, `ASSISTANT`, `SYSTEM`).
MultiAgentFormatter	Multi-agent scenarios such as debate or moderator setups. Consecutive agent messages are aggregated and tagged with the sender’s name.

To switch to multi-agent mode, just pass the MultiAgent variant — no agent code changes:

import io.agentscope.extensions.model.dashscope.formatter.DashScopeMultiAgentFormatter;
import io.agentscope.extensions.model.dashscope.DashScopeChatModel;

DashScopeChatModel model =
        DashScopeChatModel.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .modelName("qwen-plus")
                .stream(true)
                .formatter(new DashScopeMultiAgentFormatter())
                .build();

Per-provider formatters now live with their provider extension modules:

Provider	Chat	MultiAgent
DashScope	`DashScopeChatFormatter`	`DashScopeMultiAgentFormatter`
OpenAI	`OpenAIChatFormatter`	`OpenAIMultiAgentFormatter`
Anthropic	`AnthropicChatFormatter`	`AnthropicMultiAgentFormatter`
Gemini	`GeminiChatFormatter`	`GeminiMultiAgentFormatter`
Ollama	`OllamaChatFormatter`	`OllamaMultiAgentFormatter`

If your provider’s payload doesn’t fit any of these, implement the Formatter<TReq, TResp, TParams> interface (io.agentscope.core.formatter) and pass it through the same formatter(...) builder.

Custom provider

The minimal path to a new provider: implement a CredentialBase subclass and a ChatModelBase subclass.

Step 1: Define the credential

Extend CredentialBase and implement getChatModelClass():

import io.agentscope.core.credential.CredentialBase;
import io.agentscope.core.model.ChatModelBase;

public class MyProviderCredential extends CredentialBase {

    private final String apiKey;
    private final String baseUrl;

    public MyProviderCredential(String apiKey, String baseUrl) {
        super("my_provider:" + apiKey.substring(0, Math.min(4, apiKey.length())));
        this.apiKey = apiKey;
        this.baseUrl = baseUrl == null ? "https://api.myprovider.com/v1" : baseUrl;
    }

    public String getApiKey() {
        return apiKey;
    }

    public String getBaseUrl() {
        return baseUrl;
    }

    @Override
    public Class<? extends ChatModelBase> getChatModelClass() {
        return MyProviderChatModel.class;
    }
}

Step 2: Implement the chat model

Extend ChatModelBase and implement doStream:

import io.agentscope.core.message.Msg;
import io.agentscope.core.model.ChatModelBase;
import io.agentscope.core.model.ChatResponse;
import io.agentscope.core.model.GenerateOptions;
import io.agentscope.core.model.ToolSchema;
import java.util.List;
import reactor.core.publisher.Flux;

public class MyProviderChatModel extends ChatModelBase {

    private final MyProviderCredential credential;
    private final String modelName;

    public MyProviderChatModel(MyProviderCredential credential, String modelName) {
        this.credential = credential;
        this.modelName = modelName;
    }

    @Override
    protected Flux<ChatResponse> doStream(
            List<Msg> messages, List<ToolSchema> tools, GenerateOptions options) {
        // Call the provider's API, wrap responses into a Flux<ChatResponse>.
        return Flux.empty();
    }
}

Step 3: Register with the ModelRegistry (optional)

ModelRegistry lets ReActAgent.builder().model("provider:model-name") resolve models from a string:

import io.agentscope.core.model.ModelRegistry;

ModelRegistry.registerFactory(
        "myprov:.*",
        modelId -> new MyProviderChatModel(
                new MyProviderCredential(System.getenv("MYPROV_API_KEY"), null),
                modelId.substring("myprov:".length())));

// Then:
// ReActAgent.builder().model("myprov:my-model-v1")...

Frontend integration

What is ModelCard

ModelCard (credential/ModelCard.java) is a declarative description of a model’s capabilities and constraints. It powers frontends — the model picker, parameter form, and capability toggles can render dynamically against it without hard-coding any provider-specific logic. Today, ModelCard is a minimal record:

Method	Type	Description
`modelName()`	`String`	Model identifier (e.g. `"claude-sonnet-4-6"`)
`displayName()`	`String`	Human-readable label (e.g. `"Claude Sonnet 4.6"`)
`contextSize()`	`Integer`	Maximum context window (in tokens)

The ModelCard schema is intentionally minimal at this stage; capability flags (input/output MIME types) and parameter schemas will be added as model-discovery infrastructure matures.

Fetching ModelCards

Call CredentialBase#listModels(), returning Mono<List<ModelCard>>:

import io.agentscope.core.credential.ModelCard;
import io.agentscope.extensions.model.anthropic.credential.AnthropicCredential;
import java.util.List;

AnthropicCredential cred = new AnthropicCredential(System.getenv("ANTHROPIC_API_KEY"));
List<ModelCard> cards = cred.listModels().block();

for (ModelCard card : cards) {
    System.out.println(
            card.modelName() + ": context=" + card.contextSize());
}

getChatModelClass() returns the matching ChatModelBase subclass — useful for reflectively building a default model:

Class<? extends io.agentscope.core.model.ChatModelBase> modelCls = cred.getChatModelClass();

This design lets frontends discover every model available under a provider with just one credential — no hard-coded provider logic.

Quick Start

Building Blocks

Harness

Reference

Overview

Model extension modules

Migration checklist

Choose a creation path

String model id

Explicit model builder

Spring Boot applications

Builder customizers

ModelRegistry and ModelCreationContext

Advanced integration context

Cache policy

ModelProvider SPI

Chat model

Creating a chat model

Calling a chat model

Generating structured output

Structured output path selection

Default behavior per provider

Explicit configuration

Structured output with tool calling

Formatter

Custom provider

Step 1: Define the credential

Step 2: Implement the chat model

Step 3: Register with the ModelRegistry (optional)

Frontend integration

What is ModelCard

Fetching ModelCards

​Overview

​Model extension modules

​Migration checklist

​Choose a creation path

​String model id

​Explicit model builder

​Spring Boot applications

​Builder customizers

​ModelRegistry and ModelCreationContext

​Advanced integration context

​Cache policy

​ModelProvider SPI

​Chat model

​Creating a chat model

​Calling a chat model

​Generating structured output

​Structured output path selection

​Default behavior per provider

​Explicit configuration

​Structured output with tool calling

​Formatter

​Custom provider

​Step 1: Define the credential

​Step 2: Implement the chat model

​Step 3: Register with the ModelRegistry (optional)

​Frontend integration

​What is ModelCard

​Fetching ModelCards

Overview

Model extension modules

Migration checklist

Choose a creation path

String model id

Explicit model builder

Spring Boot applications

Builder customizers

ModelRegistry and ModelCreationContext

Advanced integration context

Cache policy

ModelProvider SPI

Chat model

Creating a chat model

Calling a chat model

Generating structured output

Structured output path selection

Default behavior per provider

Explicit configuration

Structured output with tool calling

Formatter

Custom provider

Step 1: Define the credential

Step 2: Implement the chat model

Step 3: Register with the ModelRegistry (optional)

Frontend integration

What is ModelCard

Fetching ModelCards