Multi-LLM Provider for Go

multi-llm-provider-go is a Go library for using hosted LLM APIs and local coding agents through a shared set of provider interfaces.

It supports two complementary ways to run a model:

• API providers call hosted models through their normal SDK or HTTP transport, including OpenAI, Anthropic, Bedrock, Vertex AI, Azure, and OpenRouter.
• Coding-agent providers run Claude Code, Codex CLI, Cursor Agent, or Pi in local tmux sessions, preserving their native tools, subscriptions, project context, and authenticated sessions.

The repository also includes llm-provider-mcp, an optional MCP server for delegating asynchronous work between coding agents. It is one way to expose the provider library—not the library's only use case.

Why This Exists

Applications should be able to choose the right model and transport for each task without rebuilding their orchestration layer.

Use a direct API when you want a conventional request/response integration, predictable infrastructure, or model-level features such as structured output, embeddings, and media generation. Use a coding-agent CLI when you want an agent that can inspect a repository, edit files, run commands, and reuse an existing local subscription. Both fit behind the same Go model abstraction.

flowchart LR
    App[Go application] --> Provider[Shared provider interfaces]
    Provider --> API[API and cloud adapters]
    Provider --> CLI[Local coding-agent adapters]
    API --> Hosted[Hosted models]
    CLI --> Tmux[tmux sessions]
    Tmux --> Agents[Claude Code / Codex / Cursor / Pi]
    MCP[Optional MCP server] --> CLI

Supported Providers

The core InitializeLLM factory returns the same llmtypes.Model interface for text and coding-agent providers:

Specialized factories expose capabilities that do not fit the text-model interface:

Gemini models are available through Vertex AI for direct API access or through Pi as a coding agent. The old Gemini CLI adapter has been removed.

Common Capabilities

Provider support varies, but the shared interfaces cover:

• Text generation and streaming
• Tool calling and structured output
• Token usage, model metadata, logging, and event emission
• Embeddings
• Image input and generation
• Video generation and conversational video editing
• Audio generation and transcription
• Music generation
• Stateful coding-agent sessions, continuation, and terminal progress

Quick Start: Go Library

Install the module:

go get github.com/manishiitg/multi-llm-provider-go@latest

The current module requires Go 1.25.12 or newer.

Initialize a provider and use the returned llmtypes.Model:

package main

import (
    "context"
    "fmt"
    "log"

    llmproviders "github.com/manishiitg/multi-llm-provider-go"
    "github.com/manishiitg/multi-llm-provider-go/llmtypes"
)

func main() {
    model, err := llmproviders.InitializeLLM(llmproviders.Config{
        Provider: llmproviders.ProviderOpenAI,
        ModelID:  "gpt-4.1-mini",
    })
    if err != nil {
        log.Fatal(err)
    }

    response, err := model.GenerateContent(
        context.Background(),
        []llmtypes.MessageContent{
            llmtypes.TextParts(llmtypes.ChatMessageTypeHuman, "Explain tmux in one sentence."),
        },
    )
    if err != nil {
        log.Fatal(err)
    }

    if len(response.Choices) == 0 {
        log.Fatal("provider returned no choices")
    }
    fmt.Println(response.Choices[0].Content)
}

Set the credential expected by the selected provider—for example, OPENAI_API_KEY for OpenAI. Credentials can also be supplied explicitly with Config.APIKeys. See the examples for streaming, tool calls, custom logging, Bedrock, and Vertex AI.

Configuration

llmproviders.Config controls provider initialization:

Common credential sources include:

See .env.example for common provider credentials. Model, endpoint, fallback, and test-specific variables are documented next to the provider adapters and tests that consume them.

Changing transports starts with changing the provider:

config.Provider = llmproviders.ProviderAnthropic  // direct API
config.Provider = llmproviders.ProviderBedrock    // cloud API
config.Provider = llmproviders.ProviderCodexCLI   // local coding agent
config.Provider = llmproviders.ProviderCursorCLI  // local coding agent

Bounded coding-agent calls can use the process working directory and a temporary session automatically. Long-lived host applications should explicitly pass CodingAgentWorkingDirOption, CodingAgentInteractiveSessionOption, and CodingAgentPersistentInteractiveOption. Provider-specific options additionally control the model, approval policy, resume ID, tools, and streaming behavior.

Coding Agents And tmux

The coding-agent adapters turn native coding CLIs into providers without reimplementing their agent loops. They use each CLI's existing login and model access, and run in a local project with that CLI's native file and shell tools.

tmux is the default transport because it supports long-lived interactive sessions, multi-turn continuation, live terminal capture, control-key input, and recovery after a caller disconnects.

Requirements for this transport:

• macOS or Linux
• tmux 3.x or newer
• At least one installed and authenticated coding CLI

The library exposes session lifecycle, resume, input, interrupt, pane capture, and cleanup helpers so a host application can manage coding agents as part of its own workflow.

Module Layout

multi-llm-provider-go/
├── providers.go                 # Provider IDs, configuration, initialization
├── provider_*.go                # Shared provider behavior and media factories
├── interfaces/                  # Logging, events, and public support contracts
├── llmtypes/                    # Messages, responses, tools, streams, options
├── pkg/adapters/                # API, cloud, media, and coding-CLI adapters
├── pkg/codingagentjob/          # Durable asynchronous job execution
├── pkg/codingagentmcp/          # Optional MCP tool surface
├── pkg/tmuxcapture/             # Terminal progress capture and cleanup
├── cmd/llm-chat/                # Local provider chat client
├── cmd/llm-test/                # Manual provider contract runner
└── cmd/llm-provider-mcp/        # Optional delegation MCP server

llmtypes.Model is the central text request/response interface. Additional interfaces cover embeddings, image generation, video generation, audio generation and transcription, and music generation.

Optional: Asynchronous Delegation Over MCP

llm-provider-mcp packages the coding-agent providers as a local stdio MCP server. A Codex or Claude Code host can queue work in another coding CLI, continue working, and retrieve the result later. Jobs are persisted in SQLite and executed in detached tmux sessions.

Install it in the project where you want delegation:

curl -fsSL https://raw.githubusercontent.com/manishiitg/llm-provider-mcp/main/scripts/install-mcp.sh | sh

The setup detects installed CLIs, registers selected hosts and targets for the current project, verifies authentication, and installs the delegation skill. The server is also published in the official MCP Registry as io.github.manishiitg/llm-provider-mcp.

It exposes five tools:

See Installation and Delegation workflow for the complete MCP workflow.

Security And Trust

• Direct API credentials remain in the host process or the provider's normal credential chain.
• Coding-agent credentials remain owned by the native CLI.
• tmux-backed agents have the local user's filesystem and process permissions; tmux is a transport, not a sandbox.
• A coding agent can modify the working tree and run commands. Review and test delegated changes before accepting them.
• LLM_PROVIDER_MCP_WORKSPACE_ROOTS can restrict directories accepted by the MCP server, but it does not create an operating-system sandbox.

Read Security and trust before enabling unattended coding-agent execution in sensitive repositories.

Testing And Coverage

The repository uses several layers of testing because hosted APIs and terminal TUIs fail in different ways:

API-provider coverage is not uniform. This inventory reflects the tests and manual commands currently present in the repository:

“Yes” does not mean every capability is covered. The API provider test contract distinguishes automated Go tests, replay/manual smoke coverage, partial coverage, and known gaps at feature level.

The coding-agent certification suite covers all four active CLI providers:

The coding-agent matrix shows the release-blocking contract areas. Broader non-P0 certification gaps remain explicitly tracked in knownCertificationGaps. These checks do not promise that every upstream CLI version behaves identically. Real tests are gated by explicit environment variables and require the relevant CLI login or provider credentials.

Run the offline suite and build the manual test client:

go test -p 1 ./...
make build
./bin/llm-test --help

Detailed, provider-by-provider coverage and real-test commands live in:

• API provider test contract
• Coding-agent tmux contract
• Structured coding-agent contract
• Testing guide

Code Quality And Secret Scanning

The project uses golangci-lint for static analysis and gitleaks for secret scanning:

make lint
make scan-secrets

Documentation

• Go provider API
• Examples
• Coding-agent providers and models
• MCP installation
• Delegation workflow
• Architecture
• Security and trust
• API provider test contract
• Coding-agent tmux contract
• Replay and manual test runner
• Roadmap

Development

make build
make build-mcp
go test -p 1 ./...
golangci-lint run --timeout=5m ./...

CI also compile-checks MCP Agent and MCP Agent Builder against the current checkout to prevent accidental public API breakage.

See CONTRIBUTING.md before opening a pull request. Report security issues using SECURITY.md, not a public issue.

License

MIT