概要
前回 Codex 版を紹介しましたが今回は Gemini CLI での方法を紹介します
環境
- Ubuntu 24.04
- nodejs 22.21.0
- gemini-cli 0.51.0
インストール
-
npm install -g @google/gemini-cli
構成概要
- gemini-cli -> python proxy (docker) -> litellm (docker) -> custom endpoint
という流れになっています
python proxy は nginx などでも代用できるケースはありますが今回はリクエストを削る必要があるため python (fastapi) を使っています
理由は custom endpoint が gemini に完全に互換しているわけではなく使える機能により受け付けないパラメータがあるためそれを削るために用意しています
なので custom endpoint が完全互換であれば litellm だけで OK です
compose.yaml
gemini-proxy が python 製のプロキシになります
fastapi で gemini-cli から来たリクエストを受け使えないパラメータを削った後 litellm にリクエストします
services:
litellm:
image: docker.litellm.ai/berriai/litellm:latest
container_name: litellm
environment:
- AI_SERVICE_API_KEY=${AI_SERVICE_API_KEY}
ports:
- "4000:4000"
volumes:
- ./litellm_config.yaml:/app/config.yaml
restart: unless-stopped
command: ["--config", "/app/config.yaml"]
gemini-proxy:
# Python proxy for gemini CLI (native Gemini API format).
# Replaces nginx because request body JSON transformation is required:
# strips unsupported 'id' field from function_call/function_response,
# rewrites :streamGenerateContent?alt=sse → :streamGenerateContentSse,
# and injects AI_SERVICE_API_KEY as x-goog-api-key header.
# Configure gemini CLI to use port 4001 instead of 4000.
build:
context: .
dockerfile: Dockerfile.proxy
container_name: gemini-proxy
environment:
- AI_SERVICE_API_KEY=${AI_SERVICE_API_KEY}
ports:
- "4001:4001"
restart: unless-stopped
proxy.py
API を互換させるためのプロキシです
先ほども紹介しましたが custom endpoint が完全互換の場合は不要です
#!/usr/bin/env python3
"""
Proxy for gemini CLI → Your custom Gemini endpoint.
Handles:
- URL mapping: :streamGenerateContent?alt=sse → :streamGenerateContentSse
- Request body: strip unsupported 'id' field from function_call / function_response
- Auth: replace incoming key with AI_SERVICE_API_KEY as x-goog-api-key header
"""
import json
import os
import httpx
from fastapi import FastAPI, Request
from fastapi.responses import Response, StreamingResponse
CUSTOM_BASE = os.environ.get(
"CUSTOM_BASE",
"https://your-gemini-custom-endpoint",
)
API_KEY = os.environ["AI_SERVICE_API_KEY"]
app = FastAPI()
def strip_unsupported_fields(body: dict) -> dict:
"""Strip 'id' from function_call/function_response — field not supported by this endpoint."""
for content in body.get("contents", []):
for part in content.get("parts", []):
for key in ("functionCall", "function_call"):
if isinstance(part.get(key), dict):
part[key].pop("id", None)
for key in ("functionResponse", "function_response"):
if isinstance(part.get(key), dict):
part[key].pop("id", None)
return body
@app.post("/v1beta/models/{model}:{method}")
async def proxy(request: Request, model: str, method: str):
is_streaming = method == "streamGenerateContent"
# Custom endpoint uses a dedicated SSE endpoint instead of ?alt=sse query param
target_method = "streamGenerateContentSse" if is_streaming else method
target_url = f"{CUSTOM_BASE}/{model}:{target_method}"
body_bytes = await request.body()
try:
body = strip_unsupported_fields(json.loads(body_bytes))
body_bytes = json.dumps(body, ensure_ascii=False).encode()
except (json.JSONDecodeError, AttributeError):
pass
upstream_headers = {"Content-Type": "application/json", "x-goog-api-key": API_KEY}
if is_streaming:
async def generate():
async with httpx.AsyncClient(timeout=120.0) as client:
async with client.stream(
"POST", target_url, content=body_bytes, headers=upstream_headers
) as resp:
async for chunk in resp.aiter_bytes():
yield chunk
return StreamingResponse(
generate(),
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
else:
async with httpx.AsyncClient(timeout=120.0) as client:
resp = await client.post(
target_url, content=body_bytes, headers=upstream_headers
)
return Response(
content=resp.content,
status_code=resp.status_code,
media_type=resp.headers.get("content-type", "application/json"),
)
Dockerfile.proxy
先程の proxy.py をビルドしてイメージを作成する dockerfile です
FROM python:3.12-slim
WORKDIR /app
RUN pip install --no-cache-dir fastapi httpx uvicorn
COPY proxy.py .
CMD ["uvicorn", "proxy:app", "--host", "0.0.0.0", "--port", "4001"]
litellm_config.yaml
gemini-3.1-flash-lite 側は基本不要ですが gemini-cli がデフォルトで gemini-3.1-flash-lite のモデルの定義を探しに行くのでエラーにならないように追加しています
pass_through_endpoints でパスのオーバライドをしていますがこれも状況によるので必須ではないです
model_list:
- model_name: gemini-3.5-flash
litellm_params:
# gemini/ prefix is required to use API key auth instead of Vertex AI ADC.
# Model name must match model_name above to avoid LiteLLM response model override error
# that corrupts streaming responses. drop_params: true handles unsupported fields.
model: gemini/gemini-3.5-flash
api_base: https://your-gemini-custom-endpoint:generateContent
api_key: os.environ/AI_SERVICE_API_KEY
max_tokens: 4096
temperature: 0.2
rpm: 30
tpm: 60000
- model_name: gemini-3.1-flash-lite
litellm_params:
# gemini/ prefix is required to use API key auth instead of Vertex AI ADC.
model: gemini/gemini-3.5-flash
api_base: https://your-gemini-custom-endpoint:generateContent
api_key: os.environ/AI_SERVICE_API_KEY
litellm_settings:
drop_params: true
general_settings:
global_max_parallel_requests: 1
max_parallel_requests: 1
# Pass-through: forward native Gemini API requests (/v1beta/models/...) directly
# to the upstream endpoint, bypassing LiteLLM's model routing and SDK layer.
# This avoids the "cannot override response model" bug caused by GenerateContentResponse
# not having a 'model' attribute in LiteLLM's google_genai provider path.
pass_through_endpoints:
- path: "/v1beta/models/{path:path}"
target: "https://your-gemini-custom-endpoint/gemini/{path}"
headers:
x-goog-api-key: os.environ/AI_SERVICE_API_KEY
router_settings:
num_retries: 0
retry_after: 20
allowed_fails: 1
cooldown_time: 180
timeout: 120
stream_timeout: 120
server_settings:
port: 4000
動作確認
- docker compose up -d
-
export GOOGLE_GEMINI_BASE_URL="http://localhost:4001"
-
gemini --model=gemini-3.5-flash
で「hi」とか打ってレスポンスが返ってくればとりあえず接続はできています
最後に
gemini-cli + litellm + custom endpoint の設定方法を紹介しました
独自のエンドポイントが提供している gemini などがあり API が完全互換していない場合は自分でリクエストやレスポンスを細工する必要があります
今回は fastapi + litellm で対応しましたが方法は様々なので自分に合う方法で互換させれば OK です
基本はエラーに合わせてプロキシや litellm 側で調整するしかないです
{
"error": {
"code": 400,
"message": "Invalid JSON payload received. Unknown name \"id\" at 'contents[2].parts[0].function_call': Cannot find field.\nInvalid JSON payload received. Unknown name \"id\" at 'contents[3].parts[0].function_response': Cannot find field.",
"status": "INVALID_ARGUMENT",
"details": [
{
"@type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"field": "contents[2].parts[0].function_call",
"description": "Invalid JSON payload received. Unknown name \"id\" at 'contents[2].parts[0].function_call': Cannot find field."
},
{
"field": "contents[3].parts[0].function_response",
"description": "Invalid JSON payload received. Unknown name \"id\" at 'contents[3].parts[0].function_response': Cannot find field."
}
]
}
]
}
}
