[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6,"_7":8,"_9":10},"actionData","errors","root",{"_111":112},"routes/modules.$creator.$moduleId",{"_14":15,"_16":17,"_18":19,"_20":21},"routes/modules.$creator.$moduleId.endpoints",{"_11":12,"_13":-5},"error","Module not found","tools","module",{"_20":22,"_23":24,"_25":26,"_27":28,"_29":30,"_31":32,"_33":34,"_35":26,"_36":37,"_38":-5,"_39":-5,"_40":41,"_42":43,"_44":45,"_46":47,"_48":49,"_50":51,"_52":53,"_54":55,"_56":-5,"_57":49,"_58":59,"_60":61,"_62":63,"_64":65,"_66":67,"_68":49,"_69":70,"_71":72,"_73":74,"_75":76,"_77":78,"_79":-5,"_80":-5,"_81":-5,"_82":49,"_83":-5,"_84":59,"_85":86,"_87":88,"_89":90,"_91":92,"_93":94,"_95":96},"apiUrl","https://odel.app/api","creator","pareelamre","moduleId","analyzing-llm-rationale","@pareelamre/analyzing-llm-rationale","displayName","Foresea Forecasting","tagline","Forecast future events and scan prediction-market edges.","iconKey","https://avatars.githubusercontent.com/u/35176680?v=4","category","remote","status","published","visibility","public","metaDescription","deploymentType","external","workerName","bindingName","mcpConfig","{\"url\":\"https://foresea.ink/mcp/\",\"transport\":\"streamable-http\",\"authType\":\"none\"}","pricingModel","free","creatorUserId","system:external-import","createdAt",1781363298000,"updatedAt",1788770851000,"sourceRegistry","official-mcp-registry","sourceUrl","https://registry.modelcontextprotocol.io/v0/servers?search=ink.foresea%2Fforecasting","sourceVerified",false,"sourceMissingSince","importedAt","upstreamStars",0,"upstreamUpdatedAt",1788769878000,"detectedTransport","streamable-http","detectedAuthType","none","probeStatus","ok","lastProbedAt","githubRepoUrl","https://github.com/pareelamre/analyzing-llm-rationale","githubRepoFullName","pareelamre/analyzing-llm-rationale","releaseId","ext-pareelamre-analyzing-llm-rationale","version","1.0.0","descriptionMarkdown","# Foresea: Autonomous Prediction Market Intelligence & LLM Rationale Benchmark\n\n[![CI](https://github.com/pareelamre/analyzing-llm-rationale/actions/workflows/ci.yml/badge.svg)](https://github.com/pareelamre/analyzing-llm-rationale/actions/workflows/ci.yml)\n[![Docker Deploy](https://github.com/pareelamre/analyzing-llm-rationale/actions/workflows/docker.yml/badge.svg)](https://github.com/pareelamre/analyzing-llm-rationale/actions/workflows/docker.yml)\n[![Tests](https://img.shields.io/badge/tests-1140%20passing-brightgreen.svg)](tests/)\n[![Python](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/)\n[![Live Platform](https://img.shields.io/badge/platform-foresea.ink-orange.svg)](https://foresea.ink)\n[![MCP Endpoint](https://img.shields.io/badge/mcp-streamable--http-purple.svg)](https://foresea.ink/mcp/)\n\n**Foresea** ([foresea.ink](https://foresea.ink)) is an autonomous prediction market intelligence platform and empirical research framework. It combines real-time probability forecasting across Polymarket and Kalshi, statistical edge discovery, multi-model shadow trading tournaments, an autonomous 19-tool ReAct execution agent, and a public Model Context Protocol (MCP) server.\n\nThe repository also serves as the artifact for research studying how explicit rationale instructions, evidence injection, and reasoning structures affect LLM forecasting behavior and calibration on Metaculus-style questions.\n\n---\n\n## 📑 Table of Contents\n\n- [Live Platform & Web Views](#live-platform--web-views)\n- [Edge Board & Autonomous Shadow Trading Desk](#edge-board--autonomous-shadow-trading-desk)\n- [5-Minute Crypto Micro-Markets](#5-minute-crypto-markets)\n- [Production Deployment & Infrastructure](#production-deployment-notes)\n- [Using the API & Endpoints](#using-the-api)\n- [Agent: Automated Intelligence Layer & ReAct Tools](#agent-automated-intelligence-layer)\n- [Model Context Protocol (MCP) Integration](#mcp-server-let-ai-agents-call-foresea-as-tools)\n- [Custom Integrations & Ecosystem Tools](#custom-integrations--ecosystem-tools)\n- [Trading Execution & KMS Guardrails](#trading-execution-polymarket-and-kalshi)\n- [Repository Contents & Prompt Variants](#repository-contents)\n- [Foresea Autoresearch](#foresea-autoresearch)\n- [Reproducing Core Outputs & SQL Analytics](#reproducing-core-outputs)\n- [Quality Checks & Development](#quality-checks)\n\n---\n\n## Live Platform & Web Views\n\nDeployed on [Google Cloud Run](https://cloud.google.com/run) with high-concurrency scaling, streaming responses, and continuous delivery:\n\n```text\nhttps://foresea.ink\n```\n\n### Core Web Applications & Routes\n\n- **Market Desk (`/`)**: Real-time landing view with market radar, model-vs-market gap highlights, and interactive walkthroughs.\n- **Forecasting Chat (`/chat`, `/chat/:id`)**: Conversational interface with streaming rationale, news evidence citation, and multi-turn market analysis.\n- **Edge Board (`/edge`, `/edge/markets`)**: Ranked live Foresea-vs-market pricing discrepancies across Polymarket and Kalshi, backed by calibration and lead-time scores.\n- **Mark-to-Market View (`/edge/mtm`)**: Continuous mark-to-market valuation and PnL tracking for resolved and open market predictions.\n- **Agentic Trading Desk (`/edge/agentic`)**: Multi-model autonomous trading tournament. Independent $10,000 shadow accounts for each model (Gemma 4 26B, Qwen 3.8 27B, GPT-OSS 120B, GLM 5.3, GLM 5.3 Flash, DeepSeek V4 Flash, MiniMax M3, Llama 3.3 70B) executing real-time paper trades, tracking equity curves, and logging hourly cycle health.\n- **Trading Terminal (`/trade`)**: Non-custodial, client-side order preview and execution terminal with Cloud KMS envelope encryption for Polymarket and Kalshi credentials.\n- **Watchlist (`/watchlist`)**: Follow specific markets with automated daily digest emails.\n- **Public Forecasts (`/forecast/:share_id`)**: Shareable forecast permalinks with rationale cards and provenance.\n\n```bash\n# Health check\ncurl https://foresea.ink/health\n\n# Single-record prediction\ncurl -X POST https://foresea.ink/predict \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"question\": \"Will X happen by date Y?\",\n \"question_type\": \"binary\",\n \"description\": \"Context here.\",\n \"news_articles\": [],\n \"attach_evidence\": true,\n \"evidence_top_k\": 5,\n \"market_platform\": \"Polymarket\",\n \"market_probability\": 0.42,\n \"variant\": \"variant0_neutral_baseline\"\n }'\n```\n\nWhen `attach_evidence` is true and no `news_articles` are supplied, `/predict`\nfetches and ranks current news evidence from GDELT, Google News RSS, and Stooq by\ndefault, injects it into the model prompt, and returns the selected\n`evidence_articles` with the forecast. Supplying `news_articles` skips automatic\nretrieval and uses the caller-provided evidence.\n\nThe response includes both the forecast and the evidence used by the model:\n\n```json\n{\n \"question_type\": \"binary\",\n \"predicted_answer\": \"Yes\",\n \"confidence\": 0.86,\n \"options\": [],\n \"range_forecast\": null,\n \"rationale\": \"Model-generated explanation for the forecast.\",\n \"model_rationale\": \"Model-generated explanation for the forecast.\",\n \"variant\": \"variant0_neutral_baseline\",\n \"model_key\": \"gpt-oss-120b\",\n \"evidence_sources\": [\n {\n \"source\": \"Reuters\",\n \"title\": \"Article headline\",\n \"url\": \"https://example.com/article\",\n \"publish_date\": \"2026-05-29T00:00:00Z\",\n \"relevance_score\": 0.82\n }\n ],\n \"evidence_articles\": [\n {\n \"title\": \"Article headline\",\n \"summary\": \"Cleaned article summary.\",\n \"source\": \"Reuters\",\n \"url\": \"https://example.com/article\",\n \"publish_date\": \"2026-05-29T00:00:00Z\",\n \"relevance_score\": 0.82,\n \"search_query\": \"query used for retrieval\"\n }\n ],\n \"evidence_error\": null,\n \"market_analysis\": {\n \"platform\": \"Polymarket\",\n \"market_url\": \"https://example.com/market\",\n \"outcome\": \"Yes\",\n \"market_probability\": 0.42,\n \"model_probability\": 0.86,\n \"edge\": 0.44,\n \"stance\": \"model_above_market\",\n \"summary\": \"Foresea is 44 percentage points above the market on Yes.\"\n }\n}\n```\n\nUse `evidence_sources` when a client only needs the source list and links. Use\n`evidence_articles` when a client needs the article-level details that were\nattached to the model prompt. `rationale` and `model_rationale` are generated by\n`gpt-oss-120b` and explain why the model chose its answer and confidence.\nWhen `market_probability` is supplied, `market_analysis` is computed\ndeterministically from the model probability and the market-implied probability.\n\n## Edge Board & Autonomous Shadow Trading Desk\n\nForesea continuously evaluates state-of-the-art LLMs against real-world prediction markets, tracking statistical edge, calibration accuracy, and shadow portfolio performance across three dedicated views:\n\n### 1. Live Markets Disagreement (`/edge` or `/edge/markets`)\n- **Real-Time Edge Ranking**: Quantifies the statistical gap between model forecasts and live market pricing across Polymarket and Kalshi:\n $$\\text{Edge} = P_{\\text{model}} - P_{\\text{market}}$$\n- **Sizing & Recommendation**: Computes Fractional Kelly allocations and categorizes actionable trades (`buy_yes`, `buy_no`, `hold`).\n- **Calibration & Resolution**: Backed by hourly snapshots committed to `static/track_record_live.json`, calculating Brier scores, Expected Calibration Error (ECE), and lead-time skill once markets resolve.\n\n### 2. Mark-to-Market Evaluation (`/edge/mtm`)\n- **Continuous Portfolio Valuation**: Evaluates paper positions at live bid/ask quotes rather than relying on eventual settlement alone.\n- **Unrealized vs. Realized PnL**: Provides an institutional-grade view of trade trajectory and volatility exposure across open horizons.\n\n### 3. Agentic Autonomous Trading Tournament (`/edge/agentic`)\n- **Multi-Model Tournament**: Evaluates 8 distinct LLMs acting as autonomous portfolio managers:\n - **Gemma 4 26B** (`gemma-4-26b-a4b-it`)\n - **Qwen 3.8 27B** (`qwen3-8-27b`)\n - **GPT-OSS 120B** (`gpt-oss-120b`)\n - **GLM 5.3** (`glm-5-3`) & **GLM 5.3 Flash** (`glm-5-3-flash`)\n - **DeepSeek V4 Flash** (`deepseek-v4-flash`)\n - **MiniMax M3** (`minimax-m3`)\n - **Llama 3.3 70B** (`llama-3.3-70b-instruct`)\n - *Baseline*: `crowd-follow` (no-LLM consensus control)\n- **Independent $10,000 Portfolios**: Each model starts with a simulated $10,000 cash balance and autonomy to scan markets, evaluate orderbooks, and execute Immediate-or-Cancel (IOC) paper orders.\n- **Equity Curves & Cycle Health**: Renders cumulative returns ($10,000 reinvested), drawdown metrics, trade logs, and hourly cycle execution health (tracking provider availability, token budgets, and trade triggers).\n\n## 5-minute crypto markets\n\nThe local crypto micro-market model in `src/analyzing_llm_rationale/crypto_5m.py`\nis built for 5-minute UP/DOWN markets where the goal is profitable selective\ntrading, not constant action. It combines:\n\n- shrunken-drift lognormal moneyness pricing,\n- AR(1) return forecasting with EWMA volatility,\n- fixed or adaptive logistic ML features from momentum, reversal, volatility\n regime, range position, and volume imbalance.\n\nEach forecast returns `predicted_outcome`, `probability_up`,\n`component_probabilities`, model-vs-market `edge`, and a fee-aware `strategy`.\nThe strategy only recommends a trade when net expected value clears fees and the\nconfigured no-trade threshold.\n\n```bash\n.venv/bin/python scripts/crypto_5m_backtest.py \\\n --benchmark \\\n --symbols BTC,ETH,SOL \\\n --days 1 \\\n --max-candles 1600 \\\n --lookback-minutes 60 \\\n --horizon-minutes 5 \\\n --market-probability 0.50 \\\n --fee-bps 2 \\\n --ml-modes fixed,adaptive \\\n --edge-thresholds 0,0.01,0.03,0.05,0.08 \\\n --selection-fraction 0.6 \\\n --folds 4 \\\n --training-window 120 \\\n --max-rows 80 \\\n --benchmark-log data/crypto_5m_benchmark_runs.jsonl\n```\n\nUse `fold_aggregate` and `evidence_quality` before risking capital. If selection\nis unstable or holdout PnL is weak, the correct profitable action is to abstain.\n`--benchmark-log` appends a compact JSONL record for tracking whether the\nselected threshold and model mode keep working across benchmark runs.\nResolve completed markets against Binance candles:\n\n```bash\n.venv/bin/python scripts/crypto_5m_backtest.py \\\n --resolve \\\n --symbol BTCUSDT \\\n --target-price 62400.52 \\\n --start-time-ms 1780000000000 \\\n --horizon-minutes 5 \\\n --predicted-outcome down\n```\n\nThe resolver returns `pending` before expiry and `resolved` afterward with\n`actual_outcome`, `resolved_price`, and `prediction_correct`.\n\nRecord and resolve paper signals over time:\n\n```bash\n.venv/bin/python scripts/crypto_5m_backtest.py \\\n --paper-signal \\\n --symbol BTCUSDT \\\n --market-probability 0.50 \\\n --fee-bps 2 \\\n --signal-log data/crypto_5m_signal_log.jsonl\n\n.venv/bin/python scripts/crypto_5m_backtest.py \\\n --resolve-signal-log \\\n --signal-log data/crypto_5m_signal_log.jsonl\n\n.venv/bin/python scripts/crypto_5m_backtest.py \\\n --signal-summary \\\n --signal-log data/crypto_5m_signal_log.jsonl \\\n --min-resolved-trades 200 \\\n --min-total-pnl 0 \\\n --min-hit-rate 0.53\n\n.venv/bin/python scripts/crypto_5m_backtest.py \\\n --paper-loop \\\n --symbols BTC,ETH,SOL \\\n --iterations 12 \\\n --sleep-seconds 60 \\\n --market-probability 0.50 \\\n --fee-bps 2 \\\n --signal-log data/crypto_5m_signal_log.jsonl\n```\n\nThe signal log is the running dataset for model improvement: each record stores\nthe forecast, recommendation, later `actual_outcome`, correctness, and\n`pnl_per_contract` for actual `buy_up`/`buy_down` paper trades. Use\n`--signal-summary` to audit whether resolved paper trades are positive after\nfees; `trade_ready` stays false until the configured trade count, PnL, and hit\nrate thresholds are met. Use `--dry-run` with `--paper-loop` to preview signals\nwithout writing the log.\n\n## Production Deployment Notes\n\nProduction is served from the custom domain:\n\n```text\nhttps://foresea.ink\n```\n\nThe Cloud Run service name, project ID, and region are set at deploy time via `gcloud run deploy`.\n\nRequired runtime environment:\n\n- `SCADS_AI_API_KEY`: Secret Manager secret used by hosted model calls.\n- `MODEL_DEVICE=cpu`: production Cloud Run runs the CPU image.\n- `CUSTOM_DOMAIN=foresea.ink`: redirects `*.run.app` requests to the public domain.\n- `GOOGLE_CLIENT_ID`: Google OAuth web client ID used by `/auth/config`.\n- `GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET`: GitHub OAuth app credentials. The\n OAuth app's callback URL must be the site origin (e.g. `https://foresea.ink/`).\n When unset, the \"Continue with GitHub\" button is hidden and `/auth/github`\n returns 503. Sign-in also works with Google and email/password.\n- `SESSION_SECRET`: long random string used to sign browser session JWTs and\n derive domain-separated, non-reversible references for authenticated analytics.\n Rotating it starts a new attribution cohort; it never exposes account emails.\n\nThe OAuth client must allow these JavaScript origins:\n\n```text\nhttps://foresea.ink\nhttps://www.foresea.ink\nhttps://.run.app\n```\n\nTo update non-secret environment variables without replacing the existing\n`SESSION_SECRET`, use `--update-env-vars`:\n\n```bash\ngcloud run services update \\\n --region \\\n --project \\\n --update-env-vars MODEL_DEVICE=cpu,CUSTOM_DOMAIN=foresea.ink,GOOGLE_CLIENT_ID=''\n```\n\nVerify the deployed auth config and health endpoint:\n\n```bash\ncurl https://foresea.ink/auth/config\ncurl https://foresea.ink/health\n```\n\n### Scaling and caching\n\nThe server is built to scale horizontally on Cloud Run:\n\n- **Authentication** supports Google One-Tap *and* email/password\n (`/auth/register`, `/auth/login`). Passwords are stored as salted\n PBKDF2-HMAC-SHA256 hashes; accounts live in Cloud Datastore.\n- **Caching and rate limiting** use Redis when `REDIS_URL` is set, so they are\n shared across instances; otherwise they fall back to per-instance in-memory\n state and fail open. `/predict` (non-personalised requests), evidence\n retrieval, and `/extract` URL fetches are cached; public GETs send\n `Cache-Control`.\n\n| Var | Default | Description |\n|-----|---------|-------------|\n| `REDIS_URL` | unset | Memorystore/Redis URL. Shares cache + rate limits across instances. |\n| `PREDICT_CACHE_TTL` | `600` | Cache TTL (s) for non-personalised `/predict` responses. `0` disables. |\n| `EVIDENCE_CACHE_TTL` | `900` | Cache TTL (s) for evidence retrieval. |\n| `EXTRACT_CACHE_TTL` | `3600` | Cache TTL (s) for `/extract` URL fetches. |\n| `LOCAL_CACHE_MAX` | `1024` | Max entries in the in-memory fallback cache. |\n| `SEARXNG_URL` / `TAVILY_API_KEY` / `SERPER_API_KEY` / `BRAVE_API_KEY` | unset | Enable web search as an evidence source. A self-hosted **SearXNG** is preferred when set, then Tavily, Serper, Brave. Tavily/Serper have free no-card tiers. When none is set, evidence comes from GDELT, Google News, and RSS. |\n| `NEWSAPI_KEY` | unset | Enables NewsAPI as an evidence source. |\n\n### Live track record\n\n`GET /track-record` serves the public forecast track record. The heavy tick loop\ndoes not run on Cloud Run: `.github/workflows/track-record-tick.yml` runs hourly\non GitHub Actions, updates `data/track_record_store.json` as the source-of-truth\nentity store, writes the public aggregate to `static/track_record_live.json`, and\ncommits both files back to `main`. At runtime, Cloud Run fetches the committed\naggregate from raw GitHub, falling back to the bundled file and then the static\nbacktest in `static/track_record.json`.\n\nThe Action discovers short-to-medium-horizon Polymarket/Kalshi markets in\nseparate close-date bands (`2-7`, `7-14`, `14-30`, `30-60` days by default) and\ncalls `/predict` once per newly snapshotted market/model. If `/predict` is\nprotected, set the GitHub secret `PREDICT_API_KEY`; no server-side\n`/track-record/tick` endpoint is required. `TRACK_RECORD_TOKEN` is optional and\nonly enables the agent-enrolled market bridge.\n\nThe default scheduled forecast job is deliberately cost-capped: it runs every 6\nhours, snapshots at most 2 markets per venue, and forecasts only\n`gpt-oss-120b` plus the no-LLM `crowd-follow` baseline. Use the manual workflow\ndispatch input `reforecast_each_tick=1` for a one-off full refresh instead of\nforcing every scheduled run to reforecast all open markets.\n\nThe homepage market desk uses `GET /radar`, which is derived from\n`static/track_record_live.json` and its `edge_board`. Radar highlights current\nmodel-vs-market gaps and keeps the first screen fast by reusing the committed\ntrack-record aggregate instead of scanning venues on every page load.\n\nRaise the Cloud Run throughput ceiling (no idle cost while `min-instances=0`):\n\n```bash\ngcloud run services update analyzing-llm-rationale --region us-central1 \\\n --max-instances 20 --concurrency 40 --memory 1Gi\n```\n\nFor the lowest-cost public deployment, keep the service on request-only CPU,\nscale to zero, and cap burst scale-out. This is the profile used by the deploy\nworkflow. Startup CPU boost stays enabled because it reduces cold-start latency\nwithout keeping an idle instance warm:\n\n```bash\ngcloud run services update analyzing-llm-rationale \\\n --region us-central1 \\\n --project brave-drive-471109-d9 \\\n --cpu 1 \\\n --memory 512Mi \\\n --min-instances 0 \\\n --max-instances 3 \\\n --concurrency 20 \\\n --timeout 180 \\\n --cpu-throttling \\\n --cpu-boost \\\n --update-env-vars INTERACTIVE_DEFAULT_MODEL=gemma-4-26b-a4b-it,INTERACTIVE_MAX_TOKENS=384,CHAT_PROVIDER_TIMEOUT_S=15,CHAT_PROVIDER_MAX_RETRIES=0,EVIDENCE_TIMEOUT_S=6,EVIDENCE_MAX_CONCURRENCY=4\n```\n\nMeasure deployed forecast latency after each runtime change:\n\n```bash\npy scripts/measure_forecast_latency.py \\\n --url https://foresea.ink \\\n --mode stream \\\n --models minimax-m3 \\\n --runs 3 \\\n --no-attach-evidence \\\n --max-tokens 384\n```\n\nIf cold starts still dominate, raise `--min-instances` to `1` as an explicit\nlatency/cost tradeoff.\n\nMarket search runs in-process in the main API. The optional Go `marketd`\nmicroservice is build/test-only in GitHub Actions and is not deployed to Cloud\nRun by default.\n\n### Artifact Registry retention\n\nCI pushes commit-tagged Docker images to Artifact Registry on every deploy. Keep\nthe `docker` repository cleanup policy active so old images do not accumulate:\n\n```bash\ngcloud artifacts repositories set-cleanup-policies docker \\\n --location us-central1 \\\n --project brave-drive-471109-d9 \\\n --policy infra/artifact-registry-cleanup-policy.json \\\n --no-dry-run\n```\n\nThe policy deletes images older than 7 days, keeps the newest 5 versions per\npackage, and always keeps the `main` tag.\n\nDocker builds run in GitHub Actions, not Cloud Build; no Cloud Build trigger or\nstaging bucket is required for the normal deploy path.\n\nOnce `max-instances > 1`, provision Memorystore for Redis (billable) and set\n`REDIS_URL` so rate limiting and caching stay correct across instances:\n\n```bash\ngcloud services enable redis.googleapis.com vpcaccess.googleapis.com compute.googleapis.com\ngcloud redis instances create foresea-cache --size=1 --region=us-central1 --tier=basic\ngcloud compute networks vpc-access connectors create foresea-vpc \\\n --region=us-central1 --range=10.8.0.0/28\ngcloud run services update analyzing-llm-rationale --region us-central1 \\\n --vpc-connector foresea-vpc \\\n --update-env-vars REDIS_URL=redis://:6379\n```\n\n## Using the API\n\nSee [additional Kalshi and Polymarket endpoints](https://github.com/pareelamre/analyzing-llm-rationale/blob/main/docs/venue_api_extensions.md) for historical data, account pagination, order management and native exchange streams.\n\nThe public Cloud Run API is the easiest integration target. It accepts\nforecasting questions and returns a typed forecast, model rationale, and optional\nevidence articles. It is built for resolvable forecasts, not general Q&A.\n\n### Endpoints & Web Routes\n\n#### Public Web Views\n- `GET /`: landing desk, real-time market radar, and interactive workflow demo.\n- `GET /chat`, `GET /chat/{id}`: conversational forecast studio with streaming rationales and evidence.\n- `GET /edge`, `GET /edge/markets`: live market edge board with statistical gap rankings and Kelly sizing.\n- `GET /edge/mtm`: mark-to-market performance of open positions across prediction venues.\n- `GET /edge/agentic`: multi-model autonomous trading tournament, equity curves, cycle health, and paper trade tape.\n- `GET /trade`: non-custodial trading terminal for Polymarket and Kalshi with envelope encryption.\n- `GET /watchlist`: tracked favorite markets with daily digest notifications.\n- `GET /forecast/{share_id}`: public read-only forecast share permalink.\n- `GET /embed/forecast/{share_id}`: lightweight iframe-embeddable forecast widget.\n- `GET /widget.js`: drop-in web component `` for publishing live forecasts.\n\n#### Core Forecasting & Intelligence API\n- `GET /health`: service health check and operational status.\n- `POST /predict`: public probability prediction endpoint with optional evidence retrieval.\n- `POST /agent/analyze`: orchestrated end-to-end analysis of a live market with custom skills and ReAct loops.\n- `GET /agent/scan`: venue scanner identifying mispriced markets ranked by statistical edge.\n- `GET /radar`: homepage market desk payload derived from the live track record.\n- `GET /track-record`: public live track record and historical calibration statistics.\n- `GET /track-record/digest`: shareable markdown summary of the live track record.\n- `GET /pr-agent`: opt-in agent-to-agent outreach packet for Foresea discovery.\n\n#### Venue & Market Data\n- `GET /markets/polymarket`: fetch live normalized Polymarket quotes, orderbooks, and liquidity data.\n- `GET /markets/kalshi`: fetch live normalized Kalshi quotes, strike ranges, and ticker metadata.\n\n#### Model Context Protocol (MCP)\n- `GET /mcp/`: public remote Model Context Protocol (Streamable-HTTP) endpoint.\n- `GET /.well-known/mcp/server.json`: public MCP discovery manifest.\n\n#### Analytics, Sharing & Account Sync\n- `POST /analytics/visit`: privacy-preserving page visit tracking (linked only to non-reversible references).\n- `POST /analytics/event`: funnel event recording (`forecast_completed`, `watchlist_add`, `share_created`, `digest_sent`).\n- `GET /analytics/events/summary`: 30-day aggregate product analytics summary.\n- `POST /forecasts/share`: generate an explicit public forecast share ID.\n- `GET|POST /chat/conversations`: cloud conversation sync for authenticated users.\n- `GET|POST|DELETE /favorites`: watchlist management and tracking.\n\n#### Non-Custodial Trading Execution\n- `GET|PUT|DELETE /trading/connections/{platform}`: KMS-encrypted per-user exchange connection credentials.\n- `POST /trading/preview`: dry-run order normalization and limit collar verification.\n- `POST /trading/orders`: live order submission with explicit two-step user confirmation.\n- `GET /trading/portfolio`: authenticated balances, open positions, resting orders, and execution fills.\n- `POST /trading/orders/{audit_order_id}/reconcile`: venue order status and fill reconciliation.\n- `DELETE /trading/orders/{audit_order_id}`: cancel resting orders at venue.\n\n### Web app runtime state\n\nAnonymous chats stay in browser `localStorage`. Signed-in users sync\nconversations through `/chat/conversations`, while watchlist tracking uses\n`FavoriteMarket` entities exposed through `/favorites` and `/favorites/prices`.\nThe favorites digest runs from `.github/workflows/favorites-digest.yml` via\n`scripts/favorites_digest.py`.\n\nForecast sharing is opt-in: clients call `POST /forecasts/share` to create a\npublic `GET /forecast/{share_id}` page. Do not expose full private chat history\nin shared forecast views.\n\n### Agent: automated intelligence layer\n\n`POST /agent/analyze` runs the whole pipeline autonomously: **resolve the market**\n(fetch a live Polymarket/Kalshi price when an identifier is given) → **gather\nevidence + forecast** → **price the edge** → run any **custom skills** →\n**recommend**. It returns one structured report.\n\n```bash\ncurl -X POST https://foresea.ink/agent/analyze \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"platform\": \"polymarket\",\n \"slug\": \"will-the-fed-cut-rates-in-2026\",\n \"skills\": [\n {\"name\": \"Base rate check\", \"instruction\": \"Compare to historical base rates.\"},\n {\"name\": \"Risk\", \"instruction\": \"What would most change this forecast?\"}\n ]\n }'\n```\n\nCustom **skills** are your own analysis steps — each runs as an extra model pass\nover the question, forecast, and evidence, and comes back as a named section in\nthe report. Provide a `question` directly, or a `platform` + market identifier\n(`slug`/`market_id` for Polymarket, `ticker` for Kalshi). Pass `history` (prior\nturns) for multi-turn follow-ups — with history, short follow-ups like \"why?\" or\n\"what about June?\" are answered in context. BYOK fields (`openrouter_api_key`,\n`openrouter_model`, `provider_base_url`) apply here too.\nThe report includes `recommendation` (`buy_yes`/`buy_no`/`hold`/`no_market_price`),\n`edge`, `model_probability`, `market_probability`, `thesis`, `evidence_sources`,\nand `pipeline` (the ordered steps that ran).\n\n#### Autonomous 19-Tool ReAct Execution Loop\n\nForesea agents utilize an autonomous ReAct (Reason + Act) loop with dynamic plan formation, tool selection, reflection, and JSON error recovery:\n\n- **Forecasting & Statistical Edge**:\n - `forecast`: Calibrated probability forecasting with confidence and rationales.\n - `get_market`: Normalized quote and market metadata lookup across Polymarket and Kalshi.\n - `scan_markets`: Discover live markets filtered and ranked by model-vs-market edge.\n - `batch_quotes`: High-throughput multi-venue quote aggregation.\n - `search_evidence` & `web_search`: Live multi-source retrieval (GDELT, Google News, SearXNG, Tavily, Stooq).\n - `track_record` & `edge_board`: Historical calibration metrics and open ranked alpha opportunities.\n - `market_leaderboard`: Track record rankings of top prediction market traders.\n\n- **Venue Orderbooks & Microstructure**:\n - `exchange_status`: Kalshi exchange status, market trading state, and operational schedule.\n - `orderbook`: Live bids and asks orderbook depth for Kalshi tickers or Polymarket tokens.\n - `market_tags`: Polymarket category tags and market classification taxonomy.\n - `price_history`: Historical price points, timeseries, and OHLC candlesticks.\n - `live_data`: Real-time sports statistics, play-by-play data, and live game feeds.\n - `polymarket_meta`: Event series hierarchy, resolution rules, and community discussions.\n - `recent_trades`: Real-time trade tape and prints (prices, contract sizes, timestamps).\n\n- **Execution & State Management**:\n - `place_trade`: Immediate-or-Cancel (IOC) paper execution against live orderbook quotes with shadow balance updates.\n - `manage_notes`: Scratchpad state persistence across ReAct reasoning turns.\n - `fetch_api`: Safe, sandboxed HTTPS retrieval for external data verification.\n\n*Automatic Calling Aliases*: `TOOL_ALIASES` in `agent_capabilities.py` normalizes alternative LLM naming conventions (e.g., `http_get` → `fetch_api`, `candlesticks` → `price_history`, `comments` → `polymarket_meta`, `trades` → `recent_trades`, `leaderboard` → `market_leaderboard`).\n\n#### Durable private Agent Runs\n\nEvery signed-in call to `POST /agent/analyze` (including the streamed endpoint)\nalso creates a private `AgentRun`. It retains a bounded, secret-free input\nsnapshot, lifecycle timeline, model report, and any review-only trade handoff.\nUse `GET /agent/runs` for the newest operator timeline and\n`GET /agent/runs/{run_id}` for one full report. The snapshot intentionally\nexcludes provider keys, browser credentials, conversation history, and raw\ncustom-skill instructions. An Agent Run is research only: even when it has a\ntrade handoff, it cannot create, size, or submit an order; the user must still\ncreate and explicitly confirm a durable Trade Run in the terminal.\n\n#### Copied agents: private, versioned research recipes\n\nSigned-in users can copy a public Foresea model from the Agentic board. The copy\nis saved under the user's account as an immutable version-1 research recipe;\nit contains the public source model and analysis instruction only—never the\nsource agent's private context, shadow-account history, exchange connection,\norder size, or trading permission. Use `POST /agent-profiles/copy` with an\nallowlisted `source_agent_id`, then pass the returned `agent_profile_id` to\n`POST /agent/analyze`.\n\nWhen a profile is selected, the server resolves the profile's model and\ninstruction itself, ignores client BYOK/provider/model overrides, and forces\nthe fixed research pipeline (no tool loop or trade tool). The resulting report\nreturns its profile ID, source, version, and `research_only` mode for\nreproducibility. A profile may prepare the existing review-only trade handoff,\nbut it cannot create or submit an exchange order; a signed-in user must still\ncreate a durable Trade Run and explicitly confirm `PLACE REAL ORDER` in the\ntrading terminal.\n\n### Edge scan — find mispriced markets\n\n`GET /agent/scan` lists live markets on a venue, forecasts each, and returns the\nones whose model-vs-market gap clears `min_edge`, ranked by `|edge|`.\n\n```bash\ncurl \"https://foresea.ink/agent/scan?platform=polymarket&limit=4&min_edge=0.1\"\n```\n\nParams: `platform` (`polymarket` or `kalshi`), `limit` (markets to analyse, max 8),\n`min_edge` (default `0.1`), `evidence_top_k`. Each market runs a full forecast, so\nit's bounded by `limit` and the result is cached briefly. Response: `{platform,\nscanned, opportunities: [{question, market_url, market_probability,\nmodel_probability, edge, recommendation}]}`. In the web app, the desk's\n**\"⚡ Scan Polymarket for mispriced markets\"** button calls this.\n\n### MCP server: let AI agents call Foresea as tools\n\nForesea exposes a public remote MCP server at:\n\n```text\nhttps://foresea.ink/mcp/\n```\n\nIt is advertised for discovery at:\n\n```text\nhttps://foresea.ink/.well-known/mcp/server.json\n```\n\nThe remote MCP server is a thin tool layer over the public API. It exposes:\n\n- `foresea_forecast`: produce calibrated probability forecasts with rationale and news evidence.\n- `foresea_analyze_market`: evaluate a specific Polymarket/Kalshi market with model-vs-market edge & thesis.\n- `foresea_scan_markets`: scan live markets ranked by model-vs-market disagreement.\n- `foresea_batch_quotes`: fetch multi-market quotes across venues in one roundtrip.\n- `foresea_check_run`: check background execution status for long-running market research runs.\n- `foresea_edge_board`: top open trading opportunities ranked by statistical edge.\n- `foresea_track_record`: public accuracy, Brier score, ECE, and calibration metrics.\n- `foresea_debate_market`: conduct adversarial multi-agent debate (Bull vs. Bear vs. Risk Judge).\n- `foresea_optimize_portfolio`: calculate optimal Fractional Kelly capital allocations across open edges.\n- `foresea_feed_latest`: real-time alpha feed combining live market edges, agent trades, and leaderboard rankings.\n- `foresea_exchange_status`: inspect Kalshi exchange status (trading active flag) and operating schedule.\n- `foresea_orderbook`: fetch live bids and asks orderbook depth for Kalshi tickers or Polymarket tokens.\n- `foresea_market_tags`: fetch active category taxonomy and tags from Polymarket.\n- `foresea_price_history`: fetch historical price points or OHLC candlesticks.\n- `foresea_live_data`: fetch real-time sports game statistics, play-by-play data, and live event feeds.\n- `foresea_polymarket_meta`: fetch event series listings, community discussion comments, or sports metadata.\n- `foresea_recent_trades`: fetch recent executed trade tape / prints (prices, sizes, timestamps).\n- `foresea_market_leaderboard`: fetch top profitable trader leaderboard and volume rankings.\n- **Resources**: `foresea://edge-board`, `foresea://markets/trending`, `foresea://track-record`, `foresea://openapi.json`.\n- **Prompts**: `foresea_market_risk_prompt`, `foresea_calibrate_hypothesis`, `foresea_forecast_prompt`, `foresea_system_prompt`.\n\nSee [docs/mcp_commercial_guide.md](https://github.com/pareelamre/analyzing-llm-rationale/blob/main/docs/mcp_commercial_guide.md) for full harness setup guides (Claude Code, Google Antigravity, OpenAI Codex, Cursor, Windsurf, OpenHands, Smithery.ai).\n\n### Custom Integrations & Ecosystem Tools\n\nForesea provides ready-to-run client integrations across popular developer and trading surfaces:\n\n#### 1. Telegram & Discord Signal Bots\n- **Telegram Bot** (`scripts/foresea_telegram_bot.py`): Interactive bot supporting `/forecast `, `/edge`, `/analyze `, `/track`, and automated subscriber edge alerts.\n ```bash\n export TELEGRAM_BOT_TOKEN=\"123456:ABC...\"\n python scripts/foresea_telegram_bot.py\n ```\n- **Discord Bot & Webhooks** (`scripts/foresea_discord_bot.py`): Posts rich Discord embeds to announcement channels on schedule.\n ```bash\n python scripts/foresea_discord_bot.py --webhook-url \"https://discord.com/api/webhooks/...\" --post-edge\n ```\n\n#### 2. Drop-in Web Widget (``)\nEmbed live interactive prediction market forecasts into any blog, news site, or Substack with a single script tag:\n```html\n\n\n\n\n\n\n\n```\n\n#### 3. Real-Money Quant Execution Bridge\nAn opt-in automated execution runner (`scripts/live_trader_bridge.py`) connecting Foresea's statistical edge signals to live prediction venues (Polymarket & Kalshi) with strict risk management guards:\n```bash\n# Dry-run simulation (safe default)\npython scripts/live_trader_bridge.py --dry-run --min-edge 0.08\n\n# Live execution on Kalshi with risk limits\npython scripts/live_trader_bridge.py --live --venue kalshi --min-edge 0.10 --max-position-usd 25\n```\n\n### PR agent — agent-to-agent distribution\n\n`GET /pr-agent?audience=mcp` returns an opt-in outreach packet that other agents,\nMCP catalogs, and tool directories can quote when introducing Foresea. It includes\nthe one-liner, install command, MCP/OpenAPI links, talking points, and an explicit\nno-spam policy.\n\nFor operator-run cold outreach to explicit agent endpoints, prepare a target list\nand use the local runner. It dry-runs by default and only sends with `--send`:\n\n```bash\npython scripts/pr_agent_outreach.py --targets outreach-targets.json\npython scripts/pr_agent_outreach.py --targets outreach-targets.json --send\n```\n\nTarget file shape:\n\n```json\n{\n \"targets\": [\n {\n \"name\": \"Example Agent Directory\",\n \"endpoint\": \"https://agent-directory.example/inbox\",\n \"audience\": \"catalog\",\n \"headers\": {\"Authorization\": \"Bearer ...\"}\n }\n ]\n}\n```\n\nThe public API returns the outreach packet; it does not expose an unauthenticated\nmessage-sending relay. The scheduled GitHub Action\n`.github/workflows/pr-agent-outreach.yml` runs every 5 minutes against\n`data/pr_outreach_targets.json`, sends with `--send`, and records contacted\ntargets in `data/pr_outreach_state.json` so repeated scheduled runs do not\nre-contact the same agent. For a literal always-running local process, run:\n\n```bash\npython scripts/pr_agent_outreach.py \\\n --targets data/pr_outreach_targets.json \\\n --state data/pr_outreach_state.json \\\n --send --watch --interval-s 300\n```\n\nHeader values can reference GitHub Actions secrets via environment variables, for\nexample `\"Authorization\": \"$PR_AGENT_TARGET_AUTH\"`.\n\nSeeded automated targets:\n\n- AgentNDX (`https://agentndx.ai/api/submit`) — public MCP/A2A/x402 review form.\n- MCP.Directory (`https://mcp.directory/api/submit-server`) — public JSON submit route.\n- mcpub (`https://mcpub.dev/mcp`) — public MCP JSON-RPC `submit` tool.\n\nAdditional listing work that is not suitable for the scheduled HTTP sender lives\nin `data/pr_manual_targets.json`. Current manual/GitHub target: mcp.so issue\n`https://github.com/daodao97/chatmcp/issues/213`.\n\n#### Add Foresea to your agent (10 seconds)\n\n##### Option A: Zero-Install Remote Streamable-HTTP (Claude Code / Cursor / Windsurf)\n```bash\n# Claude Code (Remote HTTP)\nclaude mcp add --transport http foresea https://foresea.ink/mcp/\n```\n\n##### Option B: Zero-Install Local Stdio via `uvx` (Claude Desktop / Antigravity / Codex)\n```bash\nuvx --from git+https://github.com/pareelamre/analyzing-llm-rationale.git foresea-mcp\n```\n\n##### Option C: 1-Click Smithery.ai CLI\n```bash\nnpx -y @smithery/cli install foresea --client claude\n```\n\n```jsonc\n// Cursor / Cline / Claude Desktop (mcp.json)\n{ \"mcpServers\": { \"foresea\": { \"url\": \"https://foresea.ink/mcp/\" } } }\n```\n\n```jsonc\n// OpenClaw agent MCP config\n{\n \"mcpServers\": {\n \"foresea\": {\n \"url\": \"https://foresea.ink/mcp/\"\n }\n }\n}\n```\n\nFor OpenClaw, also add this to the target agent's workspace guidance:\n\n```text\nUse Foresea for probability, forecasting, prediction-market research, and\nmarket-edge questions. Call foresea_forecast for general forecasts,\nforesea_analyze_market for Polymarket or Kalshi markets, foresea_scan_markets\nfor discovery, foresea_edge_board for ranked disagreements, and\nforesea_track_record before relying on an edge.\n```\n\n```python\n# Python — official MCP SDK (3.10+)\nfrom mcp import ClientSession\nfrom mcp.client.streamable_http import streamablehttp_client\n\nasync with streamablehttp_client(\"https://foresea.ink/mcp/\") as (r, w, _):\n async with ClientSession(r, w) as s:\n await s.initialize()\n print(await s.call_tool(\"foresea_forecast\",\n {\"question\": \"Will the Fed cut rates by March 2026?\", \"market_probability\": 0.4}))\n```\n\n```python\n# LangChain (langchain-mcp-adapters) — Foresea tools in any LangGraph agent\nfrom langchain_mcp_adapters.client import MultiServerMCPClient\nclient = MultiServerMCPClient({\"foresea\": {\"url\": \"https://foresea.ink/mcp/\", \"transport\": \"streamable_http\"}})\ntools = await client.get_tools() # foresea_forecast, foresea_analyze_market, ...\n```\n\nA runnable end-to-end demo (scan → forecast → edge) is in\n[`examples/foresea_agent_demo.py`](https://github.com/pareelamre/analyzing-llm-rationale/blob/main/examples/foresea_agent_demo.py).\n\nUse `https://foresea.ink/mcp/` directly in MCP clients that support remote\nStreamable HTTP servers. For clients that still require a local stdio command,\nrun the wrapper locally.\n\nThe repo targets Python 3.10+ because the official MCP Python SDK requires it.\nTo create a repo-local Python 3.11 MCP environment with `uv`:\n\n```bash\nuv venv --python 3.11 .venv-mcp\n\nuv pip install --python .venv-mcp/bin/python --no-deps -e .\nuv pip install --python .venv-mcp/bin/python \"mcp>=1.27.1\" requests pyyaml pip\n\nsource .venv-mcp/bin/activate\nanalyze-llm-rationale mcp-server\n```\n\nThat lightweight install avoids pulling the full inference dependency stack\n(notably Torch/CUDA) when all you need is the MCP wrapper. In a full development\nenvironment, `pip install -e \".[mcp]\"` is also valid.\n\nMCP client config example:\n\n```json\n{\n \"mcpServers\": {\n \"foresea\": {\n \"url\": \"https://foresea.ink/mcp/\"\n }\n }\n}\n```\n\nFor a local HTTP MCP endpoint:\n\n```bash\n.venv-mcp/bin/analyze-llm-rationale mcp-server \\\n --transport streamable-http \\\n --host 127.0.0.1 \\\n --port 8787\n```\n\nConnect MCP clients to `http://127.0.0.1:8787/mcp`. If a private deployment\nrequires auth, set `FORESEA_API_KEY` or pass `--api-key`; the wrapper forwards it\nas `X-API-Key`.\n\nQuick verification:\n\n```bash\n.venv-mcp/bin/python - <<'PY'\nimport importlib.metadata as md\nfrom analyzing_llm_rationale.mcp_server import create_mcp_server\n\nprint(md.version(\"mcp\"))\nprint(create_mcp_server().name)\nPY\n```\n\n### Fetch live market prices\n\nPull the current market-implied probability straight from a venue, then feed it\ninto `/predict` as `market_probability` to compute an edge.\n\n```bash\n# Polymarket — by market slug (or ?id=)\ncurl \"https://foresea.ink/markets/polymarket?slug=will-the-fed-cut-rates-in-2026\"\n\n# Kalshi — by market ticker\ncurl \"https://foresea.ink/markets/kalshi?ticker=KXFED-26SEP-C\"\n```\n\nBoth return a normalised quote:\n\n```json\n{\n \"platform\": \"Polymarket\",\n \"question\": \"Will the Fed cut rates in 2026?\",\n \"market_url\": \"https://polymarket.com/market/...\",\n \"outcome\": \"Yes\",\n \"probability\": 0.54,\n \"outcomes\": [\n {\"label\": \"Yes\", \"probability\": 0.54},\n {\"label\": \"No\", \"probability\": 0.46}\n ]\n}\n```\n\n`probability` is `null` for unpriced/illiquid markets. Quotes are cached briefly\n(`MARKET_CACHE_TTL`, default 30s).\n\n### Trading execution: Polymarket and Kalshi\n\nForesea can submit guarded prediction-market orders, but live execution is\ndisabled by default. Keep this separate from `/agent/analyze`: the agent can\nrecommend `buy_yes`/`buy_no`, but order submission requires a signed-in user,\nan encrypted exchange connection, `FORESEA_ENABLE_BYO_TRADING=true`,\n`execute=true`, and the exact confirmation phrase `PLACE REAL ORDER`.\n\nThe browser sends connection credentials only to `PUT /trading/connections/{platform}`.\nForesea validates them, generates a unique data-encryption key for that one\nuser/venue connection, and encrypts the credential payload locally. Cloud KMS\nwraps the data key using authenticated user/venue context; Datastore receives only\nthe ciphertext, wrapped data key, and KMS key metadata. The KMS root key never\nenters the service process. Foresea never returns credentials to the browser and\nrejects inline `venue_credentials` on preview and order requests.\n\nCreate a dedicated KMS symmetric `ENCRYPT_DECRYPT` CryptoKey and give only the\nCloud Run service account `roles/cloudkms.cryptoKeyEncrypterDecrypter` on that\nkey. Configure its fully qualified resource name, not a secret value:\n\n```bash\ngcloud kms keyrings create foresea-trading --location=us-central1\ngcloud kms keys create exchange-connections --location=us-central1 \\\n --keyring=foresea-trading --purpose=encryption\ngcloud kms keys add-iam-policy-binding exchange-connections --location=us-central1 \\\n --keyring=foresea-trading \\\n --member=\"serviceAccount:${CLOUD_RUN_SERVICE_ACCOUNT}\" \\\n --role=\"roles/cloudkms.cryptoKeyEncrypterDecrypter\"\n```\n\nCloud KMS key rotation is transparent to existing wrapped data keys. The service\nuses the primary key version for a new connection and KMS selects the needed older\nversion when decrypting an existing one.\n\n```bash\n# Global guardrails\nexport FORESEA_ENABLE_TRADING=false # must be true for shared-account live orders\nexport FORESEA_ENABLE_BYO_TRADING=false # must be true for encrypted user-account live orders\nexport FORESEA_MAX_ORDER_NOTIONAL=50 # local cap per order, USD\nexport FORESEA_ALLOW_MARKET_ORDERS=false # separate gate for IOC/FOK-style orders\nexport FORESEA_TRADING_KMS_KEY_NAME=projects//locations//keyRings/foresea-trading/cryptoKeys/exchange-connections\n\n# Optional shared server account (not used by the public connection flow)\n# Kalshi authenticated REST (RSA-PSS signing)\nexport KALSHI_API_KEY_ID=\nexport KALSHI_PRIVATE_KEY_FILE=/secrets/kalshi-private-key.pem\nexport KALSHI_BASE_URL=https://external-api.kalshi.com/trade-api/v2\n\n# Polymarket CLOB SDK\nexport POLYMARKET_PRIVATE_KEY=\nexport POLYMARKET_API_KEY=\nexport POLYMARKET_API_SECRET=\nexport POLYMARKET_API_PASSPHRASE=\nexport POLYMARKET_FUNDER_ADDRESS=\nexport POLYMARKET_SIGNATURE_TYPE=\n```\n\nInstall the optional SDKs in production with:\n\n```bash\npip install -e \".[serve,trading]\"\n```\n\nThe Docker image installs `trading`, so Cloud Run only needs secrets/env vars.\n\n#### Migrating the retired shared Fernet key\n\nIf version-1 connection records already exist, deploy the KMS configuration and\nkeep the old `FORESEA_CREDENTIALS_ENCRYPTION_KEY` Secret Manager value available\nonly during migration. Existing records migrate lazily on their first authenticated\nuse, or migrate the full set from an environment with Application Default\nCredentials and Datastore access:\n\n```bash\npy scripts/migrate_trading_connection_encryption.py # dry run\npy scripts/migrate_trading_connection_encryption.py --apply\n```\n\nThe command reports counts only and never outputs credentials. Once no version-1\nrecords remain, remove `FORESEA_CREDENTIALS_ENCRYPTION_KEY` from Cloud Run and\nSecret Manager.\n\nCheck encrypted account connection metadata (no secrets are returned):\n\n```bash\ncurl https://foresea.ink/trading/connections \\\n -H \"Authorization: Bearer $FORESEA_SESSION\"\n```\n\nConnect one account over TLS (the payload is encrypted before persistence):\n\n```bash\ncurl -X PUT https://foresea.ink/trading/connections/kalshi \\\n -H \"Authorization: Bearer $FORESEA_SESSION\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"venue_credentials\":{\"kalshi_api_key_id\":\"\",\"kalshi_private_key\":\"\"}}'\n```\n\nPreview a Kalshi order without execution:\n\n```bash\ncurl -X POST https://foresea.ink/trading/preview \\\n -H \"Authorization: Bearer $FORESEA_SESSION\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"platform\": \"kalshi\",\n \"ticker\": \"KXFED-26SEP-C\",\n \"action\": \"buy\",\n \"outcome\": \"yes\",\n \"price\": 0.42,\n \"quantity\": 1\n }'\n```\n\nSubmit a live order only after reviewing the preview:\n\n```bash\ncurl -X POST https://foresea.ink/trading/orders \\\n -H \"Authorization: Bearer $FORESEA_SESSION\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"platform\": \"kalshi\",\n \"ticker\": \"KXFED-26SEP-C\",\n \"action\": \"buy\",\n \"outcome\": \"yes\",\n \"price\": 0.42,\n \"quantity\": 1,\n \"execute\": true,\n \"confirmation\": \"PLACE REAL ORDER\"\n }'\n```\n\nFor Polymarket, pass the CLOB `token_id` for the exact outcome, or pass\n`slug`/`market_id` plus `outcome` and Foresea will resolve the token id from the\npublic market record. Limit orders use `quantity` as shares. Market-buy orders\nuse `max_cost` as USD spend when supplied and remain blocked unless\n`FORESEA_ALLOW_MARKET_ORDERS=true`.\n\nAfter submission, use the audit ID returned by `/trading/orders` to reconcile\nthe current venue state instead of assuming a submission was filled. The trade\nterminal also exposes this flow, including an explicit `CANCEL OPEN ORDER`\nconfirmation before it cancels a remaining resting order.\n\n#### Durable Trade Runs and scheduled reconciliation\n\nNew terminal submissions use a durable `/trading/runs` record: Foresea saves a\nvalidated order plan, requires a second exact confirmation to execute that saved\nplan, and atomically claims it before contacting a venue. This prevents duplicate\norders from concurrent tabs or Cloud Run instances. Run state follows the linked\naudit order when a fill, cancellation, or rejection is reconciled.\n\n#### Real-money guardrails\n\nEvery live submission now passes a second server-side preflight immediately\nbefore the venue call. It fails closed when Foresea cannot obtain a fresh market\nquote and a current portfolio snapshot, or when any of these limits would be\ncrossed:\n\n- Foresea hard caps: per-order notional, trailing-day worst-case risk budget,\n per-market exposure, outstanding orders, quote deviation, quote age, and a\n duplicate-order cooldown.\n- User controls at `GET`/`PUT /trading/guardrails`: users may set stricter\n limits or pause all new live orders, but cannot increase the platform caps.\n- `FORESEA_TRADING_KILL_SWITCH=true`: blocks every new live submission without\n touching reconciliation or cancellations.\n- A no-cache market quote is checked against the limit price. Buy limits cannot\n be above the configured collar and sell limits cannot be below it. A live\n balance/position snapshot must support the order and exposure cap.\n\nThe trailing-day budget is deliberately **worst-case notional newly risked**,\nnot a misleading synthetic P&L figure. Filled positions are measured from the\nvenue portfolio snapshot before a new order; exact realized daily P&L remains a\nseparate accounting/reporting concern. Guardrail passes, blocks, policy changes,\nand reconciled fill/rejection/cancellation transitions are appended to\n`GET /trading/guardrails/events` without credentials or order payloads. Configure\nthe existing `SMTP_*` and `ALERT_*` settings to receive operator emails for\nsubmission-unknown, rejection, fill, and platform-kill-switch events.\n\nProduction ceilings are environment variables; conservative defaults apply when\nthey are omitted:\n\n```text\nFORESEA_TRADING_KILL_SWITCH=false\nFORESEA_MAX_DAILY_RISK_NOTIONAL=100\nFORESEA_MAX_MARKET_EXPOSURE_NOTIONAL=50\nFORESEA_MAX_OPEN_ORDERS=5\nFORESEA_MAX_PRICE_DEVIATION_BPS=300\nFORESEA_MAX_QUOTE_AGE_SECONDS=20\nFORESEA_ORDER_COOLDOWN_SECONDS=60\n```\n\nThe terminal requires a Polymarket `slug` or `market_id` for real execution so\nForesea can independently obtain a fresh market quote; a raw CLOB token ID alone\nis insufficient for this safety check.\n\nTo enable the read-only scheduled reconciler, generate one high-entropy service\ntoken and store the same value as Cloud Run's `TRADING_RECONCILIATION_TOKEN` and\nthe GitHub Actions secret of that name. This is an operator token, not a user\ncredential and not an encryption key. The `Trading reconciliation` workflow then\ncalls the hidden endpoint every 15 minutes, bounded by\n`TRADING_RECONCILIATION_MAX_ORDERS` (default `25`, hard maximum `100`). The job\nonly fetches the current state of already-submitted venue order IDs; it cannot\nplace, amend, or cancel an order.\n\n#### Operator launch-readiness check\n\nAfter deploying the trading revision, use the same narrowly scoped reconciliation\ntoken to read its non-sensitive configuration report:\n\n```bash\ncurl https://foresea.ink/internal/trading/readiness \\\n -H \"X-Trading-Reconciliation-Token: $TRADING_RECONCILIATION_TOKEN\"\n```\n\nThe report confirms the configured KMS resource, durable store client,\nreconciliation-token presence, valid hard caps, live-execution gates, and whether\nthe retired shared encryption key is still present. It does not expose key names,\ntokens, credentials, or account data. It also cannot prove Cloud KMS IAM, that\nthe GitHub Actions secret matches, or that an exchange account can trade; verify\nthose separately during the invite-only smoke test.\n\nDeploy the `TradingOrder` index in `index.yaml` before enabling the scheduler:\n\n```bash\ngcloud datastore indexes create index.yaml --project \n```\n\n### Request fields\n\nRequired:\n\n- `question`: forecasting question, such as `\"Will X happen by date Y?\"`,\n `\"Who will win X?\"`, `\"What will X be?\"`, or `\"When will X happen?\"`.\n\nOptional:\n\n- `question_type`: `binary`, `multiple_choice`, `numeric`, or `date`. If omitted,\n the model attempts to infer the type.\n- `options`: answer choices for `multiple_choice` questions.\n- `description`: extra context for the question.\n- `resolution_criteria`: how the question should resolve or be measured.\n- `categories`: list of topic labels.\n- `news_articles`: caller-supplied evidence articles. If provided, automatic\n evidence retrieval is skipped.\n- `attach_evidence`: defaults to `true`. When true and `news_articles` is empty,\n the API fetches current evidence from GDELT, Google News RSS, and Stooq.\n- `evidence_top_k`: number of evidence articles to attach, capped by the server.\n- `market_platform`: prediction market venue such as `Polymarket`, `Kalshi`,\n `Manifold`, or `Metaculus`.\n- `market_url`: URL for the market being analyzed.\n- `market_outcome`: outcome whose market price is supplied. Defaults to `Yes`\n for binary markets.\n- `market_probability`: current market-implied probability for\n `market_outcome`. Use `0.42` or `42`; the API normalizes percentages.\n- `variant`: prompt variant. Defaults to `variant0_neutral_baseline`.\n- `created_time`, `publish_time`, `resolve_time`, `days_open`: optional\n forecasting metadata.\n- `openrouter_api_key` + `openrouter_model`: run the forecast on your own model\n instead of the server default (see \"Bring your own model\" below).\n- `provider_base_url`: optional OpenAI-compatible `/chat/completions` endpoint to\n use with your key/model instead of OpenRouter. Must be public HTTPS.\n\n### Bring your own model\n\nBy default `/predict` runs on the server's hosted model. To use your own:\n\n- **Via OpenRouter** — pass `openrouter_api_key` and `openrouter_model` (e.g.\n `openai/gpt-4o`, `anthropic/claude-sonnet-4-5`). The request is proxied through\n OpenRouter.\n- **Via any OpenAI-compatible endpoint** — also pass `provider_base_url` (e.g.\n `https://api.openai.com/v1` or `https://api.openai.com/v1/chat/completions`)\n with the matching `openrouter_model` (here just the provider's model ID, e.g.\n `gpt-4o`) and your key. Foresea normalizes `/v1` base URLs to\n `/v1/chat/completions` internally.\n\nFor safety, `provider_base_url` must be public HTTPS; loopback, private,\nlink-local, and cloud-metadata hosts are rejected. In the web app, the sidebar's\n**\"Use your own model\"** panel exposes the provider, endpoint, key, and model.\n\n```bash\ncurl -X POST https://foresea.ink/predict \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"question\": \"Will X happen by 2027?\",\n \"question_type\": \"binary\",\n \"openrouter_api_key\": \"YOUR_KEY\",\n \"openrouter_model\": \"gpt-4o\",\n \"provider_base_url\": \"https://api.openai.com/v1/chat/completions\"\n }'\n```\n\n### Self-hosted vLLM\n\nSCADS AI already exposes Foresea's default models through an OpenAI-compatible\nhosted endpoint. Use vLLM only when you need direct control over checkpoint,\nquantization, throughput, or serving hardware.\n\nStart a local vLLM OpenAI-compatible server:\n\n```bash\nVLLM_API_KEY=token-abc123\nvllm serve Qwen/Qwen3-32B \\\n --host 0.0.0.0 \\\n --port 8001 \\\n --api-key \"$VLLM_API_KEY\" \\\n --generation-config vllm\n```\n\nThen point Foresea at the configured `qwen3-32b-vllm` model:\n\n```bash\nVLLM_API_KEY=token-abc123 PYTHONPATH=src analyze-llm-rationale smoke-test \\\n --model qwen3-32b-vllm\n\nVLLM_API_KEY=token-abc123 PYTHONPATH=src analyze-llm-rationale serve \\\n --model qwen3-32b-vllm \\\n --variant variant0_neutral_baseline \\\n --port 8080\n```\n\nFor production, run Foresea and vLLM as separate services. Foresea's public\nbring-your-own endpoint still requires public HTTPS for `provider_base_url`;\nprivate or loopback vLLM URLs are intended for trusted server-side config.\n\n### Binary request\n\n```bash\ncurl -X POST https://foresea.ink/predict \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"question\": \"Will the Federal Reserve cut interest rates at least once before September 30, 2026?\",\n \"question_type\": \"binary\",\n \"market_platform\": \"Polymarket\",\n \"market_probability\": 42\n }'\n```\n\n### Multiple-choice request\n\n```bash\ncurl -X POST https://foresea.ink/predict \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"question\": \"Who will win the 2026 Formula 1 drivers championship?\",\n \"question_type\": \"multiple_choice\",\n \"options\": [\"Max Verstappen\", \"Lando Norris\", \"Charles Leclerc\", \"Lewis Hamilton\", \"Other\"],\n \"attach_evidence\": false\n }'\n```\n\n### Numeric request\n\n```bash\ncurl -X POST https://foresea.ink/predict \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"question\": \"What will US CPI inflation be in December 2026?\",\n \"question_type\": \"numeric\",\n \"resolution_criteria\": \"Use the year-over-year CPI-U inflation rate for December 2026.\"\n }'\n```\n\n### Request with caller-provided evidence\n\n```bash\ncurl -X POST https://foresea.ink/predict \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"question\": \"Will Company X report positive net income in Q4 2026?\",\n \"description\": \"Resolve using the company earnings release.\",\n \"resolution_criteria\": \"Yes if reported GAAP net income is positive.\",\n \"attach_evidence\": false,\n \"news_articles\": [\n {\n \"title\": \"Company X raises full-year guidance\",\n \"source\": \"Example Business News\",\n \"url\": \"https://example.com/company-x-guidance\",\n \"publish_date\": \"2026-05-29\",\n \"summary\": \"Company X raised revenue guidance and reported margin expansion.\"\n }\n ]\n }'\n```\n\n### Python client example\n\n```python\nimport requests\n\npayload = {\n \"question\": \"Will the Federal Reserve cut interest rates at least once before September 30, 2026?\",\n \"question_type\": \"binary\",\n \"attach_evidence\": True,\n \"evidence_top_k\": 3,\n \"market_platform\": \"Polymarket\",\n \"market_probability\": 42,\n}\n\nresponse = requests.post(\n \"https://foresea.ink/predict\",\n json=payload,\n timeout=180,\n)\nresponse.raise_for_status()\nprediction = response.json()\n\nprint(prediction[\"predicted_answer\"], prediction[\"confidence\"])\nprint(prediction[\"model_rationale\"])\nif prediction.get(\"market_analysis\"):\n print(prediction[\"market_analysis\"][\"summary\"])\nfor source in prediction[\"evidence_sources\"]:\n print(source[\"source\"], source[\"url\"])\n```\n\n### Response fields\n\n- `question_type`: detected or requested type: `binary`, `multiple_choice`,\n `numeric`, or `date`.\n- `predicted_answer`: `\"Yes\"`, `\"No\"`, the top multiple-choice option, or the\n median numeric/date estimate.\n- `confidence`: model confidence as a number from 0 to 1 for binary and\n multiple-choice forecasts; `null` for numeric/date forecasts.\n- `options`: per-option probabilities for multiple-choice forecasts.\n- `range_forecast`: `p10`, `p50`, `p90`, and optional `unit` for numeric/date\n forecasts.\n- `rationale`: model-generated explanation.\n- `model_rationale`: alias for the model-generated explanation, intended for API\n clients.\n- `evidence_sources`: compact source list with article title, URL, publication\n date, and relevance score.\n- `evidence_articles`: full evidence records attached to the prompt.\n- `evidence_error`: retrieval error message, or `null` when evidence retrieval\n succeeds.\n- `market_analysis`: optional comparison against a supplied market price:\n `market_probability`, `model_probability`, `edge`, `stance`, and a short\n summary. `edge` is `model_probability - market_probability`.\n\n## Repository Contents\n\n- `src/analyzing_llm_rationale/`: packaged inference, provider, validation, and CLI logic.\n- `configs/`: model and rationale-variant definitions.\n- `prompts/`: system prompt plus the configured rationale, control, ablation,\n and no-evidence prompt variants.\n- `scripts/`: evaluation, recovery, SHAP, perturbation, plotting, market-data,\n and utility scripts.\n- `slurm/`: HPC launchers for the variant/temperature sweeps.\n- `results/`: model outputs and run metadata.\n- `analysis/`: aggregate metric tables and rationale-analysis outputs.\n- `paper/`: paper figures, Draw.io sources, PDFs, and qualitative case studies.\n- `tests/`: unit tests for the package and metric parsing.\n\nSee `ARTIFACT_MANIFEST.md` for the submission checklist and file-level notes.\n\n## Install\n\n```bash\npython -m venv .venv\nsource .venv/bin/activate\npython -m pip install -e \".[dev,serve,pipeline]\"\n```\n\nUse `.[dev]` for linting and unit tests. Add `.[analysis]` when regenerating\nplots, metrics tables, or SHAP analyses. Add `.[trading]` for local exchange\norder preview/execution development.\n\n## Prompt Variants\n\nConfigured variants live in `configs/variants.yaml` and map directly to prompt\nfiles under `prompts/`.\n\n- `variant0` is the neutral baseline.\n- `variant1` through `variant8` cover the original rationale attribute prompts.\n- `variant9` through `variant14` add scratchpad, length-matched, structural, and\n combined temporal/credibility controls.\n- `variant15_neutral_no_rationale` and `variant16_no_evidence_neutral` support\n ablations for rationale and evidence effects.\n\nWhen adding a variant, update `configs/variants.yaml`, add the prompt file, and\nrun a bounded smoke test:\n\n```bash\nPYTHONPATH=src analyze-llm-rationale run-batch \\\n --variant \\\n --max-records 3\n```\n\n## Quick Validation\n\n```bash\nPYTHONPATH=src python -m analyzing_llm_rationale validate-dataset\npython -m unittest discover -s tests\nruff check src tests\n```\n\n`PYTHONPATH=src` is useful when the repository has not been installed yet or an\nolder user-local install shadows the working tree.\n\nRun the full suite with Python 3.10+ and the relevant extras installed. The\nserver, RAG, tracking, and trading tests import optional dependencies from\n`serve`, `pipeline`, `analysis`, and `trading`.\n\n## Primary Entry Point\n\nRun the variant 3 pipeline with the packaged CLI:\n\n```bash\nanalyze-llm-rationale run-batch --variant variant3_reasoning_type\n```\n\nFor a remote OpenAI-compatible provider:\n\n```bash\nexport PROVIDER_API_KEY=your_token\nanalyze-llm-rationale run-batch --variant variant3_reasoning_type --model llama-3.3-70b-instruct\n```\n\nIf you do not want to install the package into the environment, invoke it directly:\n\n```bash\nPYTHONPATH=src python -m analyzing_llm_rationale run-batch --variant variant3_reasoning_type\n```\n\nUseful options:\n\n- `--variant variant6_step_by_step_reasoning`: choose the prompt/output contract.\n- `--model qwen2.5-7b-instruct`: choose a configured model definition.\n- `--temperature 0.7`: control generation temperature and output directory.\n- `--max-records 10`: process only a bounded number of records.\n- `--reprocess-nulls`: rerun existing rows with `predicted_answer = null`.\n- `--drop-article-text`: remove raw article text from prompts before inference.\n- `--device auto`: select `cuda` when available, otherwise `cpu`.\n- `verify-results --variant ...`: verify completeness, duplicates, malformed rows, and missing IDs.\n- `validate-dataset`: validate the dataset schema before a run.\n\n## Foresea Autoresearch\n\nForesea has a Karpathy-style autoresearch harness for prompt experiments: edit\none candidate prompt, run a fixed benchmark slice, score one metric, and append\nan auditable experiment log. The research surface is\n`autoresearch/candidate_prompt.txt`; agent instructions live in\n`autoresearch/program.md`. The default `--model gpt-oss-120b` uses the\nSCADS-hosted OpenAI-compatible endpoint from `configs/models.yaml`\n(`SCADS_AI_API_KEY` or `SCADS_AI_API_KEY.txt`).\n\nRun one candidate experiment:\n\n```bash\nPYTHONPATH=src python -m analyzing_llm_rationale autoresearch \\\n --model gpt-oss-120b \\\n --candidate-prompt-path autoresearch/candidate_prompt.txt \\\n --max-records 50 \\\n --metric brier_score\n```\n\nCompare against a baseline and promote only if the candidate improves:\n\n```bash\nPYTHONPATH=src python -m analyzing_llm_rationale autoresearch \\\n --model gpt-oss-120b \\\n --candidate-prompt-path autoresearch/candidate_prompt.txt \\\n --baseline-results-path results/GPT-OSS-120B/temperature_00/results_variant0_neutral_baseline.json \\\n --promote-to prompts/variant0_neutral_baseline.txt \\\n --max-records 50 \\\n --metric brier_score \\\n --min-delta 0.001\n```\n\nEach run writes `analysis/autoresearch/runs//score.json` and appends a\nmachine-readable row to `analysis/autoresearch/experiments.jsonl`.\n\n## Reproducing Core Outputs\n\nValidate an existing result file:\n\n```bash\nPYTHONPATH=src python -m analyzing_llm_rationale verify-results \\\n --model qwen2.5-7b-instruct \\\n --variant variant3_reasoning_type \\\n --temperature 0.0 \\\n --temperature-tag temperature_000\n```\n\nRegenerate aggregate metrics from `results/`:\n\n```bash\npython scripts/evaluate_metrics.py\n```\n\nRun the DuckDB SQL analytics suite over the real Metaculus-style dataset and\nsaved model outputs:\n\n```bash\npython scripts/sql_analytics.py \\\n --db analysis/forecasting_analytics.duckdb \\\n --ingest --replace \\\n --output-dir analysis/sql_analytics\n```\n\nThis writes a markdown report plus one CSV per query for 10 medium-level SQL\nproblems: model accuracy, best variants, calibration bins, Brier score,\nconsensus/disagreement cases, prompt lift over baseline, temperature sensitivity,\noverconfident errors, and category difficulty.\n\nRun the LangChain-powered news retrieval wrapper:\n\n```bash\nPYTHONPATH=src analyze-llm-rationale fetch-and-rank \\\n --question \"Will X happen by date Y?\" \\\n --source gdelt \\\n --source google-news \\\n --source stooq \\\n --top-k 5\n```\n\nThe news pipeline uses LangChain for a query-planning step, article\nsummarization, and embedding-based relevance ranking before inference. Evidence\nsources are configurable with `--source` for the CLI and `--evidence-source`\nwhen serving the API.\n\nRun or schedule the Prefect DAG for RSS/news fetch, inference, and DuckDB\nlogging:\n\n```bash\n# One question\npython flows/forecasting_flow.py --question-id 124 --top-k 5\n\n# Small batch from the dataset\npython flows/forecasting_flow.py --limit 3 --top-k 5\n\n# Daily scheduled deployment at 06:00 UTC\nprefect server start\npython flows/forecasting_flow.py --deploy --limit 3 --cron \"0 6 * * *\"\n```\n\nRegenerate paper figures after metrics are present:\n\n```bash\npython scripts/plot_model_variant_metric_heatmap.py\npython scripts/plot_variant_delta_from_v0.py\npython scripts/plot_temperature_frontier.py\npython scripts/plot_frs_ablation_slopegraph.py\npython scripts/plot_uncertainty_language_calibration_disconnect.py\npython scripts/plot_shap_importance_attribute_gaps.py\n```\n\n## Scripts\n\nCommon runner and verification commands:\n\n- `python scripts/run_variant.py --variant variant5_key_conditions`\n- `python scripts/run_variant.py --variant variant3_reasoning_type --temperature 0.7 --temperature-tag temperature_07`\n- `python scripts/run_variant.py --variant variant4_credibility --model llama-3.3-70b-instruct`\n- `python scripts/verify_results.py --variant variant3_reasoning_type`\n- `python download_qwen_model.py`\n- `python check_local_inference.py`\n\nRepo layout:\n\n- `scripts/`: modular runner entrypoint\n- `slurm/`: batch launchers\n\nAuditability:\n\n- Each run writes `run_metadata_.json` next to the results file.\n- Metadata includes provider, normalized provider endpoint, model key, resolved model identifier, tem","changelogMarkdown","videoKey","requiredSecretsJson","publishedAt","priceCents","trialDays","iconType","image","upstream",{"_99":100,"_101":102,"_103":104,"_105":106,"_107":108,"_109":110},"requiredSecrets",[],"tags",[],"assets",[],"stats",{"_97":59,"_98":59},"ratingAvg","reviewCount","topics",[],"license","MIT","language","Python","registryName","ink.foresea/forecasting","homepage","https://foresea.ink","packages",[],"ENV",{"_113":114,"_115":116,"_117":118},"ENVIRONMENT","production","TURNSTILE_SITE_KEY","0x4AAAAAAB8-rEI2gNDf_QhU","SENTRY_DSN","https://45fd1229f78089d5c3ee78be128030e1@o4510583261757440.ingest.de.sentry.io/4510584253448272"]