Skip to content

feat: MetaTrader 5 (Exness) broker connector + mt5 market-data source#481

Merged
warren618 merged 5 commits into
HKUDS:mainfrom
StaniellG:feat/mt5-exness-connector
Jul 20, 2026
Merged

feat: MetaTrader 5 (Exness) broker connector + mt5 market-data source#481
warren618 merged 5 commits into
HKUDS:mainfrom
StaniellG:feat/mt5-exness-connector

Conversation

@StaniellG

Copy link
Copy Markdown

Goal

Add MetaTrader 5 as a first-class broker connector and market-data source, so MT5 brokers (reference: Exness) get the same treatment as the existing 11 connectors: read the account, place gated orders, and feed backtests/get_market_data from the broker's own terminal feed. Brokers: 11 → 12; market-data sources: 20 → 21.

What's included

Trading connector (agent/src/trading/connectors/mt5/, broker_sdk transport)

  • Reads: check_status, account snapshot, positions (per-row USD market_value, None when unpriceable so the gate fails closed), pending orders + optional deals, suffix-aware quotes (zero last omitted), history.
  • Writes: place_order (market / pending-limit, volume_step flooring — never round up, filling-mode negotiation from the symbol bitmask, order_check pre-flight before any send) and risk-reducing cancel_order with dual semantics: pending ticket → TRADE_ACTION_REMOVE; position ticket → opposite deal pinned to position= capped at the position's own volume (hedging-account safe). close_position exposed as a public helper.
  • Identity guard (bidirectional, every session): paper profile requires ACCOUNT_TRADE_MODE_DEMO, live requires REAL, contest rejected, configured login pinned against the terminal's. A paper profile attached to a real-money account is hard-rejected before any read or write.
  • Profiles: mt5-paper-sdk, mt5-live-sdk-readonly, mt5-paper-trade (orders.place), mt5-live-trade (orders.place.requires_mandate). Config in ~/.vibe-trading/mt5.json (0600 best-effort, password redacted and login masked in every payload).

Live-gate safety (the one change to shared gate code)

  • InstrumentType gains FOREX/CFD; AssetClass gains FOREX. No mandate schema bump — the store parses enums by value, v1 mandates load unchanged. CFD follows the OPTION precedent (no universe bucket, admitted only via explicit "cfd" allowance), so metals/indices are deny-by-default on existing mandates.
  • sdk_order_gate._normalize_notional now honors an optional, authoritative quantity_notional_usd(config, symbol, quantity) connector hook. MT5 quantities are lots (1 lot EURUSD = 100,000 EUR): pricing them as quantity × quote would under-state notional roughly contract-size-fold and silently disarm every USD cap, so when the hook exists there is deliberately no quote-product fallback. Hook-less connectors keep byte-identical behavior (existing gate tests pass untouched).
  • Defense-in-depth under the mandate: connector-level max_order_volume / max_order_notional_usd guards enforced on demo AND live; unpriceable guard input refuses the order.

Market-data loader (agent/backtest/loaders/mt5_loader.py)

  • copy_rates_range history with tz-aware UTC ranges (the MT5 API shifts naive datetimes), Exness-style suffix discovery via symbols_get (shortest-then-alpha, memoized), results keyed by the caller's original code, settled-range parquet cache participation, per-symbol failures degrade instead of raising so runner fallback engages.
  • mt5 heads FALLBACK_CHAINS["forex"]; unavailable (no Windows / no package / no terminal) → chain proceeds to akshare exactly as before.
  • get_market_data learns forex/metal routing: EUR/USD / XAUUSD.FX previously matched no _SOURCE_PATTERNS entry and fell through to tushare (broken). Fixes that, and akshare's _is_forex/_fetch_forex now accept the canonical slash form so the mt5 → akshare degradation works off-Windows too.
  • metrics.py gains mt5 annualisation columns (24×5 forex session: 260 trading days).

Packaging / docs

  • mt5 = ["MetaTrader5>=5.0.45; sys_platform == 'win32'"] extra — a no-op resolve off Windows; every import is lazy, so ubuntu CI never needs the SDK.
  • README (all 5 locales): data-source table row, forex fallback chain, full MT5 connector section (mt5.json, profile table, safety boundary, hedging note). SKILL.md source counts 20 → 21 (test-enforced). CHANGELOG under [Unreleased].

Affected areas

  • New: agent/src/trading/connectors/mt5/*, agent/backtest/loaders/mt5_loader.py, 3 test files (~1,100 lines of tests).
  • Shared-code edits (small, additive): live/mandate/model.py, live/enforcement.py, live/sdk_order_gate.py, live/registry.py, trading/profiles.py, trading/service.py, backtest/loaders/registry.py, backtest/loaders/akshare_loader.py, backtest/metrics.py, src/market_data.py, src/tools/market_data_tool.py, mcp_server.py.

Out of scope (documented follow-ups)

  • src/live/runtime/flatten.py flatten-on-halt submits via generic place_order, which hedges rather than closes on MT5 hedging accounts (net price exposure still neutralized; gross exposure caps still bind). A ticket-pinned MT5 flatten is a follow-up; documented in orders.py and the README.
  • A generic connector configure CLI flow for broker_sdk connectors (all 10 SDK connectors, including this one, are configured via hand-written JSON today).
  • FX normalization of non-USD account currencies in the gate (same status as the existing HK/CN note in _normalize_notional).

Live-broker risk & rollback

  • No existing connector's behavior changes; the only shared write-path edit is the sizing hook, whose absent-hook path is byte-identical (verified by the untouched test_sdk_order_gate.py suite plus new hook tests).
  • Live MT5 orders require: a real-account profile + a mandate explicitly allowing forex/cfd + kill switch clear + lot-aware USD caps + connector size guards + order_check pre-flight. Demo orders still pass connector size guards.
  • Rollback/halt: kill switch blocks new live orders (registration- and call-time); cancels stay available and are audit-logged; deleting mt5.json or the profile selection fully disables the connector.

Test plan

  • pytest --ignore=agent/tests/e2e_backtest --ignore=agent/tests/test_e2e_harness_v2.py -q5267 passed (the 2 remaining failures are pre-existing Windows-environment issues in test_qveris_tool (POSIX chmod assert) and tests/factors (symlink privilege), both green on ubuntu).
  • Safety suites green: test_sdk_order_gate, test_mandate_enforcement, test_killswitch_blocks_orders, test_readonly_default, test_sdk_connectors, test_mandate_model — plus new test_mandate_forex.py, test_mt5_connector.py (85 tests incl. gate-hook end-to-end, identity-guard matrix, request-shape capture), test_mt5_loader.py.
  • Gates: tools/ci_env_var_gate.py and tools/ci_grep_gates.sh pass.
  • Windows E2E: with the [mt5] extra installed and a terminal present but not logged in, the loader attempts a real attach, reports (-6, Terminal: Authorization failed), degrades to akshare, and connector check reports a clean "not configured" error; all four profiles render in vibe-trading connector list. Demo-account order round-trip requires a logged-in Exness demo terminal (per the contributor guide, live trading is never part of PR validation).

Per the contributor guide: no live trading was run; all validation used fakes, unit suites, and an unauthorized terminal attach.

Staniell added 5 commits July 13, 2026 00:05
Groundwork for the MetaTrader 5 connector:

- InstrumentType gains FOREX and CFD; AssetClass gains FOREX. No schema
  bump: the store parses enums by value, so v1 mandates keep loading.
  CFD follows the OPTION precedent (no universe bucket; admitted only
  via an explicit cfd instrument allowance).
- _INSTRUMENT_ASSET_CLASS maps FOREX to its bucket; forex universe
  floors fail closed (no loader market wired).
- sdk_order_gate._normalize_notional now defers to an optional,
  authoritative quantity_notional_usd connector hook. Lot-sized
  quantities (1 lot EURUSD == 100k EUR) must never be priced as
  quantity x quote — that under-states notional ~contract-size-fold
  and would silently disarm every USD cap. Without the hook the legacy
  path is byte-identical.

Signed-off-by: staniell <giostaniell12@gmail.com>
New broker_sdk connector talking to a locally running MT5 terminal via
the Windows-only MetaTrader5 package (lazy-imported; ubuntu CI never
needs it). Connectors: 11 -> 12.

- Reads: check_status, account, positions (USD market_value per row,
  None-when-unpriceable so the gate fails closed), pending orders +
  optional deals, suffix-aware quote (zero last omitted), history.
- Writes: place_order (market/pending-limit, volume_step flooring,
  filling-mode negotiation, order_check pre-flight) and risk-reducing
  cancel_order that removes a pending ticket OR closes a position by
  ticket via an opposite deal pinned to position= (hedging-safe).
- Safety: bidirectional identity guard on every session (paper profile
  requires DEMO trade_mode, live requires REAL, contest rejected,
  login pinned); connector-level max_order_volume /
  max_order_notional_usd guards on demo AND live; live orders route
  through the mandate gate with the lot-aware quantity_notional_usd
  sizing hook; per-symbol forex/CFD order classification.
- Profiles: mt5-paper-sdk, mt5-live-sdk-readonly, mt5-paper-trade,
  mt5-live-trade (requires_mandate). Config in ~/.vibe-trading/mt5.json
  (0600 best-effort, redacted in every payload).

Signed-off-by: staniell <giostaniell12@gmail.com>
Backtests and get_market_data can now use a local MT5 terminal feed
(e.g. Exness) with the broker exact symbols. Sources: 20 -> 21.

- New backtest/loaders/mt5_loader.py: copy_rates_range history with
  tz-aware UTC ranges, Exness-style suffix discovery via symbols_get
  (shortest-then-alpha, memoized), results keyed by the original input
  code, settled-range parquet cache participation, per-symbol failures
  degrade instead of raising so runner fallback engages.
- Routing: mt5 heads FALLBACK_CHAINS[forex]; forex/metal patterns
  (EUR/USD, XAUUSD.FX) added to market_data._SOURCE_PATTERNS -- they
  previously fell through to tushare and failed.
- akshare _is_forex/_fetch_forex now accept the canonical slash form,
  so the mt5 -> akshare degradation actually works for EUR/USD.
- metrics: mt5 annualisation columns (260 trading days, 24x5 bars).

Signed-off-by: staniell <giostaniell12@gmail.com>
- pyproject: [mt5] extra with a sys_platform == win32 marker so ubuntu
  CI and non-Windows installs are unaffected.
- README: data-source table + forex fallback chain + MetaTrader 5
  connector section documenting mt5.json, the four profiles, the
  demo/live identity guard, lot-aware mandate sizing, hedging
  close-by-ticket semantics, and the halt path.
- agent/SKILL.md + data-routing SKILL.md: sources 20 -> 21 (test-
  enforced counts), forex routing priority mt5 > akshare.
- CHANGELOG [Unreleased]: connector (11 -> 12) + data source (20 -> 21).

Signed-off-by: staniell <giostaniell12@gmail.com>
zh/ja/ko/ar gain the mt5 data-source table row, the forex/metals
fallback-chain bullet, the source-count bump (19 -> 20), and the full
translated MetaTrader 5 connector section. zh/ja/ko/ar lack the English
MCP Client Mode chapter, so the connector section anchors at the end of
the MCP Plugin section in all four (consistent placement). Also bumps
the QVeris paragraph built-in-source count to 20 in all 5 READMEs.

Signed-off-by: staniell <giostaniell12@gmail.com>
@warren618
warren618 merged commit 590c304 into HKUDS:main Jul 20, 2026
@warren618

Copy link
Copy Markdown
Collaborator

Thanks @StaniellG — merged. Red-lines all verified: the mandate gate routes mt5-live-trade through execute_live_order, the paper/live discriminator is a structural per-session trade_mode re-check (fail-closed), no order tool is MCP-exposed, and the mt5 loader is read-only/backtest-barred. Two small follow-ups worth a PR when you have a moment: drop env-selecting keys (profile) from _OVERRIDE_KEYS, and note that the loader and connector share the process-global MetaTrader5 module as a known Windows-only limitation. market_data =F/=X routing and mt5 forex routing were reconciled on merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants