Skip to content

feat(optimizers): add turnover-aware portfolio optimizer#466

Merged
warren618 merged 1 commit into
HKUDS:mainfrom
Robin1987China:portfolio-turnover-aware-optimizer
Jul 10, 2026
Merged

feat(optimizers): add turnover-aware portfolio optimizer#466
warren618 merged 1 commit into
HKUDS:mainfrom
Robin1987China:portfolio-turnover-aware-optimizer

Conversation

@Robin1987China

Copy link
Copy Markdown
Contributor

Summary

Adds a new opt-in turnover_aware portfolio optimizer under backtest/optimizers/. It maximizes a mean-variance utility minus an L1 penalty on the change in weights relative to the previously held weights, solved with SLSQP on the long-only simplex.

This is the first step of Portfolio Studio (#456), scoped per the maintainer guidance there: turnover-aware optimization first, existing four schemes untouched, config-driven parameters, changes confined to backtest/.

Why

The engine already loads optimizers dynamically via config.optimizerbacktest.optimizers.<name>.optimize(ret, pos, dates, **optimizer_params) (backtest/engines/base.py), with four schemes today (mean-variance, risk parity, equal-volatility, max-diversification). None of them account for turnover cost: because weights are recomputed on a rolling window each rebalance, an optimizer can churn positions every bar, so an optimized backtest can overstate net returns once real transaction costs apply. This optimizer lets the objective trade off expected risk-adjusted return against turnover, and reports realized turnover so cost-adjusted performance is visible.

Changes

  • backtest/optimizers/turnover_aware.py: new TurnoverAwareOptimizer(BaseOptimizer) + module-level optimize(). Objective min −wᵀμ + λ·wᵀΣw + γ·‖w − w_prev‖₁, long-only simplex via SLSQP, warm-started from prior weights, falling back to equal weight on non-convergence / NaN covariance. Prior weights are tracked over the full universe (entries and exits both count as turnover). Realized per-rebalance turnover is accumulated on the instance.
  • backtest/optimizers/__init__.py: document the new scheme.
  • tests/test_turnover_aware_optimizer.py: 12 tests.

Notes:

  • Opt-in only; default backtest behavior is unchanged (optimizer off → 1/N).
  • With turnover_penalty=0 the objective is path-independent and equals the un-penalized mean-variance-utility solution (verified against a brute-force simplex search).
  • The penalty is scale-sensitive relative to the return term; this is documented in the module docstring so callers can tune it to their data frequency.
  • No new dependencies (reuses numpy / scipy.optimize, already used by mean_variance.py).

Test Plan

  • ruff check — clean
  • pytest tests/test_turnover_aware_optimizer.py — 12 passed
  • pytest full suite (excl e2e) — 4895 passed, 9 skipped (2 pre-existing env failures unrelated to this change: test_factor_gate_fund_prefix, test_factor_operators::test_bottleneck_available, both reproduce on clean main)

Test coverage: zero-penalty path-independence, higher penalty lowers realized turnover, turnover monotone non-increasing across penalty levels, long-only simplex feasibility, input sign preservation, short-window / NaN / all-NaN-column graceful handling, single-asset passthrough, realized-turnover reporting.

Checklist

Refs #456

Add a new opt-in `turnover_aware` optimizer that maximizes a mean-variance
utility minus an L1 penalty on the change in weights relative to the
previously held weights, solved with SLSQP on the long-only simplex.

This trades off expected risk-adjusted return against position churn, so
optimized backtests no longer overstate net returns by rebalancing every
bar. The optimizer reuses the existing BaseOptimizer contract and the
config.optimizer / optimizer_params extension point, keeps the four existing
schemes untouched, and stays off by default. Realized per-rebalance turnover
is tracked for cost-adjusted analysis.

Refs HKUDS#456

Signed-off-by: Robin1987China <41602358+Robin1987China@users.noreply.github.com>
@warren618
warren618 merged commit 3a89001 into HKUDS:main Jul 10, 2026
1 check passed
warren618 added a commit that referenced this pull request Jul 10, 2026
- manifest guard test: ignore __pycache__ when counting zoo modules
- analyze_image: state the vision-capable-model requirement in the tool
  description; point the example path at the new uploads media root
- turnover_aware: drop the mathematically inert risk_free knob (additive
  constant on the simplex), fix the max-Sharpe docstring mislabel, document
  realized_turnover as class-API-only
- skills: surface turnover_aware in asset-allocation + strategy-generate
  so the agent can actually propose it
OXOOOOX pushed a commit to OXOOOOX/Vibe-Trading that referenced this pull request Jul 12, 2026
Fifth optimizer: mean-variance utility with an L1 turnover penalty against the previous rebalance (SLSQP, long-only simplex), so the portfolio only trades when expected improvement outweighs churn. Includes 12 seeded tests.
warren618 pushed a commit that referenced this pull request Jul 13, 2026
#478)

Refs #456. After #466, turnover_aware's realized_turnover was computed but
discarded — the engine's optimizer loader keeps only the returned positions
frame, never the optimizer instance that accumulated it.

Rather than thread the optimizer instance out of the loader lambda, this
computes turnover generically from the executed position frame the engine
already holds (calc_turnover_series: per-bar 0.5*sum|w_t - w_t-1|), so it
works for every strategy/optimizer, not just turnover_aware, and measures the
actual post-normalization weight path that drives transaction cost. Purely
additive: calc_metrics gains an optional positions arg, no existing metric
values change.

Verified locally: full backend suite green (same pre-existing unrelated
failures as main), env-var CI gate passes, ruff clean, 63/63
metrics-and-turnover tests pass. Only one calc_metrics call site
(engines/base.py), inherited by every engine.

Co-authored-by: 卫斯李 <Robin1987China@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants