Polymarket BTC 5-Minute Prediction Model

A quantitative trading model for predicting directional movements in Polymarket BTC 5-minute contracts. The model captures the latency arbitrage edge between Binance (real price) and Polymarket (prediction market).

Polymarket prices often lag behind real BTC prices by 10-30 seconds. This creates temporary inefficiencies:

• BTC moves first (ground truth from Binance)
• Polymarket reacts with a delay (retail sentiment + liquidity lag)

Core Prediction Problem: Given a market around 0.5 probability, which side (YES/NO) is more likely to reach 0.70-0.75 within 60-120 seconds?

1. Data Collection: REST API clients for Binance (OHLCV) and Polymarket (price snapshots)
2. Feature Engineering: Returns, volatility, momentum, divergence signals
3. Market Regimes: K-Means clustering to identify market conditions (calm, divergent, trending, choppy)
4. Binary Prediction: LightGBM classifier to predict YES/NO winner with confidence scores
5. Validation: Time-series split with proper data leakage prevention and regime-specific metrics

Poymarket Algo/
├── src/
│   ├── data_collection/          # API clients and data merging
│   │   ├── binance_client.py       # Binance OHLCV fetcher
│   │   ├── polymarket_client.py    # Polymarket price snapshots
│   │   └── data_aligner.py         # Align and synchronize data
│   ├── feature_engineering/      # Feature computation
│   │   ├── feature_builder.py      # Core features (returns, volatility, momentum)
│   │   └── divergence_signals.py   # Latency arbitrage features
│   ├── labeling/                 # Label creation
│   │   └── label_creator.py        # Option 2 labeling with signal filtering
│   ├── models/                   # Model architectures
│   │   ├── regime_detector.py      # K-Means clustering
│   │   └── predictor.py            # LightGBM classifier (exported to regime_detector.py)
│   ├── training/                 # Training pipeline
│   │   ├── train_pipeline.py       # Time-series split, scaling, training
│   │   └── metrics.py              # Evaluation and metrics
│   └── utils/
│       ├── config.py               # Configuration loader
│       └── logger.py               # Logging setup
├── notebooks/
│   ├── 01_data_collection.ipynb    # Fetch and explore raw data
│   ├── 02_feature_exploration.ipynb # Feature engineering and analysis
│   └── 03_model_validation.ipynb    # Training, validation, and backtesting
├── config/
│   └── params.yaml                 # All hyperparameters and configuration
├── data/
│   ├── raw/
│   │   ├── binance/                # Raw OHLCV data (.jsonl)
│   │   └── polymarket/             # Raw market snapshots (.jsonl)
│   ├── processed/                  # Aligned and cleaned data
│   └── features/                   # Feature-engineered datasets
├── requirements.txt                # Python dependencies
└── README.md                       # This file

1. Clone repository and navigate to directory:

cd "/Users/shreyashsingh/Poymarket Algo"

2. Create virtual environment (recommended):

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

pip install -r requirements.txt

from src.data_collection.binance_client import BinanceClient
from src.data_collection.polymarket_client import PolymarketClient

# Fetch Binance data (7 days of 1-minute OHLCV)
binance_client = BinanceClient()
df_binance = binance_client.fetch_klines_range(days_back=7, interval='1m')
binance_client.save_klines(df_binance)

# Fetch Polymarket data (stream snapshots, ~1 snapshot per second)
polymarket_client = PolymarketClient()
market = polymarket_client.find_btc_5min_market()
if market:
    df_polymarket = polymarket_client.stream_market_snapshots(
        market['id'], duration_seconds=3600
    )
    polymarket_client.save_market_snapshots(df_polymarket, market['id'])

from src.data_collection.data_aligner import DataAligner
from src.feature_engineering.feature_builder import FeatureBuilder
from src.feature_engineering.divergence_signals import DivergenceBuilder

# Align Binance and Polymarket data
aligned_df = DataAligner.load_and_align(
    df_binance, df_polymarket,
    interval='5s',
    forward_fill=True
)

# Build core features
features = FeatureBuilder.build_features(aligned_df)

# Add divergence features (critical for edge!)
div_features = DivergenceBuilder.build_divergence_features(
    btc_returns=features[['btc_return_5', 'btc_return_15', 'btc_return_60']],
    btc_volatility=features[['btc_volatility_15', 'btc_volatility_60']],
    btc_momentum=features[['btc_momentum_15', 'btc_momentum_60']],
    pm_returns=features[['pm_return_5', 'pm_return_15', 'pm_return_60']],
    pm_momentum=features[['pm_momentum_15', 'pm_momentum_60']],
    pm_price=features['pm_price']
)

# Combine all features
all_features = pd.concat([features, div_features], axis=1)
all_features, _ = FeatureBuilder.remove_initial_nans(all_features)

from src.labeling.label_creator import LabelCreator
from src.training.train_pipeline import TrainingPipeline

# Create labels: y=1 if YES side reaches 0.75, y=0 if NO side reaches 0.25
labels, high_signal_mask, weights = LabelCreator.prepare_labels_and_weights(
    all_features,
    target_variable='pm_price',
    lookforward_rows=18,  # ~90 seconds at 5s resolution
    filter_high_signal=True
)

# Prepare and split data
pipeline = TrainingPipeline()
data = pipeline.prepare_data(all_features, labels, weights, train_size=0.70)

# Train K-Means + LightGBM
models = pipeline.train(data, use_regime=True)

# Make predictions
preds, probs = pipeline.predict(data['X_test'])

# Evaluate with detailed metrics
from src.training.metrics import ModelEvaluator
ModelEvaluator.print_evaluation_report(
    data['y_test'].values, preds, probs,
    dataset_name="Test"
)

The DivergenceBuilder computes several divergence metrics:

• Raw Divergence: BTC_return - PM_return

  • When positive: BTC up, PM hasn't caught up → BUY YES
  • When negative: BTC down, PM still high → SELL YES

• Normalized Divergence: Scales divergence by BTC volatility

  • Accounts for regime differences (smooth vs. choppy markets)

• Lagged Momentum: BTC_momentum(t-1) - PM_momentum(t)

  • Captures the "BTC leading, PM following" pattern

K-Means identifies 4 market regimes:

Option 2 Labeling:

• y = 1 if max(PM_price[next_90s]) >= 0.75 AND price goes up
  • YES side wins: could have profitably bought YES
• y = 0 if min(PM_price[next_90s]) <= 0.25 AND price goes down
  • NO side wins: could have profitably sold YES
• y = NaN if neither
  • No clear signal: low-conviction move, excluded from training

Hybrid Weighting:

• High-signal samples (meaningful BTC move, near 0.5, decent volume): weight = 2.0
• Low-signal samples (noise, far from equilibrium): weight = 0.5
• Execution: only trade when P(YES win) > 0.65 or P(YES win) < 0.35

[0 -----70%-------- 85%-- 100%]
   TRAIN      VAL   TEST

Critical:
- No shuffling (preserve time order)
- Scaler fit on TRAIN only
- K-Means fit on TRAIN only
- Each fold moves forward in time

Edit config/params.yaml to customize:

# Feature windows
feature_engineering:
  return_windows: [5, 15, 60]
  volatility_windows: [15, 60]
  momentum_windows: [15, 60]

# Labeling
labeling:
  lookforward_rows: 18  # 90s at 5s resolution
  upper_threshold: 0.75   # YES target
  lower_threshold: 0.25   # NO target

  # High-signal filters
  btc_return_threshold: 0.0015  # 0.15% min move
  pm_distance_threshold: 0.05   # Max distance from 0.5

# Model
lightgbm:
  num_leaves: 31
  max_depth: 5
  learning_rate: 0.1

# Evaluation
evaluation:
  confidence_threshold: 0.65  # Min confidence for execution
  min_accuracy: 0.60          # Target accuracy
  min_precision: 0.65         # Precision target

✅ In-sample accuracy: >60% on high-signal samples ✅ Out-of-sample accuracy: >58% on never-seen test data ✅ Regime-specific edge: Accuracy >60% in "Divergent" + "Trending" regimes ✅ High-confidence precision: >70% precision when P > 0.65 or P < 0.35 ✅ Statistical significance: Win rate > 55% (95% CI doesn't cross 50%)

Only after meeting these criteria should you build a live trading bot.

1. Check divergence features are being computed correctly
2. Verify label definition matches your strategy (not too strict)
3. Increase lookforward window (90s might be too short)
4. Filter for high-signal setups more aggressively

1. Verify timestamps are synchronized: aligned_df['timestamp'].diff().describe()
2. Check Polymarket data isn't stale: aligned_df['pm_price'].value_counts()
3. Increase forward-fill limit if many gaps: data_aligner.forward_fill_pm_price(max_gap_seconds=10)

1. Add more L1/L2 regularization in LightGBM config
2. Reduce num_leaves or max_depth
3. Use sample weighting to reduce noise (already implemented)
4. Expand test set size

1. Data Exploration (01_data_collection.ipynb):

   • Fetch 3-7 days of data
   • Analyze distributions and missingness
   • Validate alignment

2. Feature Analysis (02_feature_exploration.ipynb):

   • Check feature correlations
   • Verify divergence signals capture lead relationship
   • Visualize regimes

3. Model Validation (03_model_validation.ipynb):

   • Train model with different lookforward windows
   • Analyze feature importance
   • Evaluate regime-specific performance
   • Measure statistical significance

4. Iterate:

   • Adjust thresholds if needed
   • Try longer prediction windows
   • Add more divergence variants

• Latency Arbitrage: Exploiting timing differences between markets
• Market Microstructure: Understanding order flow, spreads, and information dissemination
• Cross-Exchange Prediction: Using one market to predict another
• LightGBM: Fast gradient boosting for time-series
• Regime Detection: K-Means clustering for market state identification

For issues or questions:

1. Check the Jupyter notebooks (01-03) for detailed examples
2. Review the docstrings in each module
3. Examine config/params.yaml for parameter meanings
4. Enable verbose logging with level=logging.DEBUG

This is a research project for educational and authorized trading testing only. Past performance does not guarantee future results. Always validate models on out-of-sample data before any real trading.

Created: 2024 Model Type: Supervised Binary Classification + Regime Detection Data Frequency: 5-second buckets Prediction Horizon: 90 seconds Target Markets: Polymarket BTC 5-minute contracts