This project studies how different modeling choices behave when predicting or estimating signals in highly noisy, non-stationary environments, using BTC price data purely as a proxy for a chaotic sensor signal.
Version 1 applied a Kalman filter to directly extrapolate short-horizon values and benchmarked it against simple baselines such as persistence and EMA.
Empirical results showed that persistence consistently outperformed Kalman on error metrics, while Kalman extrapolation amplified noise as horizons increased.
This behavior revealed a fundamental limitation: linear state-space models are poorly suited for direct prediction in high-frequency chaotic data.
The outcome was expected and highlighted a model-role mismatch rather than an implementation error.
Version 2 reframes the Kalman filter as a fair-value estimator, not a predictor.
Instead of extrapolating noisy measurements, Kalman is used to estimate the latent signal and its uncertainty in real time.
Persistence is treated as the default short-horizon behavior, while Kalman contributes structure and noise suppression.
This design aligns with established practice in control systems and sensor fusion, where estimation precedes any decision or prediction layer.
V3 - Forecasting is performed conditionally using estimator confidence; in low-confidence regimes the system explicitly falls back to persistence.
This project evolved through three deliberate stages, each correcting a specific modeling mistake revealed by empirical evidence.
| Version | Core Role of Kalman | Design Choice | Outcome |
|---|---|---|---|
| v1 | Direct predictor | Unconditional short-horizon extrapolation | Persistence outperformed Kalman; noise amplification exposed a model-role mismatch |
| v2 | Fair-signal estimator | Removed forecasting; focused on denoising and structure estimation | Significant noise reduction with controlled lag; stable estimator behavior |
| v3 | Estimator + gatekeeper | Confidence-gated extrapolation with persistence fallback | Forecasting allowed only when justified; large errors suppressed under chaos |
- Prediction is fragile in high-noise regimes
- Estimation must precede decision-making
- Baselines are safeguards, not competitors
- Confidence-aware systems outperform unconditional intelligence
Each version was retained intentionally to preserve the learning trail and demonstrate disciplined system design under uncertainty.