Meta-Analysis Results

In-Depth Evaluation of Our High-Performing Models

Overview

We reviewed binary classification stock predictions made by our top-performing models. Notably, 56% of these predictions correctly anticipated the market’s direction—just a bit above the 50% expected by random chance. We simulated investing $1,000 on every “Up” prediction, the overall return on investment (ROI) came out to only about 1.75%. Over the same time period, if you had invested the money in $SPY you would have made approximately 2.5%. This modest gain highlights that even with a slight edge in prediction accuracy, the real-world benefit can be limited. The interactive chart below illustrates these results, showing how individual trade gains and losses over time tend to follow the broader trends of the market and hit big when market is going up.

Interactive Chart

Parameter Usage Among Top Models

We analyzed parameter choices frequently found in our top-performing models. In the tables below, Usage Rate represents the percentage of total occurrences of a parameter value that contributed to a top-performing model.

  • Optimal Prediction Horizon: 25-29 days or 30-34 days
  • Optimal Lookback Period: 700-799 days or 800-899 days
  • Feature Set: basic
  • Hyperparameter Tuning: low
Prediction Horizon (Days)
Value Range Usage Rate
25-29 16.32%
30-34 15.04%
20-24 13.16%
15-19 7.19%
10-14 3.28%
5-9 1.27%
Lookback Period (Days)
Value Range Usage Rate
700-799 11.16%
800-899 10.83%
900-999 9.28%
1000-1099 7.88%
1100-1199 6.82%
Top Features & Their Scores

Below is an ordered list of the most impactful features (highest to lowest), along with their approximate contribution scores. Each feature is briefly defined for clarity:

  1. Gross Domestic Product Growth (score: 0.85)
    Quarterly Reports of the GDP assigned to training and testing dates.
  2. 50-Day Simple Moving Average (score: 0.78)
    The average stock price over the last 50 trading days.
  3. Lower Bollinger Band (score: 0.75)
    A technical indicator plotted a specified number of standard deviations below a simple moving average, often signaling potential low points.
  4. 100-Day Simple Moving Average (score: 0.72)
    The average stock price over the last 100 trading days.
  5. 60-Day Rolling Volatility (score: 0.70)
    The standard deviation of returns over the last 60 trading days, indicating how volatile a security is.
  6. Unemployment Rate (score: 0.68)
    Quarterly Reports of the Unemployment rate assigned to training and testing dates.
  7. 60-Day Momentum (score: 0.65)
    A momentum indicator reflecting the rate of change of a stock's price over the past 60 days.
  8. Moving Average Convergence Divergence (score: 0.60)
    A trend-following momentum indicator showing the relationship between two moving averages.
  9. Upper Bollinger Band (score: 0.58)
    A technical indicator plotted a specified number of standard deviations above a simple moving average, often signaling potential high points.
  10. 60-Day Correlation with the S&P 500 (score: 0.55)
    How closely a stock's returns move in relation to the S&P 500 index over the last 60 days.