Methodology

How we score analyst predictions and rank organizations.

PSR — Prediction Success Rate

Definition

The proportion of predictions that achieved their stated objective within the evaluation timeframe.

FormulaPSR = Total Points Earned / Total Predictions Made
Example

An organization made 20 predictions. 14 were full successes (1.0 each), 3 were partial (0.5 each), and 3 were failures (0.0 each). Total points = 14 + 1.5 = 15.5. PSR = 15.5 / 20 = 0.775 (77.5%).

Full Success (Bullseye)1.0 point
Partial Success0.5 point
Failure (Miss)0.0 points

Average ROI

Definition

The mean return on investment across all BUY and SELL predictions. HOLD predictions are excluded from ROI pools.

FormulaAverage ROI = (Sum of all individual ROI values) / (Number of BUY + SELL predictions)
Example

Three predictions: BUY with +40% ROI, SELL with +15% ROI, BUY with -10% ROI. Average = (40 + 15 + (-10)) / 3 = 15%.

Calculation Methods:
  • Peak System (Default): Calculates return using the peak price reached in the direction of the prediction.
    BUY ROI = (Max Price - Start Price) / Start Price
    SELL ROI = (Start Price - Min Price) / Start Price
  • End ROI Calculation: Calculates return strictly based on the final price at the end of the timeframe.
    BUY ROI = (End Price - Start Price) / Start Price
    SELL ROI = (Start Price - End Price) / Start Price
  • Symmetric Peak ROI: Calculates Net ROI (Gains minus Losses).
    Net ROI (of buy call) = (Max Price + Min Price - 2 * Start Price) / Start Price
    Net ROI (of sell call) = (2 * Start Price - Max Price - Min Price) / Start Price
Incorrect Prediction Punishment Methods:
  • No Punishment (Default): Misses or incorrect directions are capped at 0.0% ROI (disregarded).
  • Negative ROI Accounting: Misses and incorrect direction details are calculated as negative ROI (e.g. stock peaks at -10% from start, yielding -10% ROI).

HOLD predictions are strictly excluded from all ROI calculations because the objective is stability, not directional gain.

Median ROI

Definition

The middle ROI value after sorting all individual prediction returns. Less affected by extreme outliers than the average.

FormulaSort all ROI values. If odd count: pick the middle. If even: average the two middle values.
Example

ROI values: -10%, +5%, +15%, +40%, +120%. Median = +15% (third of five values). The 120% outlier doesn't skew it.

The purpose of tracking Median ROI alongside Average ROI is to detect organizations that appear successful because of one or two outlier home runs, while their typical prediction returns are mediocre. A large gap between average and median ROI is a red flag.

Peak ROI

Definition

The highest return reached at any point during the prediction's evaluation timeframe (not just at the end).

FormulaPeak ROI = (Max Price during Timeframe - Starting Price) / Starting Price (for BUY). Inverse for SELL.
Example

A BUY prediction starts at $100. The stock reaches $152 mid-timeframe before settling at $118. Peak ROI = 52%, not 18%.

We evaluate against Peak ROI by default rather than End-of-Timeframe ROI because an analyst's call may be directionally correct for most of the evaluation period. A BUY call that peaks at +40% before macro headwinds push it to +5% still demonstrates alpha identification skill. However, users can switch to End ROI or Symmetric Peak ROI at the top of the table to view the performance using alternative methodologies.

Index Score

Definition

A proprietary composite metric that combines return magnitude with prediction accuracy. Higher is better.

FormulaIndex Score = Median ROI (as decimal coefficient) x PSR Base
Example

If an organization has Median ROI = 40% (0.40 coefficient) and PSR = 0.667, their Index Score = 0.40 x 0.667 = 0.267. Think of it as 'accuracy-weighted return'.

The Index Score is not a percentage. It's a dimensionless composite where higher values indicate better combined performance (both accuracy and magnitude). An organization with 60% ROI and 50% PSR (Index = 0.30) ranks higher than one with 100% ROI and 10% PSR (Index = 0.10), because the latter's enormous returns are too rare to be reliable.

Confidence Weighted Index

Definition

A variant of the Index Score that rewards organizations with a larger prediction history. More predictions mean greater statistical confidence, giving them a higher weight in rankings.

FormulaConfidence Index = PSR × Median ROI × Predictions^Exponent
Example

Organization A has PSR = 0.667, Median ROI = 40% (0.40), and 100 predictions. With exponent = 0.5: Index = 0.667 × 0.40 × √100 = 0.267 × 10 = 2.67. Organization B has the same PSR and ROI but only 25 predictions: 0.667 × 0.40 × √25 = 0.267 × 5 = 1.33. Organization A ranks higher because its larger sample is statistically more reliable.

The exponent is adjustable (0.00–1.00, default 0.50) via a slider in the dashboard.

  • 0.00 — Equals the standard Index Score (no confidence boost)
  • 0.50 — Square root of prediction count (default, balances weight)
  • 1.00 — Linear scaling with prediction count (maximum weight)

Scoring Rules — BUY, SELL, and HOLD

BUY Rating Scoring

  • 1.0 — Stock gained 10% or more during timeframe
  • 0.5 to 1.0 (Linear Gradient) — Stock gained between 5.0% and 9.99%.
    Formula: Points = 0.5 + 0.1 × (gain% - 5)
  • 0.0 — Stock gained less than 5.0%

SELL Rating Scoring

  • 1.0 — Stock declined 10% or more (drop) during timeframe
  • 0.5 to 1.0 (Linear Gradient) — Stock declined between 5.0% and 9.99%.
    Formula: Points = 0.5 + 0.1 × (drop% - 5)
  • 0.0 — Stock declined less than 5.0%

HOLD Rating Scoring

  • 1.0 — Stock price remained stable, fluctuating no more than 5.0% from starting index
  • 0.5 to 1.0 (Linear Gradient) — Stock price fluctuated between 5.01% and 10.0%.
    Formula: Points = 1.0 - 0.1 × (fluctuation% - 5)
  • 0.0 — Absolute movement exceeded ±10.0%
Analyst Performance Tracker — Which Wall Street Analysts Actually Make Money?