Wednesday, October 8, 2008

Algorithms: How APR works, and what it's trying to do

For those of you coming here from footballoutsiders.com, let me say right away that APR is not supposed to be a replacement or alternative to DVOA, DPAR, or any of their other metrics.

The APR algorithm:

  1. Initially, every team is assigned a power index of 1.0

  2. For each team, a power index detailed below) is assigned to it for each game played.

  3. For each team, the game power values for that team are then combined together using a weighted average (old games weighted less than new games) to give a new power index for the team.

  4. Steps 2 and 3 are repeated a number of times to create a feedback loop.

The power for a played game is computed according to the following formula:

gamePower = resultPower × marginPower × opponentPower
Where:
  • resultPower is a constant based on whether the team won or lost, and whether they were the visiting or home team, with the following constraints:

    road win > home win > tie > road loss > home loss
  • marginPower is a constant based on the margin of victory (or loss). Right now, margin power is assigned in the following ranges:

    • won by 14 or more
    • won by 7 or more
    • won (or lost) by 6 or fewer points
    • lost by 7 or more
    • lost by 14 or more
  • opponentPower is the power index of the opponent team.

Using the power index values

Once power index values have been generated, a simple sort of teams from largest to smallest power index is used to generate a power rankings. Similarly, picks for the following week's games are made by chosing the team with the largest power index value for games played to that point.

Evaluating design choices

The efficacy of the algorithm (as well as choices for the various constants) is judged strictly on how well it does picking games on the historical data set of NFL games played. I have made attempts to give more power for wider margins of victory (or loss); such attempts have lead to fewer games picked correctly and were discarded. This means teams can generate a only limited amount of power playing very weak teams, even with blow-out wins.

The design of the APR algorithm has also lead to a related phenomenon I call "power by association" (or PBA, if you like TLAs): When a weak team plays a strong team close (especially with the weak team on the road), the weak team will increase in power, even if they lose (and the strong team will decrease in power, even if they win).

The APR ranking system is by no means perfect. It doesn't take into account injuries, break-out players, fluke wins, how different teams match up, the strength of individual units within a team, or the strength of teams in different game situations. At the end of the season, it doesn't take into account that some teams will clinch their post-season fate early, and elect to rest key starters for one or more games.

Still, it does (to my way of thinking, anyway) a remarkably good job at picking games, and often reveals over- and under-rated teams before ESPN or other subjective-based power ranking systems notices.

Update: and, if you've made it this far, be sure to read this post, which expands on the consequences of APR's design.