Algorithms: How APR works, and what it's trying to do

For those of you coming here from footballoutsiders.com, let me say right away that APR is not supposed to be a replacement or alternative to DVOA, DPAR, or any of their other metrics.

The APR algorithm:

1. Initially, every team is assigned a power index of 1.0

2. For each team, a power index detailed below) is assigned to it for each game played.

3. For each team, the game power values for that team are then combined together using a weighted average (old games weighted less than new games) to give a new power index for the team.

4. Steps 2 and 3 are repeated a number of times to create a feedback loop.

The power for a played game is computed according to the following formula:

gamePower = resultPower × marginPower × opponentPower

Where:

• resultPower is a constant based on whether the team won or lost, and whether they were the visiting or home team, with the following constraints:

  road win > home win > tie > road loss > home loss

• marginPower is a constant based on the margin of victory (or loss). Right now, margin power is assigned in the following ranges:

  • won by 14 or more
  • won by 7 or more
  • won (or lost) by 6 or fewer points
  • lost by 7 or more
  • lost by 14 or more

• opponentPower is the power index of the opponent team.

Using the power index values

Once power index values have been generated, a simple sort of teams from largest to smallest power index is used to generate a power rankings. Similarly, picks for the following week's games are made by chosing the team with the largest power index value for games played to that point.

Evaluating design choices

The efficacy of the algorithm (as well as choices for the various constants) is judged strictly on how well it does picking games on the historical data set of NFL games played. I have made attempts to give more power for wider margins of victory (or loss); such attempts have lead to fewer games picked correctly and were discarded. This means teams can generate a only limited amount of power playing very weak teams, even with blow-out wins.

The design of the APR algorithm has also lead to a related phenomenon I call "power by association" (or PBA, if you like TLAs): When a weak team plays a strong team close (especially with the weak team on the road), the weak team will increase in power, even if they lose (and the strong team will decrease in power, even if they win).

The APR ranking system is by no means perfect. It doesn't take into account injuries, break-out players, fluke wins, how different teams match up, the strength of individual units within a team, or the strength of teams in different game situations. At the end of the season, it doesn't take into account that some teams will clinch their post-season fate early, and elect to rest key starters for one or more games.

Still, it does (to my way of thinking, anyway) a remarkably good job at picking games, and often reveals over- and under-rated teams before ESPN or other subjective-based power ranking systems notices.

Update: and, if you've made it this far, be sure to read this post, which expands on the consequences of APR's design.

NCAA Rankings: Some disclaimers

Here are some things that should be understood with respect to the NCAA power rankings:

• APR is an experimental algorithm; it is not intended to be absolutely authoritative (of course, this applies to the NFL rankings as well).

• This year (2008) is the first time I've really looked at how APR performs on NCAA data week-to-week. Part of the reason I'm doing this is it helps me think about the assumptions I've made (explicitly or implicitly) with regard to its design, and either defend or change them, as I see fit. (Another part of the reason is that it does a good job of revealing high-ranked teams that haven't been tested, as discussed below.)

• APR was designed and tuned with NFL teams in mind. In particular, it assumes that every team will play a significant fraction of the teams under analysis over the season; that is simply not true in the NCAA.

• APR doesn't put a premium on undefeated records and multiple score blow-outs the way other NCAA ranking schemes do.

• APR power index for a team is based strictly on the final score of games played, and the corresponding power indexes of the teams played. If a team plays a buch of cream-puff teams to start the season (not unusual in the NCAA), APR just won't rank them very high.

Here's a couple of examples from week 6 of the 2008 season:

LSU (AP ranked #4, APR ranked #37)

LSU has beat Appalachian State (#145), North Texas (#166), Auburn (#48), and Mississippi State (#106).

LSU is (probably) a very good team, and they will almost certainly rise as they get into the meat of their season. But right now, from APR's point of view, aren't ranked higher because they just haven't played a good team yet.

Auburn (AP ranked #20 APR ranked #48)

Auburn has beat LA-Monroe (#163), Southern-Miss (#102) (by 3-2!), Mississippi-St (#106), and Tennessee (#69) (14-12).

More weak teams, some beat by very close scores. They've also lost to LSU (#37) and Vanderbilt (#14). Ironically, APR gives Auburn credit for playing Vanderbilt close on the road.

Welcome to FSPI

FSPI stands for Final Score Power Index. Generically, it's any method for assigning a numerical power value to teams in a league, based on the final score(s) of the games each team has played. One example is the "Simple Ranking System", described in this blog post on pro-football-reference.com.

Once each team has a power index, teams can be sorted from most to least power to yield a power ranking. If the power index is predictive, it can also be used to predict winners of upcoming games.

This blog will use various power index algorithms (including some variant of SRS) to follow teams over the course of the NFL season. That means most of the traffic will be from August to February. But I didn't want to wait until August to figure out how to drive blogger, and this will give me an incentive to get my software ready to go.

So welcome, and thanks for reading.