Monday, October 6, 2008

NCAA Rankings: Some disclaimers

Here are some things that should be understood with respect to the NCAA power rankings:

  • APR is an experimental algorithm; it is not intended to be absolutely authoritative (of course, this applies to the NFL rankings as well).

  • This year (2008) is the first time I've really looked at how APR performs on NCAA data week-to-week. Part of the reason I'm doing this is it helps me think about the assumptions I've made (explicitly or implicitly) with regard to its design, and either defend or change them, as I see fit. (Another part of the reason is that it does a good job of revealing high-ranked teams that haven't been tested, as discussed below.)

  • APR was designed and tuned with NFL teams in mind. In particular, it assumes that every team will play a significant fraction of the teams under analysis over the season; that is simply not true in the NCAA.

  • APR doesn't put a premium on undefeated records and multiple score blow-outs the way other NCAA ranking schemes do.

  • APR power index for a team is based strictly on the final score of games played, and the corresponding power indexes of the teams played. If a team plays a buch of cream-puff teams to start the season (not unusual in the NCAA), APR just won't rank them very high.

Here's a couple of examples from week 6 of the 2008 season:

LSU (AP ranked #4, APR ranked #37)
LSU has beat Appalachian State (#145), North Texas (#166), Auburn (#48), and Mississippi State (#106).


LSU is (probably) a very good team, and they will almost certainly rise as they get into the meat of their season. But right now, from APR's point of view, aren't ranked higher because they just haven't played a good team yet.

Auburn (AP ranked #20 APR ranked #48)
Auburn has beat LA-Monroe (#163), Southern-Miss (#102) (by 3-2!), Mississippi-St (#106), and Tennessee (#69) (14-12).


More weak teams, some beat by very close scores. They've also lost to LSU (#37) and Vanderbilt (#14). Ironically, APR gives Auburn credit for playing Vanderbilt close on the road.