I’ve been playing with a college football ranking system since 1984. You might remember that was the year that BYU was the lone undefeated team in college football and eventually won the national championship. A number of commentators claimed that BYU did not deserve to be #1, because their undefeated record was earned against inferior competition. It seemed to me that the wire service rankings tended to be based primarily on teams’ win-loss record, and paid little attention to strength of competition. I decided to play with a way to come up with an objective system that looked at more than just a teams’ records.
I called my first system HPI (don’t ask what it stands for–it’s a long story). HPI used two factors: margin of victory and quality of opponent (as measured by the opponent’s win-loss record). Margins of victory (or defeat) were put into a few categories and assigned points (one point for a loss of 31 or greater, two points for a loss of between 26 and 31 points, and so on up to a victory of 31 points or greater). This point value was multiplied by the opponent’s winning percentage (with a modification to avoid awarding zero points when a team plays a winless team) to yield a score for that team for that game. A team’s overall rating was determined by dividing the sum of all of its results by the number of games it had paid.
The maximum margin of victory was capped at 31 to remove the incentive to run up the score (because, of course, college football coaches were going to let the unpublished rankings compiled by a college student affect in-game decisions).
Over the years, I’ve made three significant changes to my ranking system. First, while I was convinced that my rankings were a better way to rate teams than simply looking at their win-loss record, I was uncomfortable with using opponent’s winning percentage in the calculation. Since I felt that win-loss record was a flawed way to rate the quality of a team, I was incorporating that flaw into the way I determined my rankings. So the first change I made was to lessen the importance of win-loss record by essentially performing the calculations twice. The first time yielded a preliminary score for a team, which I then put back into the equation, by substituting an opponent’s preliminary score for win-loss record in the equation. The resulting score, I felt, reduced the importance of win-loss record in determining the quality of an opponent.
After about ten years of doing these rankings, I made my second major modification. The original method did not take into account whether a team was playing at home or on the road. I decided to modify my scoring system to give more credit for playing on the road. Each result got slightly more points if it occurred on the road than if it occurred at home. I continued to use this system until 2010, and this is the system that I will refer to as “Classic HPI” throughout this blog.
The third modification, made in 2010, was probably the most significant. While I believe that margin of victory or defeat is generally an indicator of the difference between two teams, and therefore should be reflected in the ratings, I realized that margin of victory or loss has some limitations. First, it seemed to reward certain styles of play more than others. A team with a high octane offense is likely to rack up greater margins of victory than a team that relies on its defense, but the defense-minded team might dominate a game just as much as the offense-minded team, but have a much smaller margin of victory. In other words, a 21-0 victory might be a more dominant performance than a 51-30 victory. Second, focusing solely on the final result obscures much of what happened prior to the final result. Teams sometimes play close, only to see a game fall apart in the final minutes, making what was really a close game appear to be a blowout.
This third change was significant enough that I called it by a new name: xHPI, which stands for “Experimental HPI.” It included two separate changes. First, instead of basing the rating just on the final score, I use the scores at the end of each quarter, with weights given for the lateness of the score. In other words, the second quarter score affects the final rating more than the first quarter score does, the third quarter is more important than the second, and the final score having the most weight. Second, I substituted a modified “percentage of total score” for margin of victory to address the effect of style of play on scoring margin.
I am convinced that these changes will improve the quality of rankings. During the 2011 season, I will be testing their performance by comparing the rankings to Classic HPI. I will be comparing their performance by observing how well they predict future games.
