Welcome to the 2nd NFL PT Awards. The purpose of these
awards is to help raise the publicity for those systems that
have been superior in various qualities of interest.
Most awards will be based entirely on the numbers I've
monitored with my prediction tracker web pages. I have
followed the weekly performance of 25+ computer ratings systems
over the course of the past three seasons.
BEST STRAIGHT UP WINNERS (Entire Season)
Winner: JFM Power Ratings, James McConnell
ARGH Power Ratings, Stewart Huckaby
I am going to split the award this year between James McConnell's
JFM Power Ratings, and Stewart Huckaby's ARGH Power Ratings. ARGH
won more games during the season but that was because I didn't have
the JFM ratings for week number 1. If I threw out week 1 for everyone
then JFM would be the leader by a fairly wide margin. So I will
just give them both recognition.
ARGH was 169-90 for the entire season. That is down 3 games from
CPA Rankings' record last season and 2 games lower than Ken Massey two
years ago. If JFM had had week one ratings then it likely would have
set the three year record for most wins. CPA's new retrodiction
retrodiction rankings also deserves credit for actually having the
highest picking percentage but is not eligible for the award since
it started mid season.
2001 Winner: JFM Power Ratings, ARGH Power Ratings
2000 Winner: CPA Rankings
1999 Winner: Kenneth Massey Ratings
BEST AGAINST THE SPREAD (Entire Season)
Winner: CPA Rankings, Steve Wrathell
This might not be the most useful measure of the quality of a football
rankings system but betting on the NFL is big business. This season
the best system picking against the spread was Steve Wrathell's
CPA Rankings with a record of 129-111 (53.75%). This is down
quite a bit from Yourlinx's 57.6% last season. Yourlinx also did
a decent job this year finishing in second 3 games back.
Grid Iron Gold actually had a better percentage (54.3%) but I did not
collect it for the first month of the season.
winner 2001: CPA Rankings
winner 2000: Yourlinx
winner 1999: PerformanZ Ratings
SMALLEST MEAN ABSOLUTE ERROR (Entire Season)
Winner: Vegas Line
Mean absolute error is the average of the absolute value between the game
outcome and the predicted game difference. As it always is, the
system with the lowest mean absolute error is the vegas spread. The spread
really does a good job of predicting the final outcome. No system has ever
beat it over the last three seasons for college or the pros. However, the
systems do a better job in the NFL than they did in the NCAA. Here several
of the systems at least come close to matching the spreads performance and
we have cases where they beat the spread over the second half of the season.
For the season as a whole the top computer system was JFM Power ratings, 10.53,
and Pigskin Index 10.54. The Vegas line lead the way at 10.28, improving
over last year's 10.63.
winner 2001: Vegas Line
winner 2000: Vegas Line
winner 1999: Vegas Line
SMALLEST BIAS (Entire Season)
Winner: ARGH Power Ratings, Stewart Huckaby
The smallest bias over the entire season belonged to Steward Huckaby's
ARGH Power Ratings with a bias of -0.133. This means that on average
ARGH's predictions were 0.133 points to low. Jeff Sagarin was right behind
at -0.164. For the last two years the winning number has been much closer
to zero. Pigskin Index won last year with +0.048. I think a major reason
for this year's numbers being worse than normal was the unusually low
value of the average home field advantage. It was close to 2 this year when
in a normal year it is close to 3.
winner 2001: ARGH Power Ratings
winner 2000: Pigskin Index
winner 1999: Flyman Ratings
MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Entire Season)
Winner: Pigskin Index
The smallest mean square error for the entire season goes to the Vegas Line
at 173.5. I didn't recognize it at the time but last season Pigskin
Index won this award and is the only time a system in either the NFL or
NCAA has beaten the Vegas line. Pigskin won last year with a value of
178.2. This year Hank's Power Ratings were second behind the line at
176.3.
winner 2001: Vegas Line
winner 2000: Pigskin Index
SECOND HALF AWARDS
I like to look at the results over the second half of the season. The
systems have had time to become 'burned in' to the season's data.
Second half data consists of all games from week 10 through the superbowl.
BEST STRAIGHT UP WINNERS (Second Half)
Winner: Pythagorean Ratings
Winner: Elo Ratings
Winner: Massey Ratings, Kenneth Massey
This year was extremely close at the top. The three ratings above all finished
tied with a record of 91-40. Then there were nine more systems within only
three games. This award would be going to Jeff Self but then his ratings
stopped before the season ended.
There was also a tie last year between PerformanZ Ratings and Scoring
Effeciency. And then there was another three way tie the year before
that. 2nd Half numbers were higher than they were last season but lower
than in 1999.
Winner 2001: Pythagorean Ratings, Elo Ratings, Ken Massey Ratings
Winner 2000: PerformanZ Ratings, Scoring Effeciency
Winner 1999: Ken Massey, Ed Kambor, Pythagorean Ratings
BEST AGAINST THE SPREAD ( Second Half)
Winner: Stat Fox
This category was led by two membership sites. Stat Fox in first and
Grid Iron Gold in second place. I did not really know what to expect
from these sites but it appears they do a decent job picking against
the spread while they are near the bottom in almost all other categories.
Stat Fox's 56.25% is the same percentage as the previous two winners,
Pythagorean Ratings in 2000 and Least Squares in 1999.
Winner 2001: Stat Fox
Winner 2000: Pythagorean Ratings
Winner 1999: Least squares using team specific home field advantages
SMALLEST MEAN ABSOLUTE ERROR (Second Half)
Winner: Vegas Line
The Vegas line squeezed out by the narrowest of margins over Stewart Huckaby's
ARGH Power Ratings. The line had an average error of 10.3577 and ARGH was
10.3588. Last year's leader, Pythagorean ratings, was also very close in
third at 10.36. These are all improvements of 2000. In fact 8 systems did
better than last year's leader.
Winner: Vegas Line
Winner: Pythagorean Ratings
SMALLEST BIAS (2nd Half)
Winner: Monte Carlo Markov Chain
This award was not as close as the first two. Monte Carlo Markov Chain
was about the only system that didn't do terrible. As with the entire
season results I believe the reason is that the home field advantage
was smaller than normal this year. This lead to everyone giving too
many points to the home team. Flyman Ratings had won the award the last
two years but then came in last place this year. In both of the
previous two years we underestimated the home team then this year
over estimated it.
Winner: Monte Carlo Markov Chain
Winner: Flyman Performance Ratings
Winner: Flyman Performance Ratings
MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (2nd Half)
Winner: Hanks Power Ratings , Hank Trexler
I'm not sure where this result came from. Hank Trexler's Power Ratings kind
of came out of the middle of the pack to win this with a mean square error
of 167. ARGH Power Ratings was second at 167.4 The interesting thing about
Hank is he is the only one to have a negative average bias. The key this
year must have been in not prediction blow outs by the home team.
Winner 2001: Hank's Power Ratings
Winner 2000: Pythagorean Ratings
BEST PREDICTIVE SYSTEM (Entire Season)
Winner: CPA Rankings, Steve Wrathell
I did this the same way I did last year. I gave points to a system in each
category based on how they did, then summed up their points. Hopefully this
gives an accurate read on who is the best across the board. Although the past
two years the system that won was not who I expected it to be.
Edward Kambour was the 2000 Winner. ARGH finished runner up last year as
well as this year. CPA moved up from third to first.
Of note: Self, CPA, ARGH, Vegas, Yourlinx, Matthews, and Pythagorean were the
only ratings above average in all categories.
1. CPA Rankings 87
2. ARGH Power Ratings 80
3. Vegas Line 71
BEST PREDICTIVE SYSTEM (Second Half)
Winner: Monte Carlo Markov Chain
This award was also determined by giving points for each of the above
categories and summing for each system. Another surprise winner here
in Monte Carlo Markov Chain. ARGH finishes 2nd for the 2nd half as well.
The season total winner only finished 4th in the second half.
Last season the Pythagorean Ratings won this one in a blowout.
1. Monte Carlo Markov Chain 81
2. ARGH Power Ratings 73
3. Least Squares 65
RETRO AWARDS
The retrodictive categories are based on each systems final
standings and measure things by applying those final ratings
to the entire season in retrospect.
MOST RETRODICTIVE WINS
Winner: System Average
The best retrodiction record this past season goes to the system
average. The average prediction of a game is the average of the
25 systems that have a prediction on that game. The system average's
record was 197-62, about 76%. This is an 8 game improvement over last
year's winner CPA Rankings. Last year's winner would have only been good
for 14th place this season. The system average had a big lead of 4 games
over the second place system, Monte Carlo Markov Chain.
Winner 2001: The system average
Winner 2000: CPA Rankings
SMALLEST RETRODICTIVE ABSOLUTE ERROR
Winner: Least Squares with team home field advantages
Least squares using team specific home field advantage wins this for
the second straight year. It has a winning average error of 8.77 points.
As with wins, the numbers in this category are much better than last year.
Last year the same system was the leader at 9.27
Winner 2001: Least squares with team specific home field advantage
Winner 2000: Least squares with team specific home field advantage
SMALLEST RETRODICTIVE BIAS
Winner: Herman Matthews, aka Matthews Grid, aka Scripps Howard
The smallest retrodictive bias in 2001 goes to Herman Matthew's
system at -0.003. In fairness to a couple of other systems this
was almost a 3 way tie. Sonny Moore ended with a value of -0.005
and the Dunkel Index had a value of +0.006.
Winner 2001: Herman Matthews
Winner 2000: Massey Ratings
BEST OVERALL RETRODICTIVE SYSTEM IN 2001
Winner: CPA Rankings, Steve Wrathell
I guess it is important to mention that this award, as with the
best predictive systems, go to the system that is the best overall,
not in just one category. So it happens again. CPA Rankings did
not win any single category but did well enough in all categories
to win the best overall retrodictive award rather easily.
As with the other overall awards, points are assigned to systems
in each category then summed up. The system with the most points
wins. Also note that CPA's original ratings is the winner. Not
his retro rankings.
CPA Rankings 85
Least Squares w/ team HFA 66
System Average 65
Winner 2001: CPA Rankings
Winner 2000: CPA Rankings