Welcome to the 8th annual NCAA PT Awards.  They come out a little later every
year.  The purpose of these awards is to help raise the publicity for those systems
that have been superior in various qualities of interest.  Most awards will be
based entirely on the numbers I've monitored with my prediction tracker web pages.
For the 2008  season I followed the weekly performance of 54 computer rating systems.
This is down from a peak of 61 last season.




  MOST STRAIGHT UP WINNERS (Entire Season)

Winner: System Average & System Median


This year I collected predictions on 718 games involving two 1A teams.
Now we are starting to see an interesting result.  The system average has
tied for most wins in two of the last three seasons.  I was always curious
to see if given enough "quality" systems if the average would converge to
the Vegas line or the actual result.  I think the evidence leans in this
direction.  This is why I think the BCS should use the average of 50 computers
instead of just six.  Both the average and median predictions finished the
season 542-176, 75.5%.  That is a decent mark, a good deal higher than last
year but a ways below the record of 76.48%.  This has to be considered a bad
year for the oddsmakers.  Eleven systems finished better than the line.
The top individual system was Jim Ahsburn's Atomic Football.



2008 Winner: System Average, System Median
2007 Winner: StatFox
2006 Winner: Thompson SPRS, system average, Sagarin predictive, & Sagarin overall
2005 Winner: Pigskin Index
2004 Winner: Jeff Sagarin's Overall Rating
2003 Winner: Edward Kambour & Jeff Sagarin's Overall Rating
2002 Winner: ARGH Power Ratings (record 76.48%)
2001 Winner: Massey Ratings
2000 Winner: ARGH Power Ratings
1999 Winner: Vegas line




  SMALLEST DEVIATION FROM ACTUAL GAME SCORES (Entire Season)

Winner:  The Vegas line

  Deviation from the game score is the difference between the game prediction
and the actual result.  A value of zero would mean the score difference is predicted
exactly.  One property of a good system would be to minimize the system's average
game deviation.
 The Vegas line wins this year by the slightest of margest over the computer adjusted
line.  The line's average game devation was 12.5912.  This value is quite high compared
to past seasons.  The lowest for any computer system was Ashburn's Atomic Football at  12.85.


2008 Winner: Vegas Line
2007 Winner: Vegas Line
2006 Winner: Computer Adjusted Line (record 11.81)
2005 Winner: Vegas Line
2004 Winner: Vegas Line
2003 Winner: Vegas Line
2002 Winner: Vegas Line
2001 Winner: Vegas Line
2000 Winner: Vegas Line
1999 Winner: Vegas Line




  SMALLEST AVERAGE GAME BIAS (Entire Season)

Winner:  Opening Line


  Bias is a little different from deviation.  Deviation measures the
distance between a prediction and the actual result.  Bias combines
distance and location of the prediction.   Bias measures whether the
predictions are too high or too low.  So if a sytem has an
average bias of +0.25 that means that on average the system
gives 0.25 points too many to the home team.
  The bias numbers were fairly accurate this season.  A good chunk of them
with within +/- 0.50.  The opening line had the smallest bias at +0.0216.
James Howell was very close with -0.0236.



2008 Winner; Opening Line
2007 Winner: Thompson SPRS
2006 Winner: Congrove Computer Rankings
2005 Winner: Computer Adjusted Line
2004 Winner: CPA Rankings
2003 Winner: Edward Kambour
2002 Winner: Gupta Power Ratings
2001 Winner: Flyman
2000 Winner: ARGH Power Ratings
1999 Winner: Jeff Sagarin




  BEST AGAINST THE SPREAD (Entire Season)

Winner: Stephen Kerns

  Beating the spread is not the number one goal for most computer
rating systems.  But it is something that the average person likes
to look at and probably the most popular subject that I get emails
about.  I know that looking at all the games is probably not the
most meaningful of measurements but it is what I have been reporting.
  Considering that the line did a relatively poor job picking winners
and also had a higher than normal absolute deviation it comes as
somewhat as a surprise that the numbers against the spread were not very
good this year.  Only 15 did better than 50%.  Stephen Kerns at 53.91
and StafFox at 53.88 where the only systems above 52.5%.  As usual,
your best bet would have been playing opposite a system, this year that
would have been Pigskin Index.



2007 Winner: Stephen Kerns
2006 Winner: Harmon Forecast  (record 56.4%)
2005 Winner: Keeper's College Football Ratings
2004 Winner: Edward Kambour
2003 Winner: Edward Kambour
2002 Winner: Dunkel Index
2001 Winner: BMC Picks
2000 Winner: Dunkel Index
1999 Winner: Average across all systems



  MOST ACCURATE PREDICTOR (Entire Season)

Winner:  The Vegas line

  This award is based on mean square error. Mean square error takes
into account both deviation and bias and is perhaps the most commonly
used measure of evaluating estimators.
 There may come a point were I should just stop this category.  The
Line had the smallest mean square error for the 10th consecutive season.
The lines mse was 255.867. A good dozen or so systems did better than
this last season.   The record is 220.556 sin 2006.  The numbers were
consistantly improving year to year up through 2006 but have declined
in the two seasons since then.



2008 Winner: The Vegas Line
2007 Winner: The Vegas Line
2006 Winner: The Vegas Line  (record 220.556)
2005 Winner: The Vegas Line
2004 Winner: The Vegas Line
2003 Winner: The Vegas Line
2002 Winner: The Vegas Line
2001 Winner: The Vegas Line
2000 Winner: The Vegas Line
1999 Winner: The Vegas Line



    ------------------------


  It is debatable how meaningful rating systems are in the early
parts of the season.  How can you rate the teams in the first
week of the season when no teams have played a game yet?  The BCS
waits until mid October before releasing it's first ranking
of teams.  Some systems choose to wait until around this time
before being made public. Looking at only the second half of the
season also gives an estimate of how well a system does based
only(or mostly) on data from the current year.



   MOST STRAIGHT UP WINNERS (Second Half of Season)

Winner: Nutshell Sports


  The second half data started with week 9 and consisted of 342 games
between two division 1A teams.  If nothing else, the systems were very
close to each other over the second half of the season.  Ray Wait's
Nutshell Sports predictive ratings had the best record (256-86, 74.85%)
for the second half.  Then there were 4 systems tied only one game behind.


2008 Winner:  Nutshell Sports
2007 Winner:  Edward Kambour
2006 Winner:  Ed Bemiss
2005 Winner:  Dunkel Index
2004 Winner:  Jeff Sagain, Wesley Colley
2003 Winner:  Born Power Index & Edward Kambour (record 76.7%)
2002 Winner:  Born Power Index
2001 Winner:  Chris Montgomery
2000 Winner:  Geoff Freeze




  SMALLEST DEVIATION FROM ACTUAL GAME SCORES  (Second Half of Season)


Winner: StatFox

  For the second year in a row the line, or a derivation from the line, did not win
this category.  This year StatFox had the lowest average absolute error, 12.6229, for
the second half. The Born Power Index also managed to beat the line this year.  But these
numbers are almost a full point below last season.



2008 Winner:  StatFox
2007 Winner:  Edward Kambour
2006 Winner:  Computer Adjusted Line (record 11.73)
2005 Winner:  Vegas Line
2004 Winner:  Ken Ashby AccuRatings
2003 Winner:  Vegas Line (opening)
2002 Winner:  Vegas Line
2001 Winner:  Vegas Line
2000 Winner:  Vegas Line




  SMALLEST AVERAGE GAME BIAS  (Second Half of Season)

Winner:  Dunkel Index

  The average game bias for the second half was very different from the full season data.  For
the full season they averaged out to around zero.  For the second half all but two systems had
negative values. Perhaps this explains why numbers were poor in general this year.  The average
home field advantage was not constant throughout the season.  The lowest game bias belonged to the Dunkel Index, -0.0394.


2008 Winner:  Dunkel Index
2007 Winner:  CPA Rankings
2006 Winner:  Vegas Line
2005 Winner:  Brent Moore
2004 Winner:  Least Squares Regression
2003 Winner:  Darrly Marsee
2002 Winner:  Massey BCS
2001 Winner:  PerformanZ Ratings
2000 Winner:  Darryl Marsee's Rankings




  BEST AGAINST THE SPREAD  (Second Half of Season)


Winner: StatFox

  A couple of systems did extremely well against the spread during the second half
of the season.  StatFox, 57.5%, was the best, but CPA Rankings and Stephen Kerns both
did better than 55%.


2008 Winner:  StatFox
2007 Winner:  Nutshell Sports
2006 Winner:  Ed Bemiss
2005 Winner:  The Sports Report
2004 Winner:  StatFox          (record 58.94%)
2003 Winner:  Edward Kambour
2002 Winner:  The Sports Report
2001 Winner:  BMC Picks
2000 Winner:  The Buck System




  MOST ACCURATE PREDICTOR (Second Half of Season)


Winner: Born Power Index

 It is always interesting and a little surprising when the line does not win this
category.  This year William Born's Power Index edged out the line by 0.05 points.
Born had a mean square error of 260.174.  This is much worse than the record
of 215.55 set in 2006


2008 Winner:  Born Power Index
2007 Winner:  Atomic Football
2006 Winner:  Vegas Line       (record 215.55)
2005 Winner:  Vegas Line
2004 Winner:  Ashby Accuratings
2003 Winner:  Vegas Line
2002 Winner:  Vegas Line
2001 Winner:  Vegas Line
2000 Winner:  Vegas Line




  BEST PREDICTIVE SYSTEM  

Winner: StatFox


  To come up with the best overall predictive system I give each system
points for how well they do in all of the above categories.  I then sum up
the points and the system with the highest total is dubbed predictive system
of the year.  So this award goes to the system that is the most well rounded
One flaw can totally take a system out of the running.

  As I was writing all the previous sections, no one system really stood out
as having the best overall season.  The line was generally the best for the full
season, Statfox and Born were mentioned a couple times for the second half.
It turned out to be relatively close between StatFox, and CPA rankings but
StatFox gets the nod as best predictive system this year.


2008 Winenr:  StatFox
2007 Winner:  Harmon Forecast
2006 Winner:  Computer Adjusted Line
2005 Winner:  Dunkel Index
2004 Winner:  Ken Ashby AccuRatings
2003 Winner:  Ed Kambour
2002 Winner:  Ed Kambour
2001 Winner:  Kenneth Massey (non-BCS)
2000 Winner:  ARGH Power Ratings





     RETRODICTION AWARDS 

  Retrodiction refers to 'retrodicting' or postdicting the previous results,
rather than predicting future games.  It is possible that a computer rating
system can put more emphasis on explaining past results than attempting to
predict future results.  So these systems may not neccesarily be the best
predictive systems but can still be very good at their main objective.
  The retrodiction results on my page come from taking the final
ratings and using them to repredict the entire season.

I break the retrodictive categories into what I will call complex and simple systems.
A simple system will be any system that estimates only a global home
field advantage.   Systems that estimate individual home field advantages
are always going to end up winning these awards so this opens up new
categories for the more traditional systems.


  MOST RETRODICTIVE WINS  -  Complex System

Winner:  Least squares with team specific HFA

 The Sports Report SLOTS system taking the year off we have a different winner
for the first time in 7 season.  Least squares regression with team a team
specific home field advantage had the best record 588-127 (82.2%)


2008 Winner:  Least squares with team specific HFA
2007 Winner:  The Sports Report - SLOTS
2006 Winner:  The Sports Report - SLOTS
2005 Winner:  The Sports Report - SLOTS
2004 Winner:  The Sports Report - SLOTS   (record 92.78%)
2003 Winner:  The Sports Report - SLOTS
2002 Winner:  The Sports Report - SLOTS
2001 Winner:  CPA Rankings
2000 Winner:  CPA Rankings




  MOST RETRODICTIVE WINS  -  Simple System

Winner: Jeff Sagarin

  Jeff Sagarin's main rating has now won this category for 3 of the last
4 seasons.  His record was 585-130 (81.8%).  Many of the complex systems either
did not participate this season or are no longer 'complex'.  So I might have be
recollapsing the two categories in the future.

2008 Winner:  Jeff Sagarin
2007 Winner:  Jeff Sagarin
2006 Winner:  The SuperList
2005 Winner:  Jeff Sagarin, Warren Claassen
2004 Winner:  Logistic Regression   (record 86.0%)
2003 Winner:  Anderson/Hester
2002 Winner:  Logistic Regression
2001 WInner:  System Average





  SMALLEST RETRODICTIVE MEAN ERROR  -  Complex System

Winner:   Least Square Regression with team HFA

  Least squares regression with individual team HFA finishes in first
place for the 5th year in a row. As on the predictive side, theses numbers
were steadily decreasing up to the record year of 2006 and have increased
in each of the two seasons after that.


2008 Winner:  Least square Regression with team HFA
2007 Winner:  Least Square Regression with team HFA
2006 Winner:  Least Square Regression with team HFA  (record 9.1615)
2005 Winner:  Least Square Regression with team HFA
2004 Winner:  Least Square Regression with team HFA
2003 Winner:  The Sports Report - SLOTS
2002 Winner:  The average of all systems.
2001 Winner:  Edward Kambour Football Ratings
2000 Winner:  CPA Rankings




  SMALLEST RETRODICTIVE MEAN ERROR  -  Simple System

Winner: System Average

  The winner of the small class ratings is average of all systems, breaking the
winning streak of Sagarin Predictive.  The winning mark was 10.4871 this season.


2008 Winner:  System Average
2007 Winner:  Jeff Sagarin Predictive
2006 Winner:  Jeff Sagarin Predictive
2005 Winner:  Jeff Sagarin Predictive  (record 10.0129)
2004 Winner:  Least Squares Regression
2003 Winner:  Least Squares Regression





  SMALLEST RETRODICTIVE BIAS 

Winner: Sonny Moore

 I am not breaking this one down into sub categories.  The system that had the smallest
mean bias Sonny Moore's Computer Power Ratings.  This system had a bias of -0.0045.



2008 Winner:  Sonny Moore
2007 Winner:  Least Squares w/ team HFA
2006 Winner:  Sports Report - SLOTS
2005 Winner:  Sports Report - SLOTS
2004 Winner:  Anderson/Hester
2003 Winner:  Anderson/Hester
2002 Winner:  CPA Rankings
2001 Winner:  CPA Rankings
2000 Winner:  CPA Rankings






 MOST ACCURATE RETRODICTIVE RATING  -  Complex System 


Winner:  Least Square Regression with team HFA

  The system that had the smallest mean square error this season
was Least Square Regression with team HFA.  This system also
wins this category for the 5th year in a row with an average
mean square error of 138.649.  This was an improvement over last season
but not as good as the record.


2008 Winner:  Least Square Regression with team HFA
2007 Winner:  Least Square Regression with team HFA
2006 Winner:  Least Square Regression with team HFA  (record 130.437)
2005 Winner:  Least Square Regression with team HFA
2004 Winner:  Least Square Regression with team HFA
2003 Winner:  The Sports Report - SLOTS
2002 Winner:  CPA Rankings
2001 Winner:  Edward Kambour FOotball Ratings





 MOST ACCURATE RETRODICTIVE RATING  -  Simple System 

Winner:  Least Squares Regression

  It should not be too surprising to see least squares regression do well
here since that is what it does, minimize the square errors.  Least squares
ended with a mean square error of 175.773.  About the same as last year.
I am actually very impressed with this basic regression system for college
football.  Predictively it has been near the top of the second half stats every
year and wins this retrodictive category every year.  This system is as simple
as can be and I think you can make a case that it is as good an overall rating
as anything out there.


2008 Winner:  Least Squares Regression
2007 Winner:  Least Squares Regression
2006 Winner:  Least Squares Regression  (record 157.303)
2005 Winner:  Least Squares Regression
2004 Winner:  Least Squares Regression
2003 Winner:  Least Squares Regression




BEST RETRODICTIVE SYSTEM 2007 -  Complex System 


 Winner:  Least Square Regression with team HFA

 Like the award for best predictive system this award is found by giving each system a
score for each category.  The system that has the highest total is the retroditive
system of the year.

This was clearly the year for the least square regressions


2008 Winner:  Least Square Regression with team HFA
2007 WInner:  TSR SLOTS
2006 WInner:  TSR SLOTS
2005 WInner:  TSR SLOTS
2004 Winner:  CPA Rankings
2003 WInner:  TSR SLOTS
2002 Winner:  CPA Rankings
2001 Winner:  CPA Rankings
2000 Winner:  CPA Rankings




BEST RETRODICTIVE SYSTEM 2007  -  Simple system 

Winner:  system median

In this class, system median edged out the system average


2008 Winner:  System median
2007 Winner:  Least Squares
2006 Winner:  Warren Claassen
2006 Winner:  Warren Claassen
2005 Winner:  Peter Wolfe
2004 Winner:  Frank Alder
2003 Winner:  Anderson & Hester