Welcome to the 4rd NFL PT Awards.  The purpose of these
awards is to help raise the publicity for those systems that
have been superior in various qualities of interest.
Most awards will be based entirely on the numbers I've
monitored with my prediction tracker web pages.  In 2003 I
followed the weekly performance of 43 computer systems rating
the NFL.


  BEST STRAIGHT UP WINNERS (Entire Season)

Winner: Beck Elo

  My own Elo based ratings system was the leader in correct picks this
season with a record of 180-85, (67.92%).  This was one game better than
Hank Trexler.  These two systems were the only ones to get more games
correct than the Vegas line.
  Beck Elo's record this season is the best ever to this point.  There have
been a few systems with higher winning percentages based on less than a
full season but this is the highest winning percentage seen to date for
a system followed for a complete season.  This is actually the first major
award I have ever been able to give to myself.


 2003 Winner:  Beck Elo
 2002 Winner:  Herman Matthews
 2001 Winner:  JFM Power Ratings, ARGH Power Ratings
 2000 Winner:  CPA Rankings
 1999 Winner:  Kenneth Massey Ratings




  BEST AGAINST THE SPREAD (Entire Season)

Winner: CPA Retro Rankings, Steve Wrathell

  This might not be the most useful measure of the quality of a football
rankings system but betting on the NFL is big business.  And yes I know it
would be more meaningfull if I looked at a subset of games like when a system
was more than three points away from the line but all I have is every game results.
This season the computer ratings did a very poor job overall against the spread.
Only 12 out of 43 systems were over .500 for the year, and many of those were not
based on the complete season.
 The best system picking against the spread was the CPA Retro Rankings.
CPA Retro had a record of 134-121 (52.55%).  This is 5% lower than last
year's winner.  This is the lowest winning mark seen to date.



 winner 2003:  CPA Retro Rankings
 winner 2002:  FreeSportsPlay.com
 winner 2001:  CPA Rankings
 winner 2000:  Yourlinx
 winner 1999:  PerformanZ Ratings




  SMALLEST MEAN ABSOLUTE ERROR (Entire Season) 

Winner: Vegas Line

  The line jumps back on top in this categoy.  There really wasn't
much competition this year.  Then line had an average error of
10.6057 for all games.  The closest competitors this year were
Kasulis Enhanced Spread and Hank Trexler, both at 10.89.  These
are all quite a bit below last year's winner at 10.33.  The record
10.28, obtained by the line in 2001.


 winner 2003:  Vegas Line
 winner 2002:  FreeSportsPlay.com
 winner 2001:  Vegas Line
 winner 2000:  Vegas Line
 winner 1999:  Vegas Line




  SMALLEST BIAS (Entire Season)  

Winner: CPA Rankings, Steve Wrathell

 It was also a relatively bad year for bias.  CPA Rankings had the lowest
bias with a value of -0.10 points in favor of the road team.  In most years
the winning systems have been closer to zero.  Overall this season the
systems did not give enough points to the home team.  Not a single
system had a positive bias for the entire season data.
  One thing about these NFL ratings is that in almost every category a
different system is winning each season.


 winner 2003:  CPA Rankings
 winner 2002:  Sagarin Pure Points
 winner 2001:  ARGH Power Ratings
 winner 2000:  Pigskin Index
 winner 1999:  Flyman Ratings




  MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Entire Season)

Winner:  Vegas Line (opening)

  I deem the Vegas Line to be the most accurate system this year by
having the lowest mean square error in their predictions.  The only
strange thing about that is that it was the opening line rather than
the updated line.  THe opening line had a mean square error of 182.3.
This breaks last year's record of being the worst value to ever lead
the season.  The closest person that also had data for the majority of
the season was JFM Ratings with 189.5



 winner 2003:  Vegas Line  (Opening)
 winner 2002:  FreeSportsPlay.com
 winner 2001:  Vegas Line
 winner 2000:  Pigskin Index





                        SECOND HALF AWARDS 

 I like to look at the results over the second half of the season.  The
systems have had time to become 'burned in' to the season's data and
there is less reliance on starting values or previous seasons.
Second half data consists of all games from week 10 through the superbowl.


  BEST STRAIGHT UP WINNERS (Second Half)

Winner: JFM Power Ratings
Winner: Kasulis Enhanced Spread
Winner: Beck Elo


 For the third year in a row there is a three way tie.  With only two repeat
winners that makes 7 different systems have led this category over the last
three seasons.  This year the three winners are JFM Power Ratings, Kasulis
Enhanced Spread, and the full season winner, Beck Elo.  The Three systems
tied with a record of 97-40, (70.8%), this was just slightly better than
the three way tie at 69% that won for the last two season.  So in other
words, these three system tie for the best second have record seen to date.


 Winner 2003: JFM Power Ratings, Kasulis Enhanced Spread, Beck Elo
 Winner 2002: Least absolute value regression, JFM Power Ratings, CPA Rankings
 Winner 2001: Pythagorean Ratings, Elo Ratings, Ken Massey Ratings
 Winner 2000: PerformanZ Ratings, Scoring Effeciency
 Winner 1999: Ken Massey, Ed Kambor, Pythagorean Ratings




  BEST AGAINST THE SPREAD (Second Half)

Winner: Logistic Regression
Winner: Viacheslav Ugolnikov


  We also had a tie for best record against the spread in the second half.
Logistic Regression and Viacheslav Ugolnikov tied with a mark of 72-62,
(54.14%).  This is better than the full season mark but no where close
to the 60% we saw last year by least absolute value regression.  Even
though the winning percentage for the second half was better than the
winning percentage of the full season, overall very few systems even
managed to beat the .500 mark.


 Winner 2003: Logistic Regression, Viacheslav Ugolnikov
 Winner 2002: Least absolute value regression
 Winner 2001: Stat Fox
 Winner 2000: Pythagorean Ratings
 Winner 1999: Least squares using team specific home field advantages




  SMALLEST MEAN ABSOLUTE ERROR (Second Half)

Winner: Vegas Line

  Historically, the Vegas line dominates the mean absolute error category everywhere
except for here, the second half of the NFL.  But this year the Vegas line did
manage to come out on top again.  With a mean error of 10.4051 then updated line
barely nudges out the opening line.  Viacheslav Ugolnikov had the computer system
with the best mark other than the lines with 10.4707.
  The numbers here were very bad this year.  Last year FreeSportsPlays.com had
a mean error of 9.692 and the line holds the record of 10.3588 set back in 2001.


 Winner 2003: Vegas Line
 Winner 2002: FreeSportsPlays.com
 Winner 2001: Vegas Line
 Winner 2000: Pythagorean Ratings



  SMALLEST BIAS (Second Half)    

Winner: CPA Rankings

 This category was stunningly bad this season.  The typical computer system
was selling the home team short by about two and a half points.  That is
a very large number.  In fact, only one person had a bias of less than
one point.  That was Steve Wrathell with both of his CPA system.   The
smallest bias in the second half belonged to CPA Rankings with a bias
of -0.43.  I thought this category had stooped to a new low last season
but we managed to get worse this year.  For the last couple of seasons
home field has been largely underestimated in the second half of the season.


 Winner 2003: CPA Rankings
 Winner 2002: StatFox.com
 Winner 2001: Monte Carlo Markov Chain
 Winner 2000: Flyman Performance Ratings
 Winner 1999: Flyman Performance Ratings



  MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR (Second Half)


Winner:  Viacheslav Ugolnikov

  Showing that you don't have to be from America to enjoy rating football
our winner for most accurate system in the second half comes from
Russia, Viacheslav Ugolnikov.  Ugolnikov had a mean square error of 177.272
over the second half.  This was the only system to do better than the
Vegas Line.  It is also the second year in a row to set a new record
for the worst winner.  Last year it was 172.845, this year it has gotten
another 5 points worse.  The best ever was 165.954, set by Pythagorean
Ratings back in 2000.


 Winner 2003:  Viacheslav Ugolnikov
 Winner 2002:  FreeSportsPlay.com
 Winner 2001:  Hank's Power Ratings
 Winner 2000:  Pythagorean Ratings



   BEST PREDICTIVE SYSTEM (Entire Season)

Winner: Beck Elo

  For best overall awards I gave points to a system in each
category based on how well they do.  I then sum the points.  Hopefully this
gives an accurate read on who is the best system across the board.
  There didn't look to be any clear cut favorite to win this award this year
with each of the subcategories going to a different winner.  But the system
that came up with the most points was Beck Elo.  That makes four different
winners in four seasons.


 Winner 2003:  Beck Elo
 Winner 2002:  Matthews Grid
 Winner 2001:  CPA Rankings
 Winner 2000:  Ed Kambour



  BEST PREDICTIVE SYSTEM (Second Half)

 Winner: CPA Rankings, Steve Wrathell

 This one turned out to be a little bit of an upset.  There were several systems tied
at the top in winning percentage and CPA Rankings was four games back.  But since
CPA Rankings was very competitive across all categories and far and away superior
in the bias category it come out as the best across the board predictive system
over the second half of the season.
  This also makes four winners in four seasons for the second half as well.


 Winner 2003:  CPA Rankings
 Winner 2002:  Kenneth Massey
 Winner 2001:  Monte Carlo Markov Chain
 WInner 2000:  Pythagorean Ratings





                RETRO AWARDS 

  The retrodictive categories are based on each systems final
standings and measure things by applying those final ratings
to the entire season in retrospect.

As I did in the college awards I am going to break this down
into big systems (estimate team specific home field advantage)
and simple systems (do not estimate team specific home field advantage).



  MOST RETRODICTIVE WINS - Large System 

Winner:  The Sports Report

 The best retrodiction record goes to The Sports Report SLOTS system with
a record of 222-45 (83.146)  This easily beats the previous record
of 76.1% set by the system average back in 2001.  The closest competitor
finished 15 games behind the SLOTS system.


 Winner 2003: The Sports Report SLOTS
 Winner 2002: The Sports Report SLOTS
 Winner 2001: The system average
 Winner 2000: CPA Rankings



  MOST RETRODICTIVE WINS - Simple Ssystem 

Winner:  Frank Alder

 The best retrodiction record for those not fitting a team home field
advantage come from Frank Alder, 203-64.  This is 19 games worse than
the large system winner.


 Winner 2003: Frank Alder





  SMALLEST RETRODICTIVE ABSOLUTE ERROR - Large System

Winner: Least Squares with team home field advantages

  Least squares using team specific home field advantage wins this award
for the fourth straight year.  It has a winning average error of 9.16 points.
This is a fair amount worse than the past couple of years which have all
been under 9 points.


 Winner 2003: Least squares with team specific home field advantage
 Winner 2002: Least squares with team specific home field advantage
 Winner 2001: Least squares with team specific home field advantage
 Winner 2000: Least squares with team specific home field advantage





  SMALLEST RETRODICTIVE ABSOLUTE ERROR - Simple System

Winner: Sagarin Predictive/Points

 I decided not to consider the average or median systems to be simple systems.
That leaves Sagarin Points to be have the lowest absolute error among
the simple systems.  Sagarin's error of 9.72 is only good enough for 9th
place overall.


 Winner 2003: Sagarin Points





  SMALLEST RETRODICTIVE BIAS

Winner: CPA Rankings, Steve Wrathell

  For the bias category I do not split into large and simple subclasses.
There is really no reason for a large system to be better than a simple
system in this category.
  The smallest retro bias this year was observed in Steve Wrathell's
CPA Rankings, with a bias of -0.009.   The numbers are not terrible
here this year.  On average the bias tends to be toward the road team.

 Winner 2003: CPA Rankings
 Winner 2002: PerformanZ Ratings
 Winner 2001: Herman Matthews
 Winner 2000: Massey Ratings




 MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR - Large system


Winner: Least Squares with team home field advantages

  As with absolute error, Least squares with team specific home field advantage
takes this award for the fourth straight season.  The winning mean square
error was 134.439.  This is down from last year's record of 126.085.
CPA Rankings was a reasonably close runner up at 138.824.



 Winner 2003: Least squares with team specific home field advantage
 Winner 2002: Least squares with team specific home field advantage
 Winner 2001: Least squares with team specific home field advantage
 Winner 2000: Least squares with team specific home field advantage



 MOST ACCURATE SYSTEM - SMALLEST MEAN SQUARE ERROR - Simple system

Winner: Sagarin Predictive/Points

 I decided I would not consider the average or median to be simple systems.  That means the
top simple system was Sagarin's Points or predictive system, which was good for 9th
place overall.  Sagarin's mean square error was 155.720.  Several other systems were very close
behind.


 Winner 2003: Sagarin points





      BEST OVERALL RETRODICTIVE SYSTEM IN 2003  -  Large System

Winner:  The Sports Report

  I guess it is important to mention that this award, as with the
best predictive systems, go to the system that is the best overall,
not in just one category.  As with the other overall awards, points are
assigned to systems in each category then summed up.  The system with the
most points wins.
   This year that system was TSR SLOTS.  The difference maker this year
was retro winning percentage.  There were a few other systems that were all
relatively close in all the other categories but TRS SLOTS huge lead of 15
wins clearly set it apart for the overall title.  CPA Rankings and CPA Retro
finish in almost a tie for 2nd/3rd.



 Winner 2003: TSR SLOTS
 Winner 2002: Least Squares with team home field advantages
 Winner 2001: CPA Rankings
 Winner 2000: CPA Rankings



  BEST OVERALL RETRODICTIVE SYSTEM IN 2003  -  Small System

Winner:  Frank Alder

  Among the simple systems I give the overall title to Frank Alder.  His system was
only good for 7th place overall but it was the highest among the simple systems.  The two systems
closest behind Alder were Dunkel and Beck Elo.

Winner 2003:  Frank Alder