In the interest of objectivity and illumination, we present an unusually informed addition to the debate on the merits of the RPI:
Joe,
If you wanted to compare work that I do with the RPI, you should have compared my "ELO CHESS" with the RPI (the "ELO CHESS" is the column in red on my website ratings). That's because "ELO CHESS" doesn't use score at all. And, if you were to read the explanation over my rankings, you'd see that my "RATING" is a synthesis of "ELO CHESS" and the "PREDICTOR" (which is pure points).
But I can tell that you had a preset agenda, which was to back the RPI. All mathematicians that I know who have investigated the mathematics of the RPI burst out laughing when they see what results it produces and the reasons why.
I've attached a file showing the results given by your beloved RPI when applied to this most recent college football season and the 240 Division I teams that I rate: 117 Div I-A and 123 Div I-AA. You could not possibly keep a straight face and claim that those are valid ratings.
College Football RPI ELO CHESS (BCS)
1. USC 1. Ohio State
2. Ohio State 2. Miami, Fla.
3. Georgia 3. Georgia
4. Miami, Fla. 4. USC
5. McNeese State (AA) 5. Oklahoma
6. W. Kentucky (AA) 6. Iowa
7. Oklahoma 7. Texas
8. Texas 8. Michigan
9. Michigan 9. Notre Dame
10. Kansas State 10. Alabama
11. Alabama 11. Kansas State
12. Washington State 12. NC State
13. Iowa 13. Washington State
14. Pennsylvania (AA) 14. Maryland
15. Notre Dame 15. Florida State
16. Georgia So. (AA) 16. Auburn
17. Florida State 17. Colorado
18. Villanova 18. Texas Tech
19. Maryland 19. Penn State
20. Texas Tech 20. West Virginia
21. Colorado 21. Florida
22. Dayton 22. Virginia
23. Arkansas 23. Boise State
24. N.C. State 24. South Florida
25. Auburn 25. Arkansas
Then, compare those to my final football ratings on the web, in particular the ELO CHESS (BCS), and you'll be seeing ratings that actually do what you purport the RPI does, which is have good, legitimate ratings.
Regards,
Jeff Sagarin
P.S.: The reason I print "RATING", "ELO CHESS", and "PREDICTOR" is to give the different readers what they're looking for. Those who want my best overall rating, that takes into account W-L and scores, will like the "RATING". Those who care only about W-L and don't like the use of scores will gravitate toward the "ELO CHESS". And those who care solely about who's going to win tonight and by how much will prefer to concentrate on the "PREDICTOR".
Personally, I like my "RATING", since it takes into account both W-L and scores. (But) all three of them, of course, take into account who you played and where you played them, and who they played and where they played them, ad infinitum.
P.P.S: It's frustrating to have spent so many years putting what I do together, especially the "RATING", which is truly the synthesis of the two philosophical opposites, and then have you just wave your hands and dismiss them as a totally idle exercise.
If you think about it, if you were to move to a new town and the local people were to tell you that the local high school was doing great, your natural order of questions to them might be "say, what's their W-L?" and, "by the way, what kind of scores are they playing to?" Oh, and, by the way, "what are the records of their opponents in the same vein and, furthermore, where have all these games been played?" The human mind intuitively knows that scores do mean something and are not random noise.
Remember, the purpose of the "ELO CHESS" is to rate teams as well as possible using only who won and who lost and where were the games played, while completely ignoring the score margins. If you were to compare it to the RPI, I think in your heart you'd have to concede it did a much better job.
In the RPI, you can beat a team and end up going down and have the team you beat going up. And, in the RPI, the home edge is totally ignored and yet, in real life, college basketball home teams win 2/3 of the games. The reason home-away is so important is that the months of November and December are when all the conferences interact and the big conferences get to be the home team in the huge majority of the contests, thus building up an impenetrable rating lead over the small conferences.
This is such an excellent discussion, probably the best of its kind in a forum this public, that I'm not even sure where to begin my response. So I'll make the following points, not necessarily in a particular order:
I had no preset agenda in writing about the RPI, other than to say it should be recognized equally for what it does right as well as what it does not. At this time of year, we tend to hear only the latter part of the argument.
The RPI is neither "mine" nor "beloved" by me. I use it to varying degrees because the Selection Committee does. My job is to forecast what the committee will do on Selection Sunday, not make national policy on how teams should be ranked. If the committee's published criteria listed ELO CHESS, the polls, barometric pressure, whatever, then I would have to include the same in my projections.
I'm not a mathematician, and surely Jeff has forgotten more about such formulas than I will ever know. Come to think of it, the few mathematicians I know also burst out laughing, usually at my SAT scores!
It seems to me that the Division I-AA teams included in the college football RPI above would have to be ranked separately, the same way the NCAA sub-divides its divisions in rating college basketball teams. What I find ironic (and surely it is only that) is that the two teams most folks would like to have seen in a one-game college football playoff -- Southern Cal and Ohio State -- are 1-2 on that list. But the truth is I don't follow college football closely enough to argue the merits of the two lists above, so I'll take Jeff's word on the superiority of ELO CHESS.
Is the college basketball RPI better than its football counterpart because the sample size of games is so much larger? I have no idea, but the question seems valid. I mean, I've seen partial season RPI numbers in basketball and they aren't worth squat.
I admit to not knowing nearly enough about the ELO CHESS method. What I do know is that the RPI is biased in favor of power conference members and that it doesn't reward good road/neutral play nearly enough. If, as Jeff suggests, ELO CHESS or a derivative has adequately addressed those dilemmas, I'd be the first to advocate for it.
I did not "wave my hands" and dismiss Jeff's work as an "idle exercise." I did express a preference for the RPI for the purposes of my bracket projection work, and we'll just have to agree to disagree on the importance of scoring margin in the team evaluation process. I am also under the impression (if wrong, I apologize) that scoring margin has been de-emphasized in the BCS calculations. If so, I am evidently not alone on my side of that coin.
The great thing about college basketball, of course, is that none of these formulas have to be exact. We're picking 34 at-large teams, not two BCS finalists, and there is a significant human element involved. Because of that, and as I've written many times, any quantitative rating system remains merely one tool in the toolbox of team evaluation. Today, because of the committee's emphasis, the RPI is that tool.
After the season, when I have more time, I'm going to contact Jeff and learn more about ELO CHESS (including what the name means!). In the meantime, I'm going to run RPI and ELO CHESS for college basketball side-by-side and let the rest of you begin to draw your own conclusions.
TEAM RPI (Feb. 23) ELO CHESS S-CURVE
Oklahoma 1 8 4
Texas 2 9 5
Arizona 3 2 2
Kentucky 4 1 1
Notre Dame 5 10 10
Georgia 6 15 20
Utah 7 14 19
Duke 8 7 9
Florida 9 3 3
Louisville 10 5 6
Syracuse 13 13 12
Kansas 15 16 8
Xavier 16 12 11
BYU 18 28 31
Stanford 19 18 14
Pittsburgh 21 11 15
Illinois 27 24 18
Saint Joseph's 29 19 27
Creighton 34 23 21
Connecticut 35 33 26
Butler 40 21 39
Weber State 44 39 43
Oregon 47 44 32
Just for fun, I added a third column -- my S-Curve (1-65) ranking of teams for this week -- to see how my overall analysis, both quantitative and qualitative, compares with these other methods. At a quick glance, ELO CHESS does seem to have an edge on the RPI (at least in comparison to projected seedings). And it's clear I don't use the RPI any more than the single tool that it is.
Joe Lunardi is the resident Bracketologist for ESPN, ESPN.com and ESPN Radio. He may be reached at bracketology@comcast.net.
