Evaluating Managers, Again
For those of you interested in research in trying to quantify a manager’s effect on a team’s won-loss record, you’ll enjoy this article by Chris Jaffe, the third part in a series of extremely long articles he’s written on the subject. Building off work by Phil Birnbaum, he’s taken steps to try to evaluate managers in historical context, and today’s piece takes a shot at attempting to evaluate active managers.
His methods are a bit crude, but probably in the ballpark of being useful. Essentially, he’s taken a huge sample of statistical simulations from three different projection systems and compared expected team performance to actual team performance. Now, there’s a lot of reasons a team can underachieve or overachieve a statistical projection, with injuries and flaws in the projection systems being two of the big ones. But managerial impact is at least part of the difference, and Chris is right that the noise should begin to be filtered out when looking at a decent sample. He’s using six years of data for this piece, which may not be as much as we’d like, but is enough to avoid cries of small sample size.
Anyways, that’s enough talk about the methods. What you guys care about are the results, right? Thirty four managers worked in at least three seasons, qualifying them for the study. I won’t put the whole list here (you should read Chris’ piece anyways), but here are the parts Mariner fans care about:
#1: Lou Piniella, +49
#28: Bob Melvin, -13
#34: Mike Hargrove, -28
This data needs to be taken with a grain of salt or ten, but according to the data, the Mariners replaced the best manager over the last six years with the 7th worst, then replaced him with the worst of all. In addition to the numbers, Chris wrote a little blurb on each manager. Again, the relevant parts for Mariner fans:
Well, someone’s got to come in last. His only really bad year was 2005 when the Mariners were 13 games under their projection. From 2000-6, exactly 60% have scored at least a â€“2 in their projections. Hargrove’s been â€“3 or worse four times in six years. Only one of his six clubs have exceeded expectations. That was 1992 when the O’s scored a big +2. He was â€“12 with the Orioles, and â€“16 with the Mariners. I’m glad he’s not managing my favorite team. The Birnbaum database likes him through 2001 putting him around +175 runs, which was almost entirely built up by his Pythagoras W/L record.
He actually scores well in three of his four seasons, but the 2004 Mariners were 21 games under projection. All four of their main starting pitchers fell apart. Most disturbingly, both their young studs, Gil Meche and Joel Pineiro, collapsed. I don’t know if he overworked them the year before of if there’s something else going on. Looking at it, Meche wore down considerably in the second half of 2003, and Pineiro couldn’t get anyone out that August. Now in Arizona, history’s not repeating itself with Brandon Webb. In fact, ChadBradfordWannabe thinks Webb’s mechanics have improved since 2004.
I can’t believe he’s as good as this system claims he is if for no other reason than I have too much respect for some of the other managers to think Piniella could dominate them so much. Then again, I think enough of this system that I have to think he’s done a real wing-dinger of a job over the last couple years. As mentioned earlier, he’s the best at orchestrating immediate dramatic improvement since the death of Billy Martin. His score of +49 wins here is off the charts. How the hell did he do that good? Well, the 2001 Mariners didn’t hurt his mark. They’re the biggest overachieving team under study here, at +29 games. They’re also the biggest overachievers in the Birnbaum database. Take that away from him and he’s “only” sixth best. That’s not a fair comparison, however. Let’s take everyone’s best season away from them and see how they compare. Here’s the top five without their best season:
1. Lou Piniella +20
2. Ron Gardenhire +17
3. Mike Scioscia +13
4. Ozzie Guilen +11
5. Bobby Cox +10
He’s still the king, and only one man is within two-thirds of him. Piniella’s aided by only having one negative year, a mere â€“3 in 2005. Guillen has no negative years, but he’s only been around 3 years. Gardy has two. Granted, they’re only â€“2, and â€“1, but he hasn’t managed as much.
The Birnbaum database didn’t like Piniella at all. In part two I explained why I disagreed. Anyone who can capture a ring with the 1990 Reds and win 116 wins in Seattle has to have something going for him. I’m amazed how much this system likes him, though.
Piniella scores +36 with Seattle and +13 with Tampa Bay. Maybe it’s easier to exceed projections when you’re supposed to win as few games as the Rays are always calculated at, but the other Tampa managers are a combined â€“19 wins. Perhaps he’s not as good as he once was. The aging patterns info works against him, but he’s earned the right to show if he still has it.
Anyways, like I said, the idea is interesting, but the methods are so blunt that you can’t take the results with any kind of precision, and even the general conclusions reached have to be tempered a bit. Lou Piniella comes out looking great, but 2001 was obviously a huge part of his big positive addition, and while he certainly deserves some credit for that season, I’d imagine this system is overestimating his abilities quite a bit.
So, yea, even the projection systems think Mike Hargrove is pretty lousy.