Evaluating Managers

DMZ · October 15, 2004 at 12:16 pm · Filed Under Mariners 

I rank a manager’s jobs in importance to remaining employed as:
- Keep his team happy and motivated or, barring that, motivated
- Deal with the press
- Pick the players for each day’s lineup
- Manage the pitching staff
- In-game tactics
- Team strategy
- Prepare his team for each series and each game
- Arrange the day’s lineup

This is not how important they are to his team’s success. A manager could be abjectly horrible in post-game press conferences, cold to reporters in general and terrible in his call-in radio show, and it wouldn’t cost the team a run. But it would affect perception of him tremendously, and the baying dogs would chase him night and day.

It could cause more than a run’s damage, too. You can see where if a manager was constantly under fire from the media (and we can safely assume, the fans, because he’d come off terribly on TV), it might spill over to the coaches and players and cause them pain. So even the seemingly trivial skills tie into the first and most important: keeping the team playing. If a manager’s slagging his team after every game, it might motivate them, but it’s much more likely to piss them off and make them not want to give their best effort, our of spite and because doing so would make the manager look good.

Bobby Cox is great at keeping his mouth shut: it’s hard to come up with an instance where he’s said anything bad about any of his players. It makes him a little boring, but if you’re a player, I can understand that you want him to chew you out for a mental error in private and protecting you in public.

But again, we don’t know the difference in actual team performance between a Cox-lead team and, say, a team led by a frustrated Lou Piniella.

The value of keeping a team happy and motivated is hard to objectively measure. Do the players complain about the team or praise it? Does that really have more to do with players or the manager? At the extremes, this is noticeable: players generally love being on Dusty Baker’s teams, even as he gets them to play bench roles or platoons. Some managers, who don’t last long, can’t get the respect of their players. But in between, it’s quite hard to tell the difference betweek the good and that below-average.

Trying to guage it by performance is tough. Almost every team plays horribly for some stretch of the season. They bobble balls, make stupid baserunning mistakes, chase bad pitches every at-bat. And even the worst teams put together a week or two where they make fans wonder where they’ve been all season. There are some danger signs a fan can spot. But do the players consistently seem to space out, or give half-assed efforts defensively? Do they bitch about their playing time and the team’s direction? Do plays break down when signs are missed or misunderstood?

On the whole, though, we’re a long way from being able to measure the ability of managers to motivate their players.

Let me try that list again in terms of things we can attempt to measure:
- Pick the players for each day’s lineup
- Manage the pitching staff
- In-game tactics
- Team strategy
- Arrange the day’s lineup

Daily lineups
Picking the players who take the field is the most important part of a manager’s job. A manager has great leeway on how to put together each day’s team, and they get to use it 162 times a season.

This is also where it’s possible to evaluate a manager’s preferences and do quality second-guessing.

For instance, say the team has a young corner outfielder who plays good defense and hits like crazy against both left and right-handed pitchers. But the manager insists on platooning him with a weaker, scrappy hitter he likes.

There are two possibilities: one is that the manager sees something flawed in the kid’s approach, and is covering it. The other is that he’s making a mistake, and valuing insubstantial factors over actual performance. This happens all the time with managers picking veterans over minor-leaguers, even minor-leaguers with amazing track records.

There are sometimes considerations that outweigh baseball reasons. If the team wants a manager to showcase a player for a trade, the manager might play them in a strange position, or start a horrible pitcher in the hopes that he’ll string together two good games and convince someone to bite.

But by and large, how does the manager distribute his playing time? Does he platoon and find ways to allow his players to succeed, or does he somehow get the crappy utility guy 250 ABs instead of 110? Is he able to find ways to rest his regulars without hurting the team’s defense or offense too badly? If a regular player goes down, how does he patch the lineup for the week, and how does he adjust the lineup?

Manage the pitching staff
The most obvious criteria is whether they regularly overuse pitchers. To distill the years of debate on this, throwing over 120 pitches a start is bad for the pitcher’s next start and their overall health.

Baseball’s been remarkably civilized about this in recent years. Most pitchers never go over 120 in the course of a season, and the per-game average for even the most heavily-used pitchers is around 110.

Less obvious is the “quick hook” and “too late to pull” qualities. If a pitcher’s clearly tired and throwing poorly, does the manager get them out early, or do they allow them to keep throwing and lose the game? Conversely, are they so aggressive that they remove good pitchers after brief struggles? It’s hard for us to easily measure this. We can try to look at quality starts and blown quality starts, but those are imperfect measures.

As with picking the lineup, there’s also some effect in setting pitchers up for success, and protecting them from failure. While generally a manager’s going to be better off with the pitchers on hand compared to trying someone new, there are instances in a season where there are chances worth taking. For instance, it can be worth pitching a left-hander against a team even if that means calling up a random AAA guy and re-working the rotation if the opponent is so weak against left-handers. A good manager constantly looks for chances to make adjustments that let him steal those advantages, and a poor one does not.

The bullpen management provides many way to evaluate a manager.

I reject, immediately, the notion that a team needs a 9th-inning-only closer, and that closer needs an 8th-inning setup man. This is a dead end in baseball’s evolution, and eventually we’ll get past it. If a manager displays flexibility and insight in playing matchups over roles, especially if they’re willing to call in their best pitcher when they most need them, whether or not it’s the ninth and they’re up by 1-3 runs, that’s a great thing. Few managers are that smart.

In general, how does the manager use his relievers? Does he recognize talent, and use it when it’s important, or does he keep ineffective veterans

In a more subtle sense, does the manager seem to take the strengths and weaknesses of his pitchers into account? If a reliever is overall quite good but has a home run problem, and there are two runners on with a slugger up, up by two on the road, you’re better off bringing in a less-effective groundball pitcher. In individual instances, good moves will backfire and bad moves will work out sometimes, but if you continually see the manager bring in a homer-prone reliever in situations where a home run could cost the team a game, that’s a good indication he’s not being smart. You should be especially worried if their justification for a particular move displays a remarkable ignorance of the facts — say, they bring in a homer-prone flyballer instead of the groundballer, and then claim that the flyballer was the best bet to get a ground ball double play to get out of the inning — because if they don’t know their staff’s talents and weaknesses, they’re not going to do well trying to play to them.

In both the bullpen and the rotation, are they willing to make changes when changes are required, or will they stick with a player in a role they’re unsuited for long after it’s obvious to everyone else?

In-game tactics
First, the running game, because this is something we can easily set a few metrics for. Most statistical research will tell you that a stolen base needs to work 75% of the time to be effective. There are situations where one run is important enough that the reward makes the extra base worth more, and it’s worth trying a steal if you think there’s only a two-thirds chance they’ll make it. You can also argue the threat of the steal has value, so it’s worth a few counter-productive steals to establish that.

But really, anything less than a 67% success rate means the manager, in total, is taking runs off the board and hurting his team. As I write this, the success rate of teams ranges from 80% (Mets) to 59% (Braves). The Mets advanced 90 more bases without creating an extra out. That’s a huge swing in the team’s fortunes over the course of a year, and easily two games in the standings.

So there are two things to look for:
- The team’s total success rate at stealing bases and number of attempts
- Who’s stealing

The Braves sucked at stealing bases, but they also were close to the back of the pack in total attempts. While we could argue that they should have taken it even farther back, they weren’t running themselves ragged counter-productively.

Meanwhile, the Mets were among the best and took advantage of their strength. The Phillies had a great success rate and didn’t, for reasons I’ll get into. We can say though that we know that the Mets did a good job with their running game, and though the Braves weren’t, they at least didn’t try and push it, as the Nationals and Marlins did with only slightly (66%) better success.

For the second, who’s stealing, let’s take a look at the Phillies and Mets, both of which had good years on the bases.

The Phillies have one runner, Rollins, who is 36-4, Abreu was 20-4, Chase Utley is 14-4, and Aaron Rowand is 10-4. No other player has more than 5 attempts.

The Phillies are having most of the team sit still, the guys with good speed taking good chances, and Rollins has a green light to steal at will. That’s an excellent strategy: as we don’t want teams to steal for a low success rate, it only makes sense for them to steal with players that can do it well enough to swipe successfully at least two-thirds of the time.

The Mets are nearly the same way: Reyes is the speedster, Wright and Beltran have 25 and 20 attempts, and Endy Chavez has 15. They have more players who’ve gone under ten times, but you see nearly the same pattern: they’re playing station-to-station with the slow guys and letting the faster ones win.

This seems like an obvious strategy, but it’s not at all. Many teams, for whatever reason, keep stealing with guys who get caught. Florida’s Alfredo Amezaga has 19 SB that cost the team 12 CS. Jamey Carroll has 10 SB to 12 CS.

Next, for another quick measure of a manager’s brain activity, look to the team’s count of sacrifice bunts. The sacrifice bunt is almost never a good move. The more a manager uses it, the worse they are. The more they can resist the temptation to manage by the book, the better off the team is.

And it’s also a situation where managers vary enormously. This year, Colorado was credited with 116 sacrifice bunts. Toronto made only 16. Colorado sacrificed over seven times as often. That’s enormous. Of course Colorado’s an NL team, so that’s an unfair comparison, because NL teams use their pitchers to sacrifice all the time. As you’d expect, the top 16 teams are all NL teams, and the bottom 14 AL, but it’s not as stark a division as you’d think. Here are the ranges as I write this:

AL Teams: 16 (Toronto) – 50 (Kansas City)
NL Teams: 56 (Philadelphia) – 116 (Colorado)

KC was 2.5 times as likely to sacrifice as Toronto, and Colorado 2 times as likely to sacrifice as Philadelphia.

Managing the Phillies, if you’re curious, Charlie Manuel, formerly of the Indians.

There are other subjective measures a fan can look at:
- when are they stealing?
- are the sacrificing in really stupid situations where giving an out is dumb, or are they relatively smart choices?

Beyond that, it’s hard for a fan to make evaluations of a manager’s in-game strategies. Too much of it has so many factors that go into a decision that it’s hard to call any single decision wrong, or right. You can still try, though, to answer another big question:”Does the manager make effective use of his bench during games?” Or, too see this another way, does the manager find ways that the bench players can get into games in a way that allows them to contribute?

If he has a player who is a clear liability defensively, does he find a defensive caddy for them, and make use of them? If there’s a guy on the bench who can’t play much defense but is a huge left-handed power hitter, does the manager seize chances to send him in to pinch-hit? Or does everyone on the bench rot?

Even if you hold back criticizing particular instances of pinch-hitting or substitution, you can still get a sense of the manager’s priorities and beliefs that allow you to make judgments about those beliefs.

For instance, say a manager never pinch-hits for his catcher, wishing always to have two catchers available (one in the game, one on the bench). As a philosophy, this is clearly risk-averse and can cost his team runs: if the backup catcher is a .150 hitter and it’s an extra-inning game with runners on where he has some quality pinch-hitting options, the risk is worth it.

And so on. You may also notice if a manager is particularly inflexible about rules, which may not only be bad because the rule is dumb but made worse because unwavering adherence to those rules means that other managers can take maximum advantage of them.

Team strategies
Do they run the bases aggressively, or conservatively? Should pitchers make the fastball their primary pitch and throw it at least 70% of the time?

These are extremely hard for fans to judge. We’re reliant, generally speaking, on people who have access to play-by-play databases and pitch distribution information to crunch these kind of numbers for us.

Arrange the day’s lineup
I get a little annoying about criticizing lineups because I think teams should relentlessly chase every advantage, trying to win every game any way they can, but lineup simulations and discussion around this has generally agreed that it’s not that big a deal. The effect of having Joe Scrub playing anywhere instead of a good player is much greater than the difference between an ideally ordered lineup and a badly ordered one.

That said, it’s still an area where the manager controls his own destiny, and really should try and get what he can. There are many considerations into where each player goes.

Baseball-wise
- L/R handed (force the other team to burn relievers if they seek situational advantages)
- Hitting skill profile (contact/power/pitches seen/etc)
- Special hitting abilities (can the player bunt, or steal)

For example, one of the problems with the Olerud/Martinez lineups is that while both of those guys were offensive threats, they could not run the bases worth beans. They weren’t good at advancing on hits, they were easy to double-up at second, and once on, they clogged the basepaths for players behind them. You can’t go first-to-third on a single if a slow guy is barely going to manage to get second-to-third on the same hit.

It’s an interesting dilemma. The chances that the base clogging hurts the team during the course of a game are pretty slow, but it will happen, and then you’ve hurt the team. Wherever you put them in the order, putting speedy runners after them will negate part of the value of their speed, and putting ground-ball hitters behind them will dramatically increase the GIDP rate.

Intangible-wise
- traditional fits of players to lineup roles (#1 is highest OBP, #2 high contact+bunting, #3 best hitter…)
- player mental fit (does the player try to hit to all fields when hitting #2 but not #6)

This is where a lesser manager says “screw it” and slots his guys into the traditional order their skills suggest. If that means a .245 hitter with great speed leads off, well, so be it. Good managers will come up with lineups that make the most of unconventional talents. We’ve seen this on some of the A’s teams, where devoid of speed they manage to still put together an order where good baserunners with high OBPs bat early.

For a manager, taking the easy path means that they don’t have to spend a lot of time being hassled about it in the press, because any deviation from the most conventional lineup means they’ll be at least questioned about it at length if the team loses. But if they go with the most conventional arrangement and they lose, it’ll go without comment.

Lineup arrangement is a strange art. It’s extremely complicated, with many factors, some of which we can’t know from the outside. But that doesn’t mean that we can’t think about it, and compare our work to the managers, and it certainly doesn’t mean that they’re doing it right.

What can we know?
At a high level, we can evaluate easily only the running game and use of the sacrifice bunt. They’re readily available, sortable to compare against other managers, and useful. We can track pitcher workloads easily.

If you’re willing to spend time on it, who gets playing time and how the relievers are managed are crackable, though it’s almost impossible to do that for every manager, every game, to see how

But unfortunately, so much of the information we need to make comparisons is proprietary, and there aren’t readily usable tools for fans to use. It’s really a shame.

Hargrove sucks
It’s true.

Update: for some interesting data, check out Chris Jaffee’s study: 1, 2, 3. I’m not sure why the links don’t work on their own — thank goodness for the Wayback Machine.

Comments

38 Responses to “Evaluating Managers”

  1. Aaron on October 15th, 2004 12:49 pm

    Assessing the in-game effect of a manager involves so many variables**, it won’t ever happen. A better way to look at who a tea wants is to ask, “who do we want to lead the post-game news conferences?”

    ** The out-of-game effects can be measured a little easier, but there are no stats for pitching injuries, personality clashes with players and ownership, etc.

  2. chris w on October 15th, 2004 12:54 pm

    DMZ, you said “the fact that managers are so hard to statistically evaluate should help us to realize that they’re really not that important.” That’s wrong, in my opinion. Substitute “managers” for “defense” in that sentence and you can see what I mean. Some managers, on some teams, get significantly more wins out of their teams than other managers would with the same teams. The problem is that it is impossible to know which manager will get the most out of what team, and when.

  3. PositivePaul on October 15th, 2004 12:54 pm

    Actually, I agree that it cannot be statistically proven how well a particular manager may or may not do with a particular team. The only thing we’re left with, really, is more of a personality quotient. Measuring a manager’s “success” really has more to do with the entire environment surrounding the game as a whole. A manager, then, can be measured by more abstract elements:

    – Level of player performance (i.e. do the players’ performances as a group (and individually) hover above, right at, or below their expected levels. How does the manager affect players’ performances?)
    – Clubhouse atmosphere (i.e. how does the atmosphere in the clubhouse appear after wins, big wins, losses, and big losses. Is the manager contributing to or detracting from this atmosphere)
    – Decision-making ability (i.e. do armchair managers constantly second-guess these decisions, or do they validate the decisions and approve, generally, of them).

    In the end, really, the manager acts as the face of the team. When the team sucks, it falls on the manager. When the team excels, the manager is glorified. Sure, that’s not fair, and it’s not really realistic, but that’s how it works (and why BoMel was canned).

    The type of manager we need very much depends on the type of team that is built. If we’re going with the youngsters, scattered with a few veterans, than I’d be inclined to lean on Rohn, who knows the youngsters very well. If we’re going to sign several veterans (however “old” they may be), then we’ll need someone that can fit with them. Since the manager is, really, the face of the team, it’s the manager who has the least actual affect on the outcome of the game (really, in the long run, even the GM has more control on the outcome of the season than the manager). But, a good manager can help a bad team overachieve, as can a bad manager heavy-laden a good team to underacheive.

    Okay, I’ll shut up now. I’m probably not making sense and using circular logic to try to explain something that cannot be explained concretely. It’s just nice to have y’all to help me hack out my feelings and provide some semblance of reality to formulate my opinions.

    After all, it’s just a game…

  4. Evan on October 15th, 2004 12:55 pm

    If the manager doesn’t have much impact (or much discernable impact) on the team’s performance, my preference then swings toward managers who are entertaining. Lou was entertaining. I think Brundage would be entertaining. Cito Gaston = Not entertaining.

  5. DMZ on October 15th, 2004 1:18 pm

    DMZ, you said “the fact that managers are so hard to statistically evaluate should help us to realize that they’re really not that important.” That’s wrong, in my opinion. Substitute “managers” for “defense” in that sentence and you can see what I mean.

    It’s true that defense is harder than offense to evaluate, but there’s no comparison between defense and managerial value.

    A team’s defense is easy to evaluate: do they turn balls in play into outs? If so, that team allows fewer runs.

    Individually, even the worst measures allow us to make general distinctions between players: if you make 80 errors as a 2B, you stink. If you make 0, you’re good — even as we recognize that errors aren’t a good measure to use. When you get into advanced defensive metrics for defensive evaluation, it gets even better.

    But for managers, there’s no clear evaluation metric that’s even in the ballpark.

    Some managers, on some teams, get significantly more wins out of their teams than other managers would with the same teams. The problem is that it is impossible to know which manager will get the most out of what team, and when.

    If manager ability is so subtle in its effects, it would be not worth speculating about.

    Again, take defense: put the 2003 Mariner outfield and graft it onto the worst defensive teams in baseball. They would all improve dramatically, allowing far fewer runs. Now, that number will vary based on the team’s makeup, whether their pitchers are flyballers, or whatever, but it’ll be between 20-50 runs saved for every bad defensive team you sub them in for.

    There is no manager that we can show has that kind of effect on a team. There is no manager we can show that has a consistent effect of any kind like that.

  6. mfan on October 15th, 2004 1:21 pm

    I happen to be one of the “statheads” as I’m about six months from my PhD in economics. The term “statistically proven” is wrong. Statistics never PROVE anything. Instead, statistics are meant to support arguments. I realize that there are deficiencies to any statistical study and many things that are left out for various reasons, such as the fact that you can’t measure clubhouse atmosphere. However, when people realize that statistics aren’t meant to prove a point, but instead meant to further a point that already has logic behind it, they seem to be less skeptical. To go along with the statistics should be some sort of anecdotal evidence similar to what PositivePaul referred to in 3. I think statistics can be useful for evaluating managers, but they are by no means a way to “prove” that a manager is good or bad.

  7. The Ancient Mariner on October 15th, 2004 2:09 pm

    If manager ability is so subtle in its effects, it would not be worth speculating about.

    That’s actually not the point at all. It isn’t that the effects of managers are subtle, necessarily (though one would expect that for the majority of managers, following the typical bell curve), but that they are hard to isolate. What were the effects on American domestic and foreign policy of JFK’s assassination? One can speculate, but it’s impossible to know with any concrete certainty. Does that make them “subtle”? Not in the slightest. It just means that a) there are a vast number of interlocking variables to consider in any such analysis, and b) that any such analysis essentially requires making a “what if?” concrete–which we can’t do because we have no way of actually knowing what would have happened had things been different. So it is with managers.

    However, while that seems to rule out statistical analysis of managers as a means of generating valuable information, it doesn’t rule out observational analysis. In other words, we’re back to scouting, albeit on a level as much social and psychological as anything. It can’t be anything approaching an exact science, but I for one do believe it matters, and can matter quite a bit. Baseball managers probably have less effect on their teams than head coaches in any other sport, because so much of the game comes down to uncontrollable events in the batter-pitcher matchup, but that doesn’t mean they are largely unimportant; and if you can find a great one, or especially a HOF great like Earl Weaver, it’s very important indeed.

  8. DMZ on October 15th, 2004 2:55 pm

    That’s my point! That’s my point exactly!

  9. Paul Molitor Cocktail on October 15th, 2004 3:16 pm

    OK, just because it is difficult to measure manager performance, doesn’t mean we shouldn’t try.

    Take a theoretical average team. An average team should win 81 games and lose 81 games. (Not replacement level team, but just average.)

    What would cause this team to perform better or worse than our “average” baseline?

    1. Team composition. Who is on the team? Do you have a roster with good, talented players or a roster full of Willie Bloomquists? This is the GM’s job. (Money plays a big part but you can still put together a crappy team with a high payroll.)

    2. Team tactics. Of those players, who actually plays on the field on a given day? Is the manager smart about this? Note: we are not judging what tactics he uses, just the success of these tactics.

    Now, if you can take a record and isolate the team composition then you have a decent idea of the success of team tactics – the manager.

    Before you say “that’s just the pythagorean w-l record”, it’s not. That looks at runs scored and runs allowed. That already includes a certain measure of tactical success; the ability to concentrate runs in winning games.

    I’m more interested in how much more a manager can get out of his roster than could be expected. That requires not looking at the runs scored but each individual player.

    Example: take a theoretical Japanese relief pitcher – name him “Shiggy.” “Shiggy” is on the roster and the manager can’t control that. The manager has to use him because otherwise he’d effectively be one man short in the bullpen. Maybe the GM wants “Shiggy” to play in games so he can be traded. Whatever.

    Say “Shiggy” has an average number of pitched relief innings. Perhaps he does suck this year, giving up home runs and walks and otherwise stinking up the joint.

    The manager has no control over that and shouldn’t be judged on that pitchers performance. The manager can, however, control when and where he pitches. If “Shiggy” contributed to more losses than expected, that means the manager is doing a poor job – maybe putting him in when they have a small lead. If he contributed to less losses, perhaps the manager is being smart and putting him in losing games.

    Is there a way to do this?

  10. Jeremy on October 15th, 2004 3:18 pm

    I think it’s pretty easy to figure out what the Mariners want in a manager.

    1. They want a manager who puts the right people in at the right time the highest percent of the time.
    2. They want someone who can lead and motivate a group of highly competitive athletes.
    3. They want someone who is good with the media and the community.

    Good luck measuring that statistically.

  11. John Hawkins on October 15th, 2004 3:32 pm

    Dude, Chris W. has it right with his comparison to defense, but you’re not quite getting it. Compare managerial ability to defense fifteen years ago. Fifteen years ago, statheads were saying stupid things like “defense is maybe 5% of the game, at best.” Back then there were few and poor metrics to evaluate defense and it’s impact on the game. Today, there are beter metrics and an understanding that defense is an important part of the game.

    Just because you don’t know how to measure a manager’s contribution doesn’t mean it doesn’t matter. For crying out loud, think of all the criticisms you have for “tools” evaluations. The people doing “tools” don’t understand RC/27 or ERC so they discount that stuff as “stats are meaningless for high school kids.” Sabremetricians in the 80′s said the same thing about defense because they didn’t understand how to measure it.

    DMZ, are you saying the same thing about managerial metrics today?

  12. Adam S on October 15th, 2004 3:36 pm

    I agree with you that a manager doesn’t have much impact. I figure +/- 3 games over a season. But how much impact does any one PLAYER have?

    If the Mariners upgrade from “void” to Adrian Beltre at 3B that could be a huge (8-10 wins) gain, but that’s the extreme case. How much better is Carlos Beltran than Randy Winn? Isn’t that only 4 or 5 wins? Isn’t going from Bob Melvin to possibly worth the same 4 or 5 wins?

    Or am I way overvaluing how bad Melvin was and undervaluing how good Beltran could be??

  13. Paul Mocker on October 15th, 2004 3:49 pm

    Perhaps the concept of “Replacement Level Manager” should be studied. Answers to questions such as the following could yield actionable information: 1) What would we expect from a team with no manager, IOW, a player-manager? If Edgar and Jaime were player managers what record would the M’s have had? 2) What would be the characteristics in terms of personality of a replacement level manager? 3) What should the goals of a replacement level manager be?

    IMO, managers are way overrated IF you have enough talent to win and those players can discipline and motivate themselves. Would anyone argue that great teams such as the ’27 Yankees, ’75 Reds, the 70s Orioles of Weaver’s day would not have won world championships?

    The GM’s obvious first task is to find these types of players. Second, if he can’t find them, then a manager might be able to squeeze some wins from the team. Basically, a GM’s task must simply be to avoid managers who are in the margins, ie. the feisty sort who tick off players or the lenient sort who allow drugs in the clubhouse.

  14. chris w on October 15th, 2004 4:17 pm

    To keep the defense / manager analogy going… One thing you could do with managers is to calculate “errors”. An managerial error is a decision that, statistically, decreases a team’s chances of winning. You’d have to restrict yourself to statistically measurable managerial decisions, but the resulting data could be interesting.

  15. chris w on October 15th, 2004 4:20 pm

    Oops: a managerial error is a decision that, statistically, *should* decrease a team’s chances of winnning (e.g. bunting so that the chances of scoring a run actually decreas). You can only blame a manager for his decisions, not the actual outcome.

  16. MoxMox on October 15th, 2004 4:42 pm

    My gut feeling is that a manager has a significantly greater impact on a record than +/-3 games. The problem is, there is no way of knowing what “would” have happened if the manager had made a different choice than they did. Case in point, the infamous game in August when the Ms were down with 2 outs and Bloomquist at bat. Bucky sat on the bench while Willie struck out. Who knows what would have happened had Melvin decided to pinch hit in that spot? Maybe Bucky jacks one and we win, or maybe he strikes out too.

    A manager’s primary responsibility is to be well prepared – understand the skillset of his players and of the opposition, so he can make informed decisions.

  17. Paul Molitor Cocktail on October 15th, 2004 4:47 pm

    Oops: a managerial error is a decision that, statistically, *should* decrease a team’s chances of winnning (e.g. bunting so that the chances of scoring a run actually decreas). You can only blame a manager for his decisions, not the actual outcome.

    I think you have to be careful not to take positions on certain in-game strategies. It’s akin to saying a player is bad because he walks too much; you’re in the mode of thinking “a walk isn’t as good as a hit” rather than looking at the actual outcome.

    If a manager manages to bunt and sac-fly and base-steal his way to a few WS rings, who am I to judge?

  18. John Hawkins on October 15th, 2004 5:09 pm

    There may not be enough information for the in-game errors. For instance, the case where BoMel left Bucky on the bench and didn’t pinch hit for Bloomquist. What if Bucky’s knee was really killing him that day (maybe he tweaked it taking BP)? Managerial error or not?

  19. tyler on October 15th, 2004 6:00 pm

    HOLY $*(#!!!!

    this is an esoteric conversation, even for a statistally based site. A couple thoughts if i may:
    1) an interesting concept, mapping the quality of a manager… but one that statistics will never be able to effectively deduce, unless Cruise and Dreamworks do Minority Manager Report (which MLB may need to do anyway… heh heh).
    The future probability is something that simply can’t be measured. Never will, unless we figure a way to measure “God” as well.
    2) I would agree (and i would like to think that i have some authority here as a 2 sport coach) that a baseball game is far more difficult to impose my will upon than a basketball game. Evidence? How many baseball coaches can motive their players in the middle of a game to play harder by yelling and imploring them with histrionics? Watch basketball coaches change the pace and flow of a game… I think hoops is the most (in game) coach affective sport.

    What was that famous quote about pornography in the Supreme Court… something like, “I can’t define it, but I know what it is?”… well, same thing with good coaching/good managing. And BoMel in Seattle wasn’t it.

  20. Flavor Flav on October 15th, 2004 7:12 pm

    It depends on what the meaning of the word is, is.

  21. Paul Molitor Cocktail on October 15th, 2004 8:07 pm

    Even if you can determine that the effect of a manager is negligible, you’ve still proven something.

    I’m not sure that’s the case. Most of us would consider Bobby Cox a better than average manager. Same for Earl Weaver. If our beliefs are true, then there are better than average managers and we should be able to measure that.

  22. The Ancient Mariner on October 15th, 2004 8:15 pm

    I’m quite convinced of the importance of managers, but that doesn’t necessarily mean that I expect to be able to measure it. This is rather more akin to literary criticism than chemistry, as disciplines go.

  23. Dave on October 15th, 2004 10:06 pm

    If the Mariners upgrade from “void” to Adrian Beltre at 3B that could be a huge (8-10 wins) gain, but that’s the extreme case. How much better is Carlos Beltran than Randy Winn? Isn’t that only 4 or 5 wins? Isn’t going from Bob Melvin to possibly worth the same 4 or 5 wins?

    I don’t think you’re off base, here, Adam. I think the major difference is that we have all kinds of evidence that should lead us to be confident that Carlos Beltran will be worth 4-5 more wins than Randy Winn if he’s playing center field in Seattle next year. We don’t have any evidence of any kind, objective or subjective, to give us any kind of confidence in knowing that any new manager will have a similar effect.

    We aren’t trying to say “managers don’t matter, so hire a bad one.” We’re saying that no one has any idea how to tell the good ones from the bad ones ahead of time. If you’re just firing in the dark and hoping it works out, might as well do so at as little cost as possible.

  24. Aaron on October 15th, 2004 10:40 pm

    The thing is, we KNOW how much value any given player bring (or should bring) to the team. Home runs are good. A SLG over .500 is good. An OBP over .350 is good. All these things have either always been measured, or are part of statistics that have been, so they can be recalculated for past years.

    But how do you measure Manager stats? “Time Bad Pitcher Brought Into Close Situation?” Never objectively measured. PAP might be a start, but I don’t think one number can ever tell the whole story. Even if you look at expected performance vs. actual performance, there is still a lot of noise. Injuries, off the field problems, off years….all of these unmeasurable variables can influence performance.

    The biggest key to finding a good manager is just to find someone who won’t screw the team up.

  25. Chris Robertson on October 15th, 2004 11:43 pm

    Why not ask the Astros if a manager can make a big difference in the team’s won-loss record?

  26. big chef terry on October 16th, 2004 12:34 am

    comment on the margin…it seems to me that as a season moves along a manager will fall into patterns of behavior good and bad. Lots of small things, when he gets people up to throw, how often he does that, if he always brings the reliever in after the big hit or just before it…in each of the last two years the bullpen has looked like Vicksburg after Grant had his way with it.

    Playing Willie Bloomquist at first, somewhat regularly, batting Cabrera fifth.

    With many clubs the manager has a lot of sway about who’s actually on the roster at the end of spring training and throughout the year.

    Players will suggest that the manager has an important role..

  27. eponymous coward on October 16th, 2004 1:03 am

    We’re saying that no one has any idea how to tell the good ones from the bad ones ahead of time. If you’re just firing in the dark and hoping it works out, might as well do so at as little cost as possible.

    Exactly.

    So, of course, the M’s will have hired a retread by this time next week, according to Pocket Lint.

    http://seattletimes.nwsource.com/html/sports/2002064723_mari16.html

    Seattle club officials also interviewed Joe Maddon, Anaheim’s bench coach, but a source said they prefer someone with managerial experience in the major leagues. The Mariners, who are also believed to have met with Terry Collins, also are setting up meetings, or seeking permission to do so, with Don Baylor and Jimy Williams.

    The Mariners also are interested in Buddy Bell, Jerry Manuel, Lee Elia and Mike Hargrove. Club officials plan to select several candidates next week to meet with CEO Howard Lincoln and one or more team owners, most likely Chris Larson.

    On the subject of “can we tell who good managers are”- well, it would seem to me there may be some indicators that can be useful (W-L record, teams that exceed Pythagorean W-L projections), but yeah, those things are likely to be going to be massively masked by things like luck (which is something that’s going to affect W-L record even more dramatically) unless you have a long term record to work with, plus we don’t really know if they are related to managerial decisions (well, managerial W-L record is a better indicator- but look at Casey Stengel and Joe Torre- nobody thought they were special before they hit New York). Thus we’re left with “scouting”, subjective feel factors become more important in evaluation. (One thing I’ll chime in is that Jim Tracy may be a pretty good manager, with winning teams that regularly beat Pythagorean projections, plus good press).

  28. John on October 16th, 2004 8:12 pm

    All these comments, and not a single reference to Bill James book on managers.
    BTW, I don’t remember his trying to argue statistically, but he did say that JOE McCARTHY was probably the best manager ever. The only reason I can remember is that McCarthy got more out of players than their other managers had. (He cites HACK WILSON and VERN STEPHENS.)
    It seems to me that the various commentators have mentioned many characteristics that would go into a point system; to which I would add: how often, and to what degree a team exceeds the various consensi. (E.g.: if the consensi pick the team to win 60 games, and it wins 90.)[Of course, it's pretty hard to try to determine how much of a role the manager had in that.]

  29. The Ancient Mariner on October 17th, 2004 12:07 am

    Derek, just caught your comment in #8, and no, we aren’t making the same point. We’re making a number of the same detail points, but from that you’re concluding that “the fact that managers are so hard to statistically evaluate should help us to realize that they’re really not that important,” which is a non sequitur and, I think, a wrongheaded conclusion. The fact that managers are so hard to statistically evaluate should only help us to realize that they don’t have a direct effect on statistics, apart from playing time (significant enough; you don’t think replacing Lefebvre in 1988 with a manager who would have benched Presley for Edgar might not have made a noticeable difference?). That doesn’t mean their effect is “subtle” or “not that important,” merely that it’s second-order.

    Nor does the likely fact that managers, in terms of their overall quality and effect on particular teams, likely fall along a bell curve mean that they are “really not that important.” Much the same, after all, could be said of third basemen: “There are a few clearly good ones, there are a few clearly bad ones, but for the most part, the guys in the middle of the curve don’t hurt or help a team much at all.” Does this therefore mean that evaluating one’s third baseman is unimportant?

  30. Ignatius on October 17th, 2004 7:30 am

    All managers suck all the time. I am absolutely convinced that teams would win more games if the players managed themselves against teams that didn’t. Joe Torre tried to give the series away against the Twins but Ron Gardenhire made sure he couldn’t. Ken Macha threw roses on the walkway of a 3-9 finish to blow the division. The great Mike Scioscia forgot how and why the Angels win ballgames. Bobby Cox once again managed not to lose in the playoffs. Phil Garner was a joke in games 1 and 2 of this Cards series and he tried to give the division playoff to the Braves…several times. I won’t even comment on the Red Sox but check out who he bats 2nd.
    And these are the best of the best!

  31. John on October 17th, 2004 10:43 am

    Somewhere in here, you have to talk about the arrangement.
    In some arrangements–Oakland’s, for instance–the manager has almost no control. [BILLY BEANE lamented the fact that middle management had been making the big decisions, so he quickly changed that.]
    In others the manager tells the GM what to do. [Supposedly, LOU PINIELLA would tell WOODY WOODWARD what to get him, and Woodward would do so.]
    Which type of manager are we talking about?

  32. Mark on October 17th, 2004 12:17 pm

    I’m a little suspicious of the idea that the manager’s effect is negligible (3-4 games worth). Consider a couple of the things we know a manager does: he determines when his pitchers get the hook, and he decides what the lineup should be. Simply on the basis of developing young players, these two things have a HUGE effect on a team, not just this season but in future years. That is, bad management of pitchers, and the consequent destruction of young arms, costs future victories. Bad management of players, and the failure to develop home-grown talent, also costs the team down the road. Logically it would follow that doing a good job in these two departments would manifest a net gain in wins, both this eyar and in years to come.

    Bottom line: I’m one more voice in the chorus. Just because it’s hard to measure managerial success doesn’t mean that managers have negligible effects on a team. And in those cases where we do know what makes for a bad idea, managers who embrace those ideas (like Don Baylor) should be avoided at all costs. They’ll cost us now, and they’ll keep on costing us even after they’ve been fired.

  33. eponymous coward on October 17th, 2004 2:11 pm

    Bad management of players, and the failure to develop home-grown talent, also costs the team down the road.

    Isn’t that more a function of the front office? Just for an example- outside of Raul and a half-season of Scott Podsednik, who in the Mariner farm system has gone to do anything of note that Piniella “gave up on”? The Mariner farm system’s been crappy because of mismanagement in the front office, not the dugout.

    My problem is that nobody thought Casey Stengel and Joe Torre were geniuses before they showed up in pinstripes, and there are other examples of managers who were given talent and shined (Charlie Dressen)- and suddenly became unable to do much when taken away from that talent. Why didn’t their success follow them everywhere they went, if managers are so critical a component of team success?

  34. Michael on October 17th, 2004 8:19 pm

    There is an article in today’s NY Times looking at this and the relationship between Pythag and games over or below for different managers. It doesn’t add much to the discussion here, except ot effectively back up Bill James’ stuff and then to use other stats to look into post season mangerial effectiveness.

  35. Chris W. on October 17th, 2004 8:46 pm

    Maybe the answer isn’t to find a manager. Maybe a smart organization would create a manager… find a smart guy with good people skills and tell him you’ll hire him if he’ll adhere to certain principles, passed down from above. Most of the things that bug us the most about Melvin, for example, are not unalterable.

    I guess what I’m saying is this: players are limited by their genetic makeup, but managers aren’t. If a team hires a manager with a known predilection for doing stupid things and an unwillingness to change, that team deserves to fail.

  36. The Ancient Mariner on October 17th, 2004 10:18 pm

    Re #30: The key word here is “component.” Managers affect the success of their teams by managing the talent they’ve been given (and, for some, by managing to convince their bosses to get them the talent they want). No manager in the world can win consistently with an untalented team. That’s possible to an extent in basketball and football (just look at the Patriots–middle of the NFL pack in talent, but great coaching, smarts and execution have made them a juggernaut), but not in baseball. However, that bad team in the hands of an Earl Weaver has better odds, and likely a brighter future, than it would in the hands of someone like Don Baylor.

  37. eponymous coward on October 17th, 2004 11:02 pm

    Well, that’s fair…except the argument was that managers are worth MORE than 3-4 games a year, if you go back to 29. I would think if it were so, we could figure this out easier. 3-4 games a year is a pretty large component, equal to the difference between a very good ballplayer and an average one at a position, no?

  38. The Ancient Mariner on October 18th, 2004 1:17 pm

    On my best judgement, they’re worth more than 3-4 games a year, but I doubt anyone short of Earl Weaver, Joe McGraw and Joe McCarthy would be worth more than 10 wins a year most of the time. I suspect that there are a fair number of managers who have been able to give their teams that sort of boost for a season or two, but not to sustain it–Billy Martin comes to mind–and that the thing which marks truly great managers is their ability to consistently improve their teams. That is, however, just my intuitive sense–I couldn’t prove it if you paid me.