What Do We Know About Catcher Defense?
From Dave Cameron to Larry Stone, people have seen Kelly Shoppach’s signing with the M’s as something of a formality. He’s a veteran, he’s played for Eric Wedge before, he’s got a solid defensive reputation. Meanwhile, the normally tight-lipped M’s front office has made it clear that John Jaso’s defensive problems limited his value both to the M’s and potential trading partners. So, how do we know that catcher A is good and that catcher B is terrible? Perhaps no other area of analysis has changed so much in the past five years, so it may be time to look at what we think we know about measuring a catcher’s impact. For those looking for an air-tight conclusion, or a single number we can confidently slap onto Ronny Paulino, Shoppach or Jaso, this piece may disappoint. Those of you who are used to my epistemological musings will understand that we can’t yet do that, but yet it’s still interesting to think through the process.
So: what does a catcher do? What’s in the job description? They’re there to call pitches (sometimes), catch/block wayward pitches, to present somewhat less wayward pitches in the best possible light so that they may be called strikes, throw out runners attempting to steal bases, and function as psychologist, coach and motivational speaker to pitchers. Measuring passed balls/wild pitches is easy, though comparing it to an appropriate baseline is. Same with stolen bases – adding up the runs saved/lost in the running game is easy, but apportioning those runs to catchers and pitchers can be a bit trickier. Still, sabermetrics is on pretty firm ground in assigning values to these two components of catching; Fangraphs’ catcher defense metric is the sum of these two measures. However, the range of this metric simply isn’t all that big. The best/worst catchers range from about -5 to +5 runs in each one, so the gap between the best and worst catchers would realistically be in the neighborhood of +1 to -1 wins per year. That’s…not a lot, and several years ago, the idea that teams were too willing to move “questionable” defenders off the position were popular in some sabermetric circles. The squishy stuff like pitch framing and “managing a pitching staff” seemed near impossible to measure, and no one was certain if it was really true, or if it was just a baseball equivalent of an old wive’s tale.
The first real attempt at trying to measure the impact of the psychologist-cum-strategist role of the catcher is Catcher ERA, or cERA. In its simplest form, it just the earned runs scored while a catcher was playing – crediting the pitcher’s ERAs to the catcher. The problems with this are legion, and we’ve dealt with this issue several times at USSM. As Dave said back when people were clamoring for more Rob Johnson (this really happened! There was clamoring!), the fatal flaw with the measure is selection bias. John Jaso looks great by cERA in 2012 because he caught a lot of Felix’s starts.* Craig Wright updated it years ago in his book “The Diamond Appraised” by focusing solely on pitcher/catcher pairs. That is, he took selection bias out by controlling for the pitcher. Still, methodological problems remain – two catchers could have identical ERAs with a set of hurlers and come out with vastly different cERAs depending on how frequently they caught each of them.
The combination of small samples, a lack of evidence for a real effect, potential biases and the lack of an identifiable *reason* for catchers to have vastly different results led many in sabermetrics to dismiss cERA, leaving just the running game and pitch blocking to measure. Two things fundamentally changed that. First, Dan Turkenkopf and others tried to assign a value, in runs, of a pitch being called either a ball or a strike. An fastball on the black is ruled a ball by a vindictive umpire – what did that cost the pitcher and his team? The answer he got (0.161 runs) was stunningly high – if catchers had a repeatable skill in getting on-the-edge pitches called strikes, the impact of it could dwarf the value of CS% or wild pitches. Initial studies of the impact of pitch framing saw massive, almost impossible gaps between the best and the worst catchers, until Mike Fast’s definitive study on the matter put the range at about +/- two wins per year. For some catchers, this skill does in fact dwarf the impact of CS% and passed pitches. The measure is so new, and researchers are still tweaking it to ensure that they control for everything from the umpire, the park’s pitch fx calibration, the hitter, and the pitch type. This gap remains, and, more excitingly, this appears to be an actual skill – good pitch-framers in year X are generally good pitch-framers in year Y. Maybe the old baseball saw about catcher defense wasn’t just precedent mixed with hokum.
Now that pitch fx has been in use for 5 full seasons, and now that researchers can correct for the park-by-park idiosyncrasies in the data, sabermetrics is again looking at something like cERA to see if there’s any other effect a catcher might have on a pitcher’s performance. In reality, something like a new-fangled cERA (or what Tom Tango always called a WOWY -with or without you- study) is more about capturing everything than it is about isolating a particular skill. Still, might framing and, say, pitch blocking taken together exert a bigger impact on a pitcher’s confidence? Might a catcher’s pitch calling or the way he sets a target have an impact? Controlling for pitch fx, pitcher, batter and just about everything else, Max Marchi tried to measure the runs saved or lost by each catcher.
All of this suggests that teams should be focused on catcher defense. Indeed, the Astros snapped up Mike Fast not long after his article on pitch framing appeared (and another just hired Dan Turkenkopf), so we know that teams *are* paying attention to developments in this field. Marchi’s work on handling a staff and Fast’s work on pitch framing are so new, they’re not incorporated into WAR measures.** This may lead to WAR understating the importance of catcher defense, but I think it’s reasonable for sites to wait a few years to see if anyone’s able to shoot holes in these studies. There’s plenty of work to be done teasing out what it is a catcher does to get a pitcher more comfortable, and, perhaps more importantly, why a catcher will work quite well with one pitcher (or kind of pitcher) and not at all well with another.
There’s also the problem that catchers who are great at one aspect of the job are often poor at others. To bring this back to the Mariners’ catching depth (finally!), we see that Ronny Paulino isn’t great at blocking pitches or controlling the running game, but he’s above average in framing. Miguel Olivo was (surprise!) one of the worst in baseball in blocking pitches, but one of the best (at least from 2008-2011) at handling a staff. John Jaso rates poor in blocking and not-so-hot in pitch framing, but is a touch above average in pitcher results/handling a staff. Kelly Shoppach was a bit above average in framing and in the running game, but a bit below in pitch blocking and hide-your-eyes-bad in pitcher results. Jesus Montero’s not been catching long enough to show up on a lot of these lists, but his results in 2012 weren’t great. He was noticeably worse than Jaso/Olivo with many pitchers last year, and posted the worst K:BB ratio and OPS-against of the three backstops. We’ve got a heck of a lot more data, and even some new theories about how to use it, but I’m not sure we can definitively say that John Jaso was terrible*** or that Kelly Shoppach/Ronny Paulino/anyone else is really good. So help me out: what did YOU think of the M’s catching defense in 2012, and how do you think it’ll change in 2013? What aspects of the catcher’s role is sabermetrics ignoring, even now? What do YOU think the M’s care about in assessing a catcher’s performance, and if you had your way, what WOULD they care about?
* – It’s sort of interesting how catching a bunch of Felix’s starts doesn’t count as a credit, though. I mean, why would Felix keep throwing to him – and so well – if he Jaso was god-awful?
** – Baseball-Reference’s WAR measure for catchers – Defensive Runs Saved – apparently incorporates a cERA-like measure, though it’s tough to tell.
*** – John Jaso’s cERA highlights something that may be the next frontier in researching catcher defense. Over his career, he’s been pretty good compared to other catchers he’s played with (like Kelly Shoppach) in things like K:BB ratio and plain old BB%. Where he’s had a problem is giving up long balls. With the M’s in 2012, Jaso yielded runs and walks at the lowest rate of the three M’s catchers, but saw a lot more HRs per PA against him – particularly with Blake Beavan and Jason Vargas. In Tampa, Jaso had solid RA numbers and decent BB% rates, but was beset by HR problems, particularly when paired with fly-ball pitchers like James Shields and Matt Garza. But just as he had good results – better than Olivo/Montero’s results- catching Felix, Jaso seemed to work well with David Price. I initially thought the issue may be the result of issues with a specific pitch: the change-up. But most of the HRs came on fastballs, and Blake Beavan barely even throws a change-up. Still, it’d be interesting to see if there are specific characteristics of catchers that can lead to problems or successes in things like homers, double plays, walks, or strikeouts. Is Jaso’s low BB% and high HR% the product of a catcher who doesn’t call enough breaking pitches out of the zone? This would be an interesting topic for further research.
Comments
16 Responses to “What Do We Know About Catcher Defense?”
Leave a Reply
You must be logged in to post a comment.
I’ve been wondering the same thing and appreciate this synthesis.
What bothers me the most is when managers obviously rate their catchers but don’t/won’t/can’t give specific examples of what leads to their opinion. Everything seems to be clouded in baseball mysticism. I can imagine a conversation between Wedge and a coach about Jaso:
Wedge: Do you see what I see?
Coach: yup.
Wedge: I don’t see how we can play him over Olivo.
Coach: yep.
Wedge: Certainly not consecutive games; see how his handling of pitchers deteriorates?
Coach: I do.
Wedge: It’s settled. If you see the same things I’m seeing, we’ve got to limit Jaso, no matter how well he hits.
Coach: It’s pretty obvious!
It seems like a difficult statistic to gauge– because having no effect on pitchers’ performances is essentially measured like having a negative effect, yeah?
Meaning, non elite flyball pitchers gave up a lot of HR’s with Jaso pitching per your info… Is that because he’s bad at catching, or because they didn’t abnormally benefit from him being behind the plate so– as would be expected– non elite flyball pitchers gave up lots of homeruns.
I mean, I don’t see how a bad catcher would cause more homeruns, unless a pitcher was afraid to shake him off and he called mid-zone fastballs all game, or if he put his mitt up in the zone, in the fat of the plate, as a target for pitches, and the pitcher refused to throw it anywhere else for some strange reason?
There are just SO MANY variables to take into account that the work would seem to outweigh the results (if they really are a 1-2 run swing, either plus or minus)…
Take:
CS%
highly dependent upon the pitcher’s speed to the plate, and the speed of whomever gets on base, how many guys get on base, and the score of the game, and how many outs, the count against the batter, etc., etc. Virtually impossible to determine a control that would equalize every catcher’s results.
Framing:
Umpire to umpire, pitcher to pitcher, the score of the game (umps undoubtedly call a looser zone when it’s 12-0 and the loser’s are pitching in the 8th/9th inning), how many outs, the hitters being faced (a veteran’s strike zone, versus a rookie’s), home or away games (guessing here), etc.
Wild pitches and passed balls are even more arbitrary than errors and hits in most instances for scorers (and 1 or 2 against you can be 5-15% of your year’s total). The talent and arsenal of the guy pitching can have a huge effect on that.
If you’re dealing with a young staff, probably harder to get good results simply because the umps, different from start to start, don’t know the guy’s pitches yet, or how they break, or what to look for in movement, etc. If major league hitters have trouble picking up a rookie’s fastball, imagine the difficulty of being a 70 year old umpire trying to make sense of it.
Basically it’s exponentially more difficult to get a grip on catching stats because at a minimum, four people are involved…
A pitcher, a batter, a catcher, and an umpire.
And each guys’ performance affects the next’s.
If you’re playing the M’s in 2012 you’re 3 wild pitches to Carlos Peguero might be a beautiful 3 pitch strikeout.
If your pitcher is in a feud with the ump, his 3 pitch K of Brendan Ryan might end up a 3-0 hitter’s count instead– with all 3 pitches framed perfectly on the black.
Or if you’re catching R.A. Dickey, you might get a passed ball on a called strike 3 (which was actually strike 5, but the ump was having difficulty), and the ball goes to the backstop where you end up throwing out the guy on 2nd trying to advance to 3rd on your missed catch, while the batter who struck out is safe at first, gets pinch run for by Michael Bourn (who happened to have the day off), who steals 2nd when your SS misses the tag, takes 3rd on a knuckler wild pitch, and then scores the game winning run on a softly hit roller to the SS.
Just not easy to score how a catcher is doing, beyond passed balls/wild pitches and his caught stealing %.
Good (and interesting) article Marc. I’m curious to see how all this develops.
Wow. Awesome article. I remember reading the Mike Fast article but the Marchi article is the one I missed. I continually said that while Olivo was bad, he is great at handling the staff.
One aspect that is extremely important for a catcher is preparation and learning the tendencies of players. This is one possible reason why younger catchers can have more difficulty because they don’t know the tendencies of most batters in certain situations. I remember a college coach telling our guy that a good memory and preperation serves a catcher just as much as a good arm.
I’d be curious to see how all of these factors come together when analyzing Paulino and Shoppach with say Jarrod Saltalamacchia. On pure defense, he looks to barely surpass both of them (and certainly on offense), but these items could tell a different story. Oh well it’s not like Boston has too many catchers or a need for something that we have a surplus of.
Another factor to consider is: are the batter’s hot zones being taken into account by the catcher? Also, is the pitcher hitting the target? There were times last season where it did not seem that the catcher bothered to go over the batters before the game. Maybe the fault was the pitcher missing the location.
Overall, I thought the Mariners catcher’s defense was below average. The pitchers should have the confidence in the catcher to use whatever pitch he feels he needs in any situation. None of the catchers were good at stopping errant pitches. How many times were pitches left up in the zone because the pitcher was afraid of having a passed ball?
But you can’t measure the significant chemistry Jaso had with the fans (or just me). He was about as likeable as I’ve been able to find on the team for past 10 years. I absolutely hate that he now has to wear white shoes every day.
Montero’s passed ball/wild pitch rate was lower than the MLB catcher average last year.
If the coach doesn’t like how his catcher is calling the game, have someone else call it from the bench. It’s not a good reason to take a guy out of the lineup.
Maybe Jaso was having conferences up on the mound where he would say “You know, Blake, what you should do is throw that absolutely mediocre fastball over the plate at a solid 90 miles per hour. They’ll never expect it.”
And maybe then he gave those cute little puppy dog eyes, and so we got the result we did. That beard, and those eyes… They’re so convincing.
If I had my way the Mariners would care about implementing automated ball/strike calls so that pitch framing would no longer be an issue.
Great article, Marc. I don’t know if I can really give an opinion on some of your questions/requested input, but I will say the M’s should care about a more balanced catcher. Jaso may have been below average defensively, but he was the best option we had, and he wasn’t an embarrassment. That’s more than can be said of Olivo and Montero. Hopefully Montero will show some improvement (a lot!) early, so Zunino isn’t rushed. Real curious who else is will be brought in before the season starts.
Kelly Shoppach… Just announced on MLBTradeRumors, via ESPN…
1.5Million plus 500K in incentives.
So, I posted that last comment before I heard about the Shoppach signing. Not a huge surprise.
Quick question about the use of pitch fx that I should already know, but I don’t. Are the actual locations figured in somehow, or only the results as called?
There are cameras in the stadium that are used to determine where the ball crosses the (front?) plane of the plate. The ad they used to show during Mariners games stated there were three cameras dedicated to this.