There’s No Virtue in Signaling. But Is There Any Benefit?

On November 12, when The Athletic’s Ken Rosenthal and Evan Drellich published the report that blew the lid off of the 2017 Astros’ sign-stealing scheme, they included key details about the origins of the conspiracy. The sign-stealing, they wrote, had started “early in the 2017 season,” instigated by “a hitter who was struggling at the plate and had benefited from sign stealing with a previous team” and “a coach who wanted to help.” The duo had devised a hybrid, high/low-tech system to tip off hitters about what was coming. A camera beyond the outfield fence at Minute Maid Park would capture the opposing catcher’s signs and relay them in real time to a monitor in the tunnel between the Astros’ dugout and clubhouse. There, team employees and players would decode the signs and bang on a trash can to tell the hitter what to expect—“one if by land, two if by sea,” except with thumps instead of lanterns and breaking balls instead of British troops.

What the Astros hitters heard then, we can hear now. When my Effectively Wild podcast cohost, ESPN’s Sam Miller, reviewed video of 2017 Astros home games prior to our discussion on November 14, he found that the banging had begun sometime during the home stand that ended on May 28. At the start of that home stand, on May 19, Miller discovered, no banging could be heard, but by the 28th, it was audible, starting with an 0-1 pitch to Houston’s leadoff hitter, George Springer. That jibes with the timeline Rosenthal and Drellich described.

On May 28, very few Houston hitters were obviously struggling. One hitter who was off to a slow start, and who had played for plenty of previous teams, was Carlos Beltrán. Sure enough, in a subsequent report, Rosenthal and Drellich reported that the ringleaders were Beltrán and bench coach Alex Cora, now the managers of the Mets and Red Sox, respectively. On the morning of May 28, Beltrán was batting .235/.277/.389 (78 wRC+). From that day on, armed with powerful knowledge that theoretically robbed opposing pitchers of much of the uncertainty they depend upon to get hitters out, Beltrán batted … almost exactly the same: 230/.286/.380 (76 wRC+). Huh.

Beltrán was 40 years old, so maybe his bat was simply too slow to take advantage of what he’d learned. As a team, the Astros hit .268/.335/.447 (111 wRC+) through May 27, and .288/.350/.491 (126 wRC+) afterward, a 15 wRC+ point improvement. There’s the sign-stealing effect!

Well, maybe. Except that in-season variation of that magnitude isn’t at all unusual. This year alone, six teams batted at least 15 percent better relative to the league after May 27 than they had before—and this season’s schedule started earlier than 2017’s, which made that more difficult to do. What’s more, most of the Astros’ improvement after May 27, 2017, came on the road. In Houston, where they had the home-trash-can advantage, their wRC+ after May 27 improved by only five points, from 121 to 126.

Before we go on, let’s make some stipulations. The evidence overwhelmingly indicates that the Astros cheated via electronic sign-stealing in 2017. We have the testimony of former Astro Mike Fiers, backed up by anonymous sources cited and quoted by Rosenthal, Drellich, and other reporters. We have our ears and our eyes, which can clue us in to many examples of the Astros’ system at work, whether via video or through the telltale sound spikes in the game audio tracks that denote the thumping in action. We have our knowledge of MLB’s actions, which include putting stricter anti-sign-stealing measures in place during the 2018 postseason, reportedly warning official observers to listen for banging at Minute Maid Park in particular in 2019, and launching an official inquiry in response to recent reports. We have the revelation that Kevin Goldstein, an Astros special assistant to GM Jeff Luhnow, sent an email in August 2017 in which he encouraged scouts to use cameras, if necessary, to steal signs sent from the dugout—a separate scheme from the Beltrán-Cora conspiracy, and potentially a less egregious one if the cameras weren’t relaying information midgame, but still an indication that the Astros were willing to flout MLB taboos. (“Go find a major league rulebook and find where sign-stealing—or signs at all—are covered,” Goldstein wrote in 2012. “They’re not.”) We’ve even gotten a glimpse at what seems to have been the scene of the trash-can crime.

In other words, we don’t need stats to prove that the Astros stole signs. Nor does it matter, morally speaking, whether the Astros’ sign-stealing worked. If the Astros disobeyed MLB’s ban on electronic sign-stealing because they thought it would give them an advantage, it was wrong whether they gained that advantage or not. Regardless of what they were thinking, they repeatedly acted in a way that was forbidden by MLB, and they should be punished accordingly.

But that needn’t stop us from satisfying our curiosity. What we have here, thanks to skilled reporting, archived video, and granular stats, is a rare opportunity to assess something fundamental about baseball: the importance of knowing which pitch is coming (and by extension, the importance of preventing hitters from finding that out). We shouldn’t waste this window into the sport’s inner workings. So, how well did the Astros’ sign-stealing work?

We can try to solve this mystery from a few different directions. As many people have pointed out, the Astros slashed their strikeout rate dramatically between 2016 and 2017. Astros nonpitchers struck out in 23.4 percent of their plate appearances in 2016, the league’s fourth-most-frequent rate. In 2017, the league-wide nonpitcher strikeout rate rose by 0.6 percentage points, but the Astros’ K-rate dropped precipitously to 17.2 percent, the lowest in the league. That’s one of the largest year-to-year reductions on record. One would imagine that knowing which pitch to expect would help hitters make increased contact, so this seems like a statistical smoking gun.

Here’s the problem with that persuasive story: The 2016 and 2017 Astros were different teams, with different hitters and different distributions of playing time. And the 2017 team seemed designed to make more contact. “When we looked at players we wanted to add … we wanted players who could do damage with a lot more contact hitting,” Luhnow said in August 2017. OK, that could have been a cover story, a narrative manufactured after the fact to provide a plausible explanation for the Astros’ sign-stealing-aided improvement. But the Astros’ contact-rate decrease was mostly foreseeable before that season started, without any knowledge of the sign-stealing to come.

On January 31, 2017, Jeff Sullivan wrote a blog post for FanGraphs entitled, “The Astros Have a Completely New Look.” Sullivan noted that the Steamer projection system foresaw the formerly strikeout-prone squad posting the biggest strikeout-rate decrease since 1950. Eleven players had made at least 250 plate appearances for the Astros in 2016, and 11 were projected to make at least that many in 2017. The five holdovers in that group were projected to strike out roughly as often as they had the season before. But the six new players were much more contact-oriented than the six they’d replaced. As Sullivan explained:

Last year’s six struck out a combined 27% of the time. This year’s replacement six are projected to strike out just 16% of the time. Jason Castro is gone, replaced by Brian McCann. Colby Rasmus and Carlos Gómez are gone, replaced by Josh Reddick and Nori Aoki. Carlos Beltrán isn’t particularly strikeout-prone, and he’ll bat most days. Alex Bregman has an excellent minor-league record of contact. We know the least about Yulieski Gurriel, but in last year’s cup of coffee, he whiffed just a dozen times in 137 trips.

Per Sullivan, Steamer’s projections, paired with FanGraphs’ playing-time estimates, yielded a projected nonpitcher team strikeout rate of 17.7 percent, only half a percentage point higher than the actual full-season figure. Steamer knew nothing about future sign-stealing; it only knew how those hitters had performed in the past. Yet months before Opening Day, and even more months before the sign-stealing seemingly started, the Astros already looked like a great contact team.

A couple of caveats here: According to Steamer proprietor Jared Cross, those preseason projections were calibrated to an MLB baseline nonpitcher strikeout rate of 20.1 percent, which is lower than what the leaguewide rate turned out to be. What’s more, Steamer couldn’t perfectly predict how the team’s plate appearances would be distributed before the season started. If we recalculate based on the real league-wide strikeout rate and the Astros’ actual playing time, then Steamer would have expected Astros nonpitchers to strike out 19.2 percent of the time, 2 percentage points higher than where they wound up. But that’s still a lot closer to the Astros’ 2017 strikeout rate than their 2016 figure. The Astros’ plummeting team strikeout rate was mostly a product of changes in personnel, not changes in sign-stealing.

How about home/road splits? To the best of our knowledge, the Astros had help from the trash can only at home, so we would expect them to have hit especially well in Houston. As one would expect, MLB nonpitcher batters on the whole hit better at home, by 10 points of wRC+ and 14 points of wOBA. The Astros hit slightly worse at home, both over the full season and after May 27. They hit the ball hard more often, and whiffed and chased pitches outside the strike zone less often, on the road.

Granted, Minute Maid has mysteriously morphed into a pitcher’s park, and even in neutral parks, teams sometimes hit worse at home purely as a result of random variation. It takes 67 games for an MLB team’s record to reflect half skill and half luck, so there’s still a lot of noise in an 81-game sample. It’s possible that through chance alone, the Astros would have hit even worse at home, relative to their road performance, without their sign-stealing. But these splits aren’t a strong signal that the Astros did benefit from their special sauce at home. For what it’s worth, Astros base runners don’t seem to have used their foreknowledge of signs to pick ideal spots to steal, either; Astros runners at home had a 71.4 percent stolen-base success rate, below the 74.2 success rate for non-Astros home teams. (After May 27, the Astros were slightly more successful than the league on steal attempts at home, but only by 1.2 percentage points.)

One might object that we don’t know precisely how widespread the Astros’ sign-stealing operation at Minute Maid was; if the sign-stealing was sporadic, its impact might be difficult to detect. (The 2017 Astros won the AL West by 21 games, so it’s not as if every pitch was do-or die.) However, we know it was happening for most of the regular season. In Miller’s noncomprehensive video review, he heard banging with 10 different Houston hitters at the plate, including almost all of the ones with significant playing time. And it wasn’t just once in a while; in the September game that Rob Arthur digitized to search for sign-stealing’s sound signature, he says, “the vast majority” of breaking pitches were accompanied by bangs.

Even so, let’s entertain the possibility that the Astros picked their spots and only paid attention to the banging at the moments that mattered the most. The table below displays the Astros’ offensive performance at home in medium- and low-leverage plate appearances to their performance at home in high-leverage plate appearances, both over the entire 2017 regular season and only after May 27. Pitcher hitters are excluded, and MLB baselines are provided for comparative purposes.

The league as a whole hit worse at home in high leverage, when better pitchers tend to be on the mound. But the Astros suffered a steeper drop-off than the typical team. If anything, the Astros underperformed at home in high leverage, garbage-can bangs be damned.

During their 2017 World Series run, though, Astros hitters shone at home. Rosenthal and Drellich’s sources conflicted on whether the Astros were still illegally stealing signs in the postseason, and it’s hard to hear any bangs on the playoff broadcasts. (It may also have been hard to hear them in the packed park.) Although they may have been spooked by the prospect of increased scrutiny or the fine the Red Sox were assessed on September 15 for their own, Apple Watch–based sign-stealing solution, it seems unlikely that the Astros wouldn’t have tried to preserve what they believed to be an advantage at the most important time of the season. They may have simply switched to a system that didn’t depend on sound.

Whatever they were up to, the Astros did hit better at home in the playoffs, to the tune of 230 points of OPS. Of the 111 teams with at least 10 games played in a single postseason, the 2017 Astros had the sixth-largest differential (in either direction) between home and road OPS. So yes, that was unusual, but not unheard of or outside the bounds of what could have occurred naturally. The Astros didn’t even boast the biggest split in the 2017 postseason; their ALCS opponent, the Yankees, surpassed them by more than 50 points (albeit in fewer games). Even for the Astros, who played 18 postseason games, we’re talking about only a little more than 300 plate appearances on either side of the split, so we’re squarely in small-sample territory.

Let’s look at this one more way. Until now, we’ve been acting under the assumption that the Astros were only sign-stealing at home. Let’s not underestimate their unscrupulousness. The Astros had a .593 winning percentage at home in 2017, and a .654 winning percentage on the road. In fact, the Astros are the only team with a better combined road record than home record from 2017-19. Maybe that means sign-stealing didn’t help them, or maybe it means that they cheated wherever they went. It’s harder to steal signs via cameras and monitors on the road, because visiting teams have less control over conditions. And if they did relay signs on the road via nonpercussion methods, Fiers either wasn’t aware of it or wasn’t whistleblowing all the way. But we shouldn’t put it past them.

As Sullivan and Mike Petriello wrote in January 2017, the Astros projected to have the deepest, and potentially the best, lineup in baseball without any adjustment for illicit sign-stealing. But they finished the season with the best lineup by far. Applying preseason Steamer projections to the actual 2017 playing time of Astros non-pitcher hitters yields a projected team wRC+ of 109. Instead, they came in at 122. Of the 11 Astros hitters with at least 250 plate appearances, only Beltrán (by a wide margin) and Evan Gattis (by a slim margin) underperformed their preseason offensive projections. This is the most compelling statistical corroboration we’ve come across so far.

Now, that’s not conclusive. Teams often outperform their projections, and the Astros have also been at the forefront of using training and technology to enhance player performance in legal ways. Plus, if the Astros weren’t cheating on the road in 2017—at least not as often or as effectively as they were at home—then the fact that they hit so well away from home would argue against the idea that sign-stealing gave a big boost to their bats. If they were cheating everywhere, though, then their overall overperformance, relative to their preseason projections, would be consistent with the intuitive assumption that sign-stealing helps.

This may seem like a lot of words to devote to searching for statistical evidence of something that sounds so obvious. Knowing the next pitch just has to help, right? But no matter how we slice and dice the data, the statistical case is less compelling than it would be if sign-stealing made hitting as simple as it seems like it should. Great as the Astros were at the plate in 2017, the most fascinating aspect of their sign-stealing scandal is that it didn’t make them even better. When we watch an Astros hitter homer after getting tipped off, it’s easy to assume that he homered because he had inside info.

Then again, that could be a textbook case of the post hoc fallacy. Hitters crush plenty of pitches without advance warning. There’s some chance that the Astros incurred the scorn of the sport, forever tainted their title, and opened themselves up to severe penalties—likely fines, loss of draft picks and international bonus pool money, suspensions, or even a John Coppolella–esque ban—for little to no return.

Many players must see value in sign-stealing, or it wouldn’t keep occurring, but others forswear it for performance-related reasons. Bobby Thomson (of the famously sign-stealing 1951 Giants) said in 1968, “I never wanted to know what pitch was coming. I was so overeager, if I’d known a fastball was coming I’d likely have swung too soon and missed it.” Many Hall of Famers, including Joe DiMaggio, Yogi Berra, Hank Aaron, Eddie Mathews, Tony Gwynn, Cal Ripken Jr., and Reggie Jackson, have insisted that they never wanted anyone to tell them a sign, but lesser players might have more to lose. Eddie Robinson, first baseman for the 1948 Indians—another group of World Series–winning sign-stealers—didn’t ask for signs because Rogers Hornsby had advised him against it. “Of course, Hornsby was so good he could just react to the pitch,” Robinson later wrote. “I probably shouldn’t have followed his advice because I wasn’t as good a hitter and needed all the help I could get.”

Great hitters may not think they need the help, and players who are guilty of frowned-upon sign-stealing may be motivated to minimize their sins. But lines like Thomson’s have been such a frequent refrain through baseball history that one starts to suspect something more at the root of some players’ professed reluctance.

Major league hitters don’t have superhuman reflexes. What they have is learned perceptual skills, honed through picking up patterns over thousands and thousands of pitches. Simply telling them which pitch is coming, instead of making their brains work for it, sounds like it would simply allow them to skip a step and be even better. But disrupting their regular process might make them worse.

Thad Meeks, an associate professor of cognitive psychology at Southern Illinois University Edwardsville, says, “If a behavior is well-learned, to the point that it is almost automatic, it is very possible that imposing additional thought processes into working memory may interfere with those behaviors. … Thus, it is possible that batters may have more automatized reactions to pitches without knowing what is coming over decades of learning. And it is certainly possible for some that overriding that automatic response with a different approach, even if that approach is on the surface advantageous, may interfere with [their] natural approach.”

Maybe there’s a mental benefit to thinking through an at-bat rather than taking the passive approach. Maybe it’s distracting to listen to loud banging in the seconds prior to a pitch. Hitting is hard even with an answer key; last week at MLB Network, I eavesdropped on John Smoltz as he recounted a time when his team had decoded an opposing pitcher’s signs but couldn’t hit him anyway. And because hitters aren’t used to knowing what’s coming, they may have a hard time talking themselves into trusting the intel, just as runners on first base couldn’t bring themselves to stroll to second against Jon Lester even though the scouting report said he’d never throw over.

All it would take to plant a seed of doubt is a single errant thump. The Astros’ sign-stealing was accurate, but not flawless. On September 19, 2017, two thumps told Carlos Correa to expect a changeup, but Lucas Giolito threw him a fastball instead. Maybe the person on the other end of the trash can made a mistake, or maybe the White Sox—who were aware of the banging earlier in the series—were trying to turn the Astros’ system against them.

Again, whether the Astros profited from their infractions is immaterial to the disciplinary decisions Rob Manfred must make. MLB should punish the Astros because they defied the directives of the league. However, it’s also somewhat circular to say that electronic sign-stealing is wrong because MLB told teams not to do it. Electronic sign-stealing isn’t illegal because the Baseball Gods decreed it. We have rules for a reason: to prevent an undesirable behavior. Electronic sign-stealing is banned because of the perception that it provides an unfair edge.

No one would claim that improper sign-stealing never helped Houston. But if it doesn’t help as much as is widely believed; or if enough teams are doing it that it doesn’t upset the competitive equilibrium; or if teams have grown adept enough at counterintelligence to defuse the effects without slowing the game down dramatically; or if we’re on the verge of switching to headsets or haptic feedback devices that could convey signs to the pitcher, catcher, and other defenders without a visual reveal; then this might be another incarnation of a never-ending dispute, not a new, existential crisis for the sport.

In his book about signs and sign-stealing, The Hidden Language of Baseball, Paul Dickson wrote that “the lore and history of baseball are so intertwined with signs and sign-stealing that it would seem all but impossible to stop it.” It’s easy to see how prohibited sign-stealing could be contagious. Rosenthal and Drellich implied that Beltrán may have adapted the Astros’ method from a previous club. Their report also suggested that some Astros saw their system as a form of self-defense against perceived sign-stealing opponents. The Dodgers suspected the Astros of electronic sign-stealing in 2017, and then the Brewers suspected the Dodgers in 2018. Now the Brewers are the subjects of suspicion. And on and on it goes. None of the other sign-stealing rumors are as credible or well-documented as the Astros reports. But it’s hard to take the high road against a crooked rival when wins and seven- or eight-figure salaries are at stake.

One might be able to form an unbroken daisy chain of sign-stealing schemes that dates back to the beginning of baseball. Dickson notes that the first recorded allegations of sign-stealing from beyond the confines of the field arose in 1876, the inaugural year of the National League, when the Hartford Dark Blues were said to derive “a lot of dope” from a shack attached to a telegraph pole. The Phillies of the turn of the 20th century pioneered the approach of positioning a player beyond the outfield wall, where he would steal signs with binoculars and relay them to the hitter in real-time by manipulating a fence sign or signaling via a telegraph wire attached to a mechanical buzzer beneath the third-base coach’s box. The Pirates employed a similar scheme, and they and the Phillies agreed not to spy on each other.

Dickson’s accounts of dozens of sign-stealing episodes make it sound as if virtually every 20th-century team placed spies in its scoreboard at some point to peer at the signs and transmit them to hitters. “Periodically, someone would complain that they were being spied upon by men out of uniform hanging out in the scoreboard, and they would be answered by the official equivalent of a shrug and a scowl,” Dickson writes. When the TV camera came in and zoom lenses proliferated, sign-stealing teams switched to video (when they weren’t bugging the clubhouse), and many more minor scandals ensued. Over and over, the pattern repeats: Public accusations surface, fans are scandalized, players and pundits debate what is and isn’t acceptable, and the league sweats it out until tensions subside or reluctantly tries to play sign-stealer whack-a-mole.

The Astros’ brand of sign-stealing was more brazen than most, and possibly longer-lasting. In its 2017 form, at least, it’s also easier to see, now that we know what we’re looking and listening for. (They didn’t have YouTube, Twitter, and sound-processing software in 1899.) That’s embarrassing for baseball, so the Astros will pay in some way for their crimes. While the baseball world waits to hear Houston’s sentence, it will wrestle anew with some of the sport’s peskiest questions. How widespread is sign-stealing? Can (and should) sign-stealers be stopped? And maybe most unanswerable: How well does sign-stealing work, anyway?

Thanks to Dan Hirsch of Baseball-Reference for research assistance.