Do you know how well your team does at challenging calls? I don’t know how well the Indians do.
I was listening to a podcast last week and was surprised to hear them say that MLB’s instant replay challenge system has been in place since 2014. I would have sworn it had only been a couple of years, but they were correct. There were some minor tweaks in 2015, but the system as we know it originated in 2014. But it got me thinking how good are teams at doing this and are there any interesting usage habits across the league.
Quick Replay Recap
In January 2014, MLB officially announced the approval of expanded instant replay for use during the 2014 season. Managers were allotted one challenge per game, but a manager will retain his challenge after every call that is overturned (in 2014, you were only allowed two in total even if both were successful, but this was changed for the 2015 season). Once a call is challenged an umpire requests a video review, fellow umpires in New York’s Replay Operations Center will watch video of the play in question.
A review can have one of three outcomes: confirmed, stands or overturned. The first and last of them are when the video replay shows enough evidence for the umpires to either confirm or overturn the decision. Stands is when the video replay is inconclusive, so the on-field decision stands.
The following plays are reviewable under the system:
- Ground-rule doubles
- Fan interference calls
- Boundary calls (managers may not, however, challenge home run or potential home run calls)
- Hit-by-pitch calls
- Force plays at all bases, except whether a middle infielder touched second base during the attempt to “turn” a double play
- Tag plays on the base paths—whether a runner was tagged or whether the runner touched a base (an appeal is still required ahead of the latter)
- Fair/foul calls on balls hit into the outfield
- Catch/trap calls on balls hit into the outfield
- Whether a runner left the base early or properly touched a base on a tag-up play
- Time plays (whether or not a run scored prior to the third out)
- Slide rule calls
- Whether a runner passed a preceding runner
- Scorekeeping issues, including the count, number of outs, score or substitutions
Force & tag plays account for 85% of all challenges since 2014, with hit-by-pitch calls the next highest at just 5%.
Overall Usage
In 2019, we saw 1,139 managerial challenges, which was the lowest total since the inaugural season. Also, the success rate of these challenges is at the lowest level with just 47% being successfully overturned.
The difference over the last six seasons has come from more calls standing and not more calls being confirmed. So, to me, teams aren’t getting worse at asking for challenges, they are just asking for replays on plays more often when the evidence is inconclusive. Like you would expect on tighter plays.
Making games longer?
In 2019, the 1,139 reviews lasted for 77 seconds on average which, was the lowest average time and 45% lower than the longest when reviews were talking 111 seconds back in 2015. But given there were 2,429 games last season these replays added just 36 seconds to the average game.
Given that the average regular season game length in 2019 was 3 hours, 5 minutes and 25 seconds, perhaps fans and the commission should look elsewhere if they want to speed up the game as there isn’t much time to be saved here as replays account for 0.7% of that time.
When to expect
It is not surprising given the numerical restriction of one incorrect call, but challenges are less likely to occur in early innings of games. With the likelihood increasing by over 50% by the time games reach the 9th inning or extras. But the success rate in later innings is worse.
These results line up with a game theory approach that teams are only going for the replay challenges they are more confident on early in games because they don’t want to be in the situation in the game later one where they need it but have wasted their one replay. Whereas later on in the game, they are far more liberal with their challenges as there is less time afterwards in which they may need the challenge.
If we combine the likelihood of the challenge occurring with the success rate of each inning, we get a rate of a successful challenge around 1.5% for all innings.
Team by Team
So, which teams have the highest challenge success rate.
Most of the teams are within ±5% of 50% across all six seasons, but the Yankees and the Royals stand out as the most successful, while the Blue Jays stand out as the least successful. But they don’t stand as clear atop the standings for most successful challenges.
The Royals are only one clear of the Cubs who had a significantly lower success rate (51.2%) and the Yankees drop down to 6th overall. The Yankees seem to have a very high standard for making a challenge; in 2019 they didn’t have a single replay challenge where the on-field call was confirmed. All their challenges calls were either overturned or stood. To me, there does seem to be some skill to this but also very different approaches which teams, who are described as analytical, take.
The Rays take a completely different approach to the Yankees, they have six more successful challenges (147 to 141) but have made 325 challenges over the six seasons compared to the Yankees 198. Despite their different approaches, they both have ended up close to the optimal result level, but the same cannot be said for all other teams.
Challenge more or challenge better?
Without watching every game, we cannot know how many incorrect calls are made against each team. But given the names we see at the top of that list, with good, bad and average teams being up there, it would suggest that the numbers might be fairly even across all teams. If that is the case, we need to look at the teams who are at the bottom of the successful overturns table and ask what is going on.
In six seasons of MLB, the Orioles have had less than 100 calls successfully overturned, but they have an above-league-average success rate of 52.1%. The reason they are at the bottom is that they have only challenged calls 188 times across all 6 seasons. That is the lowest number of any team. If my hypothesis above is correct, then Baltimore should be challenging more often than they are.
The Blue Jays have a success rate of just 37.8%, which was significantly the worst. Interestingly on challenges in the first innings, they are at 77.%, which is 7.7% better than league average (70%). But from the 2nd inning onward they only succeed 34.5% of the time which, is 14.4% worse than league average and 6.4% worse than any other team. From the looks of it, the Blue Jays are being far too aggressive with their challenges and it is probably costing them later in some games.
So, some teams look like they have gotten this all worked out, but others seem to be behind. It will be interesting to see if these trends continue in the season to come.
Oddities
There are plenty of things I found in looking through this data that don’t fit in any of these sections so here are my favourites:
- Terry Collins is the only manager to have challenged a call in an All-Star game in the six-year window; he successfully had a force play overturned.
- The most successful challenges there has ever been in one game is three, and it has been done eight times to date. Chris Woodward was the last person to achieve this on 6 April 2019.
- Rocco Baldelli has the worst success rate of any full-time manager at 32.7%, whereas Joe Giradi leads with a 71.6% success rate.
Russell will shortly be presenting research at the 2020 SABR Analytics Conference in Arizona. Make sure you’re following him on Twitter @REassom
Have you ever wanted to write about baseball? Do you want to tell the story about your baseball fandom journey? How about a favourite player? Or, just get silly and write something to make people laugh? Slide into our DMs, contribute now. We’ve never rejected a post.
Make sure you subscribe to the Bat Flips and Nerds podcasts and follow us on Twitter @BatFlips_Nerds for news, views and interviews, all with a British twist.