For extremely online baseball fans the offseason has been one of the most active in recent years. A huge cheating scandal landed very quickly after the World Series finished. The top free agent players signed early with further deals happening at a reasonable rate afterwards. The resurfacing of the scandal now implicating 2017 & 2018 World Series winners.
But I am not writing this to talk about any of these, I am writing this because the baseball analytics world has dropped two very important defensive metric changes.
To a bit on fanfare (which I was partially responsible for) on Wed 8th Jan MLB Statcast finally released their Infield Outs Above Average (OAA) metric after teasing it for nearly two season.
A few days earlier to less fan fanfare Sports Info Solutions via The Fielding Bible update their flagship stat, Defensive Runs Saved (DRS), to account for where an infielder was positioned. This has been for quite a while one of the best, if not the best, defensive metric publicly available. These new DRS numbers haven’t been updated on FanGraphs or Baseball Reference yet.
So, let’s have a look at these new-fangled defensive metrics and see what’s new. But quickly before that if you want to know more about defensive metrics in general, and how we got to here, I suggest you read this fantastic primer which was created by Tom Tango to go with the release of the new Statcast metric.
The real difference between these metrics and the ones we have had previously is that they both take into consideration the positioning of the infielder. This is mainly because there wasn’t any player tracking data from before 2015 and both of the two metrics have been slowly worked on since that data became available.
New Methodology
The above primer goes into more detail for the new infield OAA approach but it boils down to three key parts to give an overall out probability for each play. They are the following three action events
- Chance of arriving at the ball
- Then given that he has arrived at the ball, the chance of retrieving the ball
- Then given that he’s retrieved the ball, the chance he’ll win the race to the bag (ball beats runner)
With all three Action Events probabilities estimated they simply multiply them to get an estimated success rate, or the Out Probability, on this play.
For the new DRS system is very similar and involves breaking the DRS system into PARTs. PART stands for Positioning, Air Balls, Range, and Throwing. This new system isolates each of these elements for individual fielders.
The system evaluates the chance a play would be made without considering fielder positioning (using information about the batted ball’s trajectory, location, and velocity and the batter’s speed) and compares it to the chance it would be made considering those variables and the fielder’s positioning. If a batted ball is estimated to be a high percentage out, but a fielder isn’t close to it, then the team will get penalized in the form of Positioning Runs Saved—but the fielder will not.
New vs Old
The previous form of DRS and Ultimate Zone Rating (UZR) had been the gold standard of defensive metrics but their creators knew they were just stepping stones. Improvements on previous attempts to analyse fielding capability but by no means perfect.
I compared these two older metrics and against the two new ones and here is the Pearson Correlation I got. This isn’t an exact apples-to-apples comparison as the OAA metric is based on outs not runs, but given that it is just a linear weight to convert them it shouldn’t make a difference for this correlation. Also, OAA doesn’t have measure anything for double plays yet.
From these we can see that the new metrics match better to each other than any other combo except in the old DRS to new DRS. And across all of them they is pretty good correlation the difference we now see with OAA and the new DRS is there are not many where they wildly disagree.
But there still is a few which are note worthy. Marcus Semien and Ozzie Albies are rated highly by the new DRS but not OAA where as Asdrubal Cabrera and Rafael Devers (hidden behind Cabrera as both has same values) are rated highly by OAA and not the new DRS.
Statcast’s New Tools
What is truly amazing about the introduction of Statcast’s Infield OAA is the interactivity and granular level of data one can get on Baseball Savant. The brain child of Daren Williams, Savant for me is the new ‘creme de la creme’ of what sport analytics websites should aim for. Daren’s hard work along with Jason Bernard and many others in the MLB Statcast team has put a wealth of information at the finger tips of anyone who dare enter.
There are interactive leaderboards, shown above, for Infield OAA (as well as many other metrics). These give us details on which direction, from their starting position, players are good/bad at compared to their peers. If we look at the 4 shortstops you can see that all four of them are much better than at going forward to the ball and moving laterally towards 1B but it is only Javier Baez who is much better going laterally towards 3B.
You can even click on an individual player an get further details on their performance. When you do you have plethora of information available as shown below.
If you ever wanted to know if Baez, or any infielder, was good at going in any direction then here is your chance. Currently the data isn’t split down into the 3 event point metrics but according to Tom Tango that is in their road map. But you can still do amazing things with this data, let us use it to compare a couple of shortstops. Javier Baez and Marcus Semien.
The images below show where both players have started on plays they are responsible for in 2019. The size of the squares show the volume of attempt in those positions (note that legend in each is different) and the colour shows the outs above average on plays that started there (scale is the same).
From this we can deduce that Baez sets up in much more varied locations than Semien and is more often close to 3B and home plate than Semien. This is very interesting if you pair it with the fact that the Cubs shift their infields less than any other team at just 12.7%, league average at 25.6%. This could suggest that the Cubs like to move their infielders around slightly but not into full shifts.
We can also see that Baez performs above average no matter where he starts where as Semien performs worst in the position he is placed most often. Semien’s OAA for 2019 was -4 and he has -8.6 OAA from that spot alone.
We can also compare them on were they finished the attempts.
Here is where a pattern emerges for Semien this shows that he is an above average on plays to his left (3 OAA) but below average on plays to his right (-8 OAA). Baez is average or above in all directions.
This is complete conjecture but if you have Matt Chapman one side of you and Jurickson Profar on the other, which side might you lean to when setting up to field. It is hard to find footage but i would be interested to see Semien’s body position to see if he leans more to the left when readying up.
Fielding Bible’s Summaries
Sports Info Solutions via The Fielding Bible have provided player breakdowns for the new DRS. It has two pages where you can see the breakdown for any player and go into the details when split in a variety of ways.
The website isn’t as fancy as Savant but also has some interesting tidbits which can be found. You can see the runs splits between the PARTs categories, split right/left/straight on as well as you can see the values in terms of plays instead of runs.
Comparing Metrics for a Player
As I stated before Semien is an outlier between the new metrics with the new DRS has at +16 plays but interestingly they both have him being much better going to his left than right. DRS has him at +15 plays to the left.
The difficulty in direct comparison between these two metrics not only comes from them being calculated differently but also the determining on who is responsible for each play is most likely different as well. I will be continuing to look at the these outliers but this is very difficult without the raw data for each of them.
The Future is Bright
There is a lot to pick through here but there will be some gems here but tools like Baseball Savant means even those not analytically inclined can find them. Also, we are getting closer to MLB via Statcat being able to create there own proprietary win above replacement (WAR) metric.
And as Tom Tango says. For every saber question answered, it should spawn two more questions. That has most definitely been done.
I also did some of my own research to look at year-to-year internal correlations for each metric (i.e. how well the previous year predicted the next one). DRS 2.0 was around 0.45, OAA was around 0.32, and UZR was around 0.35. Not that a higher number means any of these metrics are more valid than another; I could make a perfectly invalid metric that had an internal correl of 1.00. But as we dig further into this, it will be interesting to see how much variation is actually involved in personal defensive performance from year-to-year. Maybe then we can create the first defensive aging curve (by position!).