One of the great mysteries in basketball is how to grade individual defense. Â For years, the basketball fan glanced at a player’s steals and blocks to derive their opinions on whether a player was a good defender or not. Â Blocks and steals were the extent of objective Â information at our fingertips and conceptually, it made sense to use blocks and steals as a proxy for quality of defense. Â While blocks and steals are both good contributions as a defender, they aloneÂ offer just a small window into the challenge of stopping the opposition from scoring– the ultimate goal of the defense.
Thankfully, we have more data these days. Â The great resource that is Basketballvalue.com publishes the on court/off court lineup data for every player in the league. Â What do on/off court defensive numbers tell us? They answer the following question: Â How many points does the opposing team score when the player is on the court relative to when he’s on the bench?
In many basketball analytics circles, this is the most useful measurement available. Â It ignores box score statistics all together and strictly looks at how a defender affects the bottom line: the scoreboard.
What happens when we compare the on/off court defensive stat to the box score stats? Â Maybe you are interested in who tallies loads of blocks and steals, but fails to impact the team’s efforts on the scoreboard. Â Perhaps you can’t sleep because J.J. Redick totaled just 32 steals and blocks in 1,808 minutes and you believe those numbers do a poor job of measuring his defensive ability. Â Well, I’m gonna touch on all that in today’s post.
First, I gathered every player’s steal percentage (estimate of steals per possession on the floor) and block percentage (estimate of blocks per possession on the floor) from Basketball-Reference.com. Â They are estimated because B-R’s calculation doesn’t have actual possession data but the estimate is very accurate. I used these instead of other stats measures because they remove the pace and playing time bias from overall stats numbers on the back of your old Fleer Ultras. Â Then, I only looked at player’s who logged more than 500 minutes this season to remove the wild variances due to little playing time.
Next, steal percentage:
The negative trendline tells us that the players who steal the ball a lot tend to have a better (more negative) defensive on/off court differential and steal percentage is statistically significant in predicting defensive on/off differential (p=.0493). Â While stealing is significant, you’ll notice that only a small relationship exists. Â If they were perfectly correlated, you’d see a straight line of points but the distribution is much more scattered.
There are several reasons steals have a low correlation (-0.108) with on/off differential. For one, a player can steal the ball without playing good defense. Â Most steals come from the stripping variety but players also can “steal” the ball by being in the right place at the right time and picking up a loose ball that falls to their feet. Â Just like you can’t assume a double play in baseball, you can’t assume the possession would change until a player physically picks up the ball. Â Additionally, players who go for steals all the time are playing risky basketball. Â What we’d really like to look at is net steals, or the ratio of successful steal attempts to failed steal attempts. Â Many players get tons of steals without actually improving their teams defense. Who are they? One way to find these sly cats is to compare their percentile ranks in steal percentage and defensive on/off court differential. Â Don’t look at Andrei Kirilenko. He racks up a ton of steals (2.5 stl%, 95 percentile) and also helps the teams bottom line on defense (-5.1 points, 94 percentile). Â Direct your eyes to these folk.
This should be a good time to mention that defensive on/off court stats have their faults as well. Â It is a fact statistic but it isn’t adjusted to take into account the quality of their substitute players and the lineup they tend to play in. Â Additionally, it’s still subject to random variation since there’s a lot of different lineups to tease out one player’s contribution.
Nonetheless, this is still an eye-opening exercise. Â All these players saw their teams play better defense (fewer points allowed) when they were off the court. Â Monta Ellis has one of the highest steals rates in the league but opposing teams scored more 4.1 points when he was playing defense than when he was on the bench. Â Many of these guys are speedy guards but you also have Rasheed Wallace and Jeff Green in there too. Â Jeff Green was this year’s worst defensive player if you use the on/off court number as the measuring stick.
I wouldn’t say all of these players are categorically overrated as defenders but we also shouldn’t let their steal numbers color our evaluations. Â Steals are good but these players probably exhibit more risky and opportunistic defensive strategies rather than staying home and forcing bad shots.
Let’s move on to blocks. Â Here’s what it looks like when we plot block percentage vs. defensive on/off court differential:
There’s Â a stronger relationship between blocks and defensive on/off court differential than there was for steals. Â Shot blockers clog the paint and prevent high percentage shots whereas guys who accumulate steals only get a couple per game and have less of an impact on preventing scoring.
Still, only 2.7 percent of the variation in a player’s defensive on/off court differential can be explained by their block percentage. Â That might seem miniscule but also consider that there’s a fairly large percentage of variation that is simply random/luck. Â But there are players who are empty shot blockers. Â Typically, you see these guys swat every ball in sight, thereby leaving themselves vulnerable to pump fakes and weak side cuts. Â Additionally, they tend to foul a lot and fouling a player going up for a shot is just about the worst thing a defender can do aside from laying down on the floor.
Let’s take a look at the shot blockers who actually don’t improve their team defense (as measured by defensive on/off court).
Theo Ratliff saw a huge boost in minutes after arriving to Charlotte from San Antonio this season. Â Always a big-time shot blocker, he’s no longer the defensive asset of old. Â He doesn’t take advantage of his height on the boards and he’s certainly slowed down at age 36. Â The Bobcats were about 4 points better defensively with him on the bench, despite his shot blocking talents.
With Kurt Thomas, Nazr Mohammed, and Ratliff, we have a nice group of older veterans who may not have the quickness to stay in front of their younger opponents. Â But we also have some youngsters (DeAndre Jordan, Brook Lopez, and JaVale McGee) who may have some more work to do.
As a reminder, the on/off court data aren’t adjusted for substitute effects and they are still vulnerable to random variance. Â For non-rookies a good follow-up would be to do a 3-year average on/off court to get more accurate data but that would ignore yearly improvement and development.
In the end, it’s best to look at several measures to get an objective defensive evaluation. Clearly, there’s more to defense than just steals and blocks but you wouldn’t know that by looking strictly at the box score. Â Steals percentage has little effect on the aggregate scoreboard and it’s best to not let them paint your overall evaluation of the player.
View the data here and you can see that J.J. Redick’s defensive on/off court differential sits in the 9th percentile. Â Not so good.
Big thanks to Aaron Barzilai of basketballvalue.com for publishing on his site.