The Fallacy Of Small Sample Size

After their Monday night thrashing of the Houston Rockets, a game they won 137-118, the Los Angeles Clippers are looking pretty, pretty good.

They’re lead by this one guy who’s really, really good at basketball. You’ve probably heard of him. His name is Chris Paul. He has a twin who sells insurance. He’s averaging 26 points and 13 assists through his first four games. And if he were able to do that for the rest of the season, it’d put him in rather elite company.


So, yeah. That’s a nifty list.

Of course, he’s probably not going to do it. It’s a long season, after all, and there are too many factors working against him. Teams will undoubtedly adapt to the Clippers’ renegade offense. The wear and tear of an 82-game season will take its toll, particularly on a player with knees made of Lincoln logs like CP3. Even considering the possibility is an exercise in extrapolation, an exploration of the infinite limits of small sample sizes.

And what good is a small sample size, anyway? There’s not much to take from it. Trends that seem readily apparent will soon be washed away in the tidal fury of an ever-growing data set. A few shots rattle out, and a player is the worst shooter in the history of the NBA. A tipped loose ball caroms off a shin — and it decides the game. The longitudinal tenacity of the regular season tends to even out these trends in their own time, but that can only be the case if we’re patient enough to wait for the market correction to take place.

Another example: Derrick Rose probably isn’t going to be quite this awful all year:

That stat comes courtesy of the shiny new SportVU data on NBA.com. To this point, said data is the definition of small sample size. I’ve seen it derided time and time again for being misleading at best and worthless at worst. “What is there to glean from four games?” tends to be the line of thinking.

In the predictive sense, the answer is little, if anything. Those who try to use such a limited view to guess at what’s yet to come might as well be searching for extraterrestrial life through the lens of a glass onion.

But dismissing small sample sizes as worthless is equally disconcerting. True, there’s little predictive value in the results of four games, but it’s rather pertinent that Rose, for instance, has shot so poorly on drives so far. The Bulls have looked out of sorts; knowing that Rose has been unable to finish at the rim, and knowing the extent of his troubles, helps to add depth to the portrait we’ve seen take shape.

It might be forced perspective, but it’s perspective worth having. No, Brook Lopez isn’t going to continue to allow opponents to shoot just 26.7% at the rim when he’s in the vicinity.* But his ability to protect the basket has been huge for Brooklyn in the early going. When analyzing that which has transpired, a small sample size is plenty big enough.

*As much as one can credit him for such defensive prowess versus variance, his teammates and the quality of the shooters in question, but that’s a whole other hill of beans to tackle at a later date.

So by all means, dismiss the relevance of small sample size when your gaze falls on the horizon. But when you look to the past and grasp at understanding, let what has been inform your analysis. It might not mean much going forward, but history is still history.

Statistical support courtesy of NBA.com/stats

Andrew Lynch

When God Shammgod created the basketball universe, Andrew Lynch was there. His belief in the superiority of advanced statistics and the eventual triumph of expected value-based analytics stems from the fact that he’s roughly as old as the concept of counting. With that said, he still loves the beauty of basketball played at the highest level — it reminds him of the splendor of the first Olympics — and the stories that spring forth from the games, since he once beat Homer in a game of rock-paper-scissors over a cup of hemlock. Dude’s old.

  • Pingback: ICYMI: Suggested Basketball Reading From Across The Web()

  • RajivB

    “In the predictive sense, the answer is little, if anything. Those who try to use such a limited view to guess at what’s yet to come might as well be searching for extraterrestrial life through the lens of a glass onion.”
    Nicely put! Agreed!