Are Officials Biased Against Nebraska?

This article was sent to me by someone who doesn't wish to have their name published other than their screen name, moosew648.

It includes statistics, histograms and scatter plots, and it's long. If you don't like reading that sort of thing, then look away.

I'm not going to add any of my own commentary so feel free to add your own. (For complete disclosure, I took courses in statistics for Engineers while at Nebraska - but I probably killed most of those brain cells along the way.)

First, some preliminaries.  The most appropriate way of assessing whether penalties are disproportionately biased against Nebraska is to compare the penalties in Nebraska games to other games in which our opponents play.  This way, we can hold constant one of the important factors in this relationship: the tendencies/abilities of Nebraska's opponents.  To this end, I create a data set of each game that Nebraska's opponents play.  For each game, I collect the number of penalties and yards by the Nebraska opponent, and the opponent's opponent. 

Are refs biased against Nebraska?  There are two ways where this bias can manifest: in penalizing Nebraska or failing to penalize the opponent.  Let's examine each of these in turn.

Nebraska is ranked 101 nationally in penalties, with 70 penalties for 652 yards.  On average, this means that Nebraska has more penalties than the other team.  Perhaps there is something about each team that causes the other team to experience more penalties.

Average number of penalties and penalty/yards for the opponents of Nebraska's opponents, and Nebraska's penalties for that game.

This table shows the average number of penalties and yards from penalties for the opponents of our opponents this year.  We can also compare these numbers to the penalties that Nebraska had during those games.  For the most part, this tells us something that we already know; Nebraska is a highly penalized team.  What this doesn't answer, is whether there is a statistical difference between the average level of penalization for Nebraska and the other opponents. 

To answer this question, we can divide up the data set into two categories: games involving Nebraska, and games involving Nebraska's opponents.  We can then determine whether the average number of penalties committed by Nebraska is substantially different than the average level of penalties typically committed by other teams when facing the same opponents. 

Number of penalties per game:

This shows that Nebraska has more penalties, on average, than the opponents of all the teams that Nebraska played this year.  We can also be reasonably confident that the difference in average penalties per game is statistically different from 0 (p-value < .10). 

The story is similar for the number penalty yards per game.

This shows that Nebraska is penalized, on average, for more yards than the opponents of all the teams we played this year.  Moreover, we can also be quite confident that this difference is statistically significant (p-value < .05). 

So the primary take-home point here is that Nebraska gets penalized more often than other teams.  This could be due to a number of different explanations.  First, Nebraska could be a poorly-disciplined team.  This is certainly possible, and we have seen time and time again the sort of unsportsmanlike penalties that either kill drives or save drives.  Unfortunately, without data on the types of penalties (i.e., holding versus unsportsmanlike), we are unable to rule out this explanation.  The second explanation is that the refs may simply be more flag-happy in those games involving Nebraska.  In other words, Nebraska is penalized more because both teams are penalized more.  To examine this, I create two different scatterplots intended to demonstrate the relationship between two teams' penalties.  I then provide a scatterplot for the games not involving Nebraska and then for the games involving Nebraska.  If this explanation is correct, then we would see a strong, positive relationship between these two variables (meaning that both teams are either penalized more or less, depending on the referee's disposition).

Scatterplot of Teams' Penalties (without Nebraska games)

Scatterplot of Teams' Penalties (with Nebraska games)

Now, one must be careful in drawing too strong of inferences from a sample of only 11 games, but the differences between the linear regression lines in these two figures are striking.  In the first figure, we see a weak, positive relationship.  This indicates that for the sample excluding Husker games, increasing the number of penalties by one team also slightly increases the penalties for the other team.  In these cases, there are either refs who are either more lenient or more strict in calling the game. 

In the sample with the Husker games, the relationship is weak and negative.  This indicates that when Nebraska's opponent receives more calls, then Nebraska will receive less.  This is most likely driven by the presence of the outlier on the right side of the graph (Texas A&M game), but even if we exclude this observation, the relationship is flat, indicating no relationship.

From these preliminary graphs, we can conclude that the reason for Nebraska's penalties is most likely not due to referee style (either lenient or strict), at least not for the sample of Husker games. 

The third and final explanation is that the refs are somehow biased against Nebraska and are more willing to penalize them.  This is by far the most unrealistic and most far-fetched of the three explanations.  Unfortunately, we cannot directly test this proposition, but we can look at another characteristic of penalties that might shed some light on whether the refs are biased against Nebraska.

The second way in which the refs' biases can appear is by failing to penalize Nebraska's opponents.  One way to get a gauge of the average level of penalties is to compare how the opponents do in games against the Huskers compared to all their other games.  If Nebraska's opponents are penalized much less in games against the Huskers, then that might be evidence that refs favor the opponents.  Let's take a preliminary look at the results.

Average number of penalties and penalty/yards when playing Nebraska versus all other teams

The average number and the average yards columns represent the opponents' average penalties for the other games on their schedule (excluding Nebraska).  This gives an idea of how prone that team is to committing penalties.  We can then use this average "penalty proneness" to games involving the Huskers.

Of the 11 games, four of the opponents experienced more penalties than they usually did.  Of these four, Oklahoma State and Missouri were the only two teams that experienced more than one penalty higher against Nebraska than their average (1.3, and 1.2, respectively).  On the other hand, of the 7 games where teams were penalized less than their average, five teams were penalized less than half as much as they usually were.  The most striking example is either the Kansas game (penalized once compared to their average of 6.8 times a game), or the Texas A&M game (penalized twice for ten yards compared to their average of 8.7 times for almost 69 yards a game).  This would seem to support the hypothesis that Nebraska's opponents are penalized less than usual. 

A much more systematic test is to examine the different average levels of penalties and penalty/yards between those games involving Nebraska and those that don't.  If we find that Nebraska's opponents have lower average number of penalties when they are not playing Nebraska, then there may be a bias.

Average number of penalties by Nebraska's opponents when playing Nebraska and when playing other teams:

And, for penalty/yards:

These two tables show that, on average, when Nebraska's opponents play teams other than Nebraska, they will have 2.2 more penalties and 20.8 more penalty yards than they would if they played Nebraska.  Substantively, this means that simply by playing Nebraska, Nebraska's opponents will receive fewer penalties resulting in fewer lost yards.  We can also be extremely confident that this result is statistically significant (p-value < 0.01 in both cases).  Thus, we know that there is less than a 1 in a 1000 chance that we would see this difference in averages if there was actually no referee bias.

While these numbers are particularly damaging for those who suggest that there is no anti-Nebraska bias, it is not a direct test of the motivation behind the supposed bias.  Some Nebraska fans are paranoid that Nebraska's move to the Big Ten has triggered this backlash.  If this is correct, then we would expect to see an even bigger bias in favor of penalties for opponents in Big XII games. 

In the next two tables, we replicate the above tests but for a sample of only Big XII games. 

Average number of penalties by Nebraska's opponents when playing Nebraska and when playing other teams (Big XII games only):

And, for penalty/yards:

In Big XII play, on average, a team should expect to have about 7 penalties a game (6.9), for a total of 60 penalty/yards.  This changes if they are playing Nebraska, however.  If they are playing the Huskers, then they should only expect, on average, 4.1 penalties for 38 yards.  When we compare the entire schedule to only Big XII play, we can see that the anti-Nebraska bias is slightly larger for Big XII play.

Therefore, we can make two observations: first, that Nebraska is penalized to a greater extent than other teams (which is most likely unrelated to referee bias), and second, that Nebraska's opponents are penalized less on average.  Altogether, this appears to be a double whammy because not only is Nebraska moving backwards on the field, but the other team is moving forward.  The true damage in terms of changing the outcomes of games is probably un-estimable. 

A final observation to make can be made by looking at the difference in penalties and penalty/yards for each team and opponent.  By doing so, we can examine the (im)balance of penalties in a game.  For example, we can determine if Team A had 8 penalties for 40 yards and Team B had 7 penalties for 30 yards.  While this difference may be positive or negative at the individual-game level, on average, as the number of games increases, this should be close to 0.  This would imply no difference in the average penalties across teams.  Of course, this would be the case if there was no bias. 

Let's look at a histogram, which gives a sense of the distribution of these values in our sample.  Keep in mind that a positive value on this difference means that Nebraska's opponent is penalized more than its opponent in that game.  The first histogram excludes Nebraska games.

This is shaped like you would expect.  The height of the bars represents the number of games that had that difference in penalties.  Most of the games are quite close to zero, indicating a relative balance of penalties for both sides.  There are some larger positive numbers and negative numbers, but much like a bell-curve (or a normal distribution), these are relative rare.  The mean reflects this parity, as the mean is close to zero as well (0.07). 

Now, compare this to a histogram of the 11 Husker games (with wider boxes because of the smaller sample size):

The maximum value is 1, which indicates that the biggest difference in penalties in favor of Nebraska was +1.  On the other hand, there were five cases where Nebraska had 5 more penalties against it than its opponent.  However, take caution in interpreting this in favor of the anti-Husker hypothesis because this is a function of Husker penalties as well as opponent penalties.

When we look at penalty/yards, the results are even more striking.   Recall that the average (mean, median and mode) should all be close to 0.

The mean for all the games (excluding Nebraska) is 2.04, which is close to 0. 

The mean for the Nebraska sample is -34.3, which implies that Nebraska, on average, is penalized for 34 more yards than the opponent.  Not only are these differences statistically meaningful, but it is probable that they have the effect of either putting points on the board for the opponent or leaving points off the board for the Huskers.  Thus, the influence of penalties on the outcomes of game is incalculable, but most likely substantial.

To conclude, the differences in penalties and penalty/yards in Husker games is quite large, and in favor of Nebraska's opponents.  This is a function of two components: first, that Huskers are penalized more often, and second, that Nebraska's opponents are penalized less.  While the former is most likely unrelated to referees' dispositions, the latter is possibly a more direct test of the bias hypothesis.  Again, we are extremely confident in the difference in penalty yards, as there is a less than 1 in a 1000 chance that we received this result due to random chance.  This implies that we are extremely confident that there is a bias against Nebraska.  There is also evidence that this bias is larger in Big XII play.