Investigating the relationship between the diversity index and frequency of offending

Abstract

Purpose<br/> Recent work has suggested that specialization is correlated with frequency of offending, but this observed relationship may actually depend on the measuring instrument used. The diversity index is a common method of measuring specialization in such studies, and this paper investigates whether this observed correlation is due in part to the mathematical form of the diversity index itself. The criminological question as to whether specialization increases or decreases with offense frequency cannot be answered until the behavior of the diversity index is better understood.<br/><br/>We use simulations to investigate the behavior of the diversity index where the number of crimes is small (the small sample problem), simulating from known distributions of offending. Two of the distributions used in the simulation are defined to be unspecialized. The first uses an equiprobable distribution of offenses across offense categories. The second uses the distribution of offenses in the British population. The third distribution is from a specialist distribution, and assumes that different offenders have different probabilities of choosing particular offenses. We report these simulations for both three and ten crime categories. To set the simulated results in context, we use an extract from the UK Police National Computer to investigate the criminological question as to whether specialization increases with offense frequency. <br/><br/>For all three simulation schemes, the diversity index $D$ increases steeply with the frequency of offending $N$ at low frequencies, with the increase slowing around $N=20$, and becoming flat when the number of offenses $N$ reaches 500. This relationship is observed for both three crime categories and ten crime categories. The observed relationship of D with N can be used to correct the diversity index to allow the true relationship of specialization with offense frequency to be investigated. <br/><br/>We recommend that the diversity index be used with caution when there are small numbers of crimes over fixed time periods. Any increase or decrease of the diversity index over the criminal career life course may reflect the behavior of the measurement tool with the number of offenses, rather than any change in specialization itself. Applying one of the suggested suitable correction methods to $D$ will mitigate this problem.