To best understand this article in the context of the behavior evaluation literature, please see National Canine Research Council’s complete analysis here.
Article Citation:
Patronek, G. J., & Bradley, J. (2016). No better than flipping a coin: Reconsidering canine behavior evaluations in animal shelters. Journal of Veterinary Behavior: Clinical Applications and Research, 15, 66-77. doi: https://doi.org/10.1016/j.jveb.2016.08.001
National Canine Research Council Summary and Analysis:
Using a detailed hypothetical with statistics drawn from the canine literature, Patronek and Bradley* explain why, even in the most optimistic circumstances, behavior evaluations are still “no better than flipping a coin.” It is important to note that in all situations where estimates were required, the authors chose the most generous numbers (those in favor of behavior evaluations as a useful tool). The authors define and explain the key attributes of diagnostic tests, including sensitivity, specificity, positive predictive value, negative predictive value, and the prevalence of the behavior which is meant to be predicted. Sensitivity and specificity are inherent characteristics of a diagnostic test, and they refer to the ability of the test to correctly identify individuals with the target condition, and the ability of the test to correctly identify individuals who do not have the condition, respectively. Though often confused with sensitivity and specificity, a test’s positive and negative predictive value (PPV and NPV) are separate concepts. PPV refers to the proportion of individuals who test positive who actually have the condition; NPV to the proportion of individuals who test negative who do not have the condition. Positive predictive value of behavior evaluation reveals the proportion of false positives (i.e., the number of dogs deemed aggressive who would not show aggressive behavior in the home). Because dogs who fail behavior evaluations are more likely to be euthanized, it is important to know the positive predictive value of the tests.
The authors used existing data on dog bites and owner surrenders attributed to problematic aggression to determine the likely prevalence of these behaviors among dogs living in shelters. They estimated the sensitivity and specificity of evaluations based on the best results achieved in analogous human behavioral diagnostic testing in the context of these prevalence estimates. This parallel is useful because unlike diagnostic tests for disease and illness where the condition is discrete and static (e.g., a tumor is present or not present), behavior evaluations are meant to measure plastic behaviors that fall on a spectrum with variance in prevalence and intensity.
Several hypothetical scenarios were discussed in this report, including realistic outcomes and unrealistic, best-case outcomes. This demonstrated that even if tests that yielded very good rates of specificity and sensitivity could be developed, statistically, the predictive value of behavior evaluations would still be ~50% for predicting whether a dog will exhibit threat or biting behavior problematic to the owner after adoption. In the more realistic scenario representative of today’s tests, the predictive value falls to a miserable 12%, meaning that ~88% of dogs who fail the evaluation are false positives. When this is adjusted to predict medically treated dog bites (a much lower prevalence than bites in general), the number of false positives jumps to ~97%.
The simulation carried out by Patronek and Bradley demonstrates the futility of behavior evaluations. In conjunction with other studies which show weak reliability and validity, this paper makes a strong case for discontinuing the use of formal evaluations in shelters. The authors suggest that in lieu of provocative tests with little predictive value, shelter staff should collect behavior histories when possible, verify serious reported incidents, and screen out dogs that are too threatening to handle safely. Beyond these measures, resources should be spent on positive, enjoyable activities, such as walks, human socialization, and dog play groups.
*Please note that the authors are affiliated with National Canine Research Council. Gary Patronek is a paid consultant for National Canine Research Council, and Janis Bradley is an employee of National Canine Research Council.
