National Canine Research Council Summary and Analysis:
This study explored two aspects of visual breed identification: consistency among experts and validity when compared to DNA analysis. The researchers specifically focused on visual identification of “pit bull-type dogs,” as they are most commonly targeted in breed-specific legislation (BSL). The term “pit bull” does not denote a specific breed and there is no agreed-upon definition of “pit bull-type dogs” in science, the law, kennel clubs, or animal shelters. The researchers in this study defined the term for their sample. Any such definition is arbitrary and cannot be assumed to apply to any other contexts. The authors accepted a wide range of identifiers on the part of the participants as designated “pit bull-type” labeling.
A strength of this study is the sample itself; shelter managers specifically selected the participants due to their routine visual breed-assignment duties—staff who visually identify dogs’ breeds when they come into a shelter and may be called upon to testify to a dog’s breed in BSL disputes. Hence, the decisions made by this survey group regularly have real-world consequences.
Participants included sixteen raters (three staff members and one veterinarian from four different Florida shelters) who visually identified a total of 120 dogs. Thirty dogs with a wide range of morphology were selected at each shelter. The dogs’ breed labels at intake were noted and later compared to the raters’ responses, but were not visible to the raters during the experimental process. However, individual raters may or may not have had previous knowledge of the dogs in the study, so the identifications were not truly “blind.”
Raters examined the dogs in their kennels and assigned a primary breed and a secondary breed if they chose. If the rater could not see enough of what he or she considered “defining characteristics” for a primary breed identification, they could indicate “mixed breed.” If a rater identified the dog as American Staffordshire terrier, Staffordshire bull terrier, “pit bull,” “American Pit Bull Terrier,” or “mix” with any of these labels, they were counted in this study as having identified the dog as a “pit bull-type.” The aforementioned labels were considered a DNA match in this study if the test results showed 12.5% or more American Staffordshire terrier or Staffordshire bull terrier, the equivalent of one great grandparent. Thus, the criteria for a match were extremely broad, including multiple breeds and labels and very low DNA material percentage.
Blood samples from each dog were sent to the Wisdom Panel Professional Canine Genetic Analysis, MARS Veterinary for DNA identification. DNA analysis revealed that 21% of the sample dogs had “pit bull-type” heritage (as defined by the study: 12.5% or more of American Staffordshire terrier or Staffordshire bull terrier), though none were purebred and none contained more than 50% of the “pit bull-type” breeds. Median agreement between visual identification and DNA analysis for discriminating between “pit bull-type dogs” and “non pit bull-type dogs” as defined in this study ranged from 67-78%. Twenty of the twenty-five dogs identified as “pit bull-type” by DNA analysis, as defined by the study, were so identified by at least one shelter staff, while only 12 had been identified as such at the time of intake. The average sensitivity for identifying “pit bull-type dogs” as defined by the study was 50%. Of the remaining 95 dogs who fell into the “not pit bull-type” group by DNA analysis, 38% were nevertheless given the label by at least one breed assessor. Of the dogs whose DNA did not reveal contributions from Staffordshire bull terrier, or American Staffordshire terrier, more than 1 out of 3 were labeled what the study grouped as “pit bull-type” by at least one staff member.
In sum, the overall validity (specificity and sensitivity results) even with such a broad target for identification, did not reach the level of a coin toss. There is a flaw in this study, as in other studies (Hoffman, Harrison, Wolff, & Westgarth, 2014; and Gunter, Barber, & Wynne, 2018) that cannot be ignored: the arbitrary classification of “pit bull-type” dogs. The authors define it to mean any dog with a DNA signature comprising of at least 12.5% American Staffordshire Terrier or Staffordshire Bull Terrier. However, they include other labels and breeds in their definition of what constituted a match for a “pit bull-type” dog. While these breeds were defined in this sample as “pit bull-types”, in reality there is no agreed upon definition for the classification—to reiterate, not in science, the law, kennel clubs, nor animal shelters. The researchers created their own definition, and definition for a match, but it is meaningless in any other context than this individual study.
Like the studies that preceded it, these data underscore the inconsistency and unreliability of visual breed identification and thus undermines any attempts to correlate breed (given that a dog’s breed was determined by visual assessment) with behavior on the part of scientists, media outlets, veterinarians, shelter staff, and policy makers.
Page last updated May 15, 2019
