It's Evident

Back

Interpretation of Crime Laboratory Reports
Barry A. J. Fisher, M.S., MBA, Retired Crime Laboratory Director for the Los Angeles County Sheriff's Department1

This short paper is designed to give trial attorneys some basics on how to examine expert witnesses. Naturally, it is not possible to become “pros” in this area right way. It takes practice, but this information may give you some hints on how you may proceed. At least, hopefully, you may have a better understanding of conclusions stated in crime laboratory reports.

What do terms like “match,” “similar to,” “the same as,” “identical,” and a host of other vague terms actually mean? What does the expert mean when he reports interpreting crime lab reports.

A brief introduction to expert opinions proffered by forensic scientists.

Lawyers and laypeople alike, often have difficulty understanding the meaning of conclusions offered in crime lab reports. For example, paint collected at the scene of a hit-and-run “matches” the paint from the suspect’s vehicle? What does this mean? Did the suspect’s vehicle leave that paint sample at the scene for certain or is it likely or perhaps, just “pretty certain?” Language fails us when it comes to explaining these subjective conclusions. More importantly, how might a defense lawyer cross-examine a government expert to help the trier of fact better understand the meaning of an expert’s assertions about physical evidence?

There are a number of classifications within physical evidence generally examined in a police forensic lab. Some evidence involves chemical testing of substances in question to establish an element of the crime. Narcotics tests (controlled drug in question) and blood alcohol tests (blood alcohol level at or above the legal limit) are the most common analyses conducted in crime labs. These tests are straight-forward and involve qualitative tests for the material in question, e.g. the presence of cocaine. A qualitative analysis test is conducted for the presence of a chemical and a quantitative examination measures the amount of material present. Tests for blood alcohol concentration are typical of quantitative analyses. Conclusions in qualitative and quantitative tests are uncomplicated: cocaine was present or the blood alcohol test result was 0.15%.

Other types of evidence may be referred to as pattern evidence and generally involve comparing one item of evidence to a known or standard sample. Test results in these classifications often use terms that can be ambiguous. The examiner compares one thing against another and reports a conclusion. Experts try to express some degree of connection or interrelation between a sample and another item. However, terms to express these relationships between the items are not standardized and are subjective.

When you think about how statements are used to express commonality or similarity, it becomes obvious that experts are trying to convey a notion of the interrelationship between a known sample and an evidence sample - often referred to as the “K” (for known) and the “Q” for questioned. The expert, through his education, training and experience opines that K and the Q have some relationship between each other - they came from the same source or perhaps a common source. By using terms such as “similar,” “consistent with,” “match,” “identification,” “unique,” “identical,” and so forth, the expert is trying to express the level of confidence that he has concerning the results of his identification.

Unlike DNA evidence, which uses statistics to convey the likelihood that two specimens are from a common source, statistics are not readily available in pattern evidence cases. We do not have likelihood ratios to tell a jury that a latent specimen came from a specific person. A conclusion that an expert might offer is that “based on my knowledge, training and experience, it is my opinion that this print came from the defendant.” If pressed on the certainty of his opinion, the expert could state that “while I haven’t looked at every fingerprint in existent, I am of a very strong opinion that the print came from the defendant.” That conclusion is far different from one that indicates the print came from the defendant, to the exclusion of every other person on the planet - a conclusion that is beginning to be used less.

Historically, it has been a basic tenet in criminalistics that certain types of physical evidence may be individualized, that is, shown to have come from a unique or sole source. Thus an expert might contend that, with sufficient data, bullets can be shown to have been fired from single gun, latent fingerprints came from a single person, shoe prints and tire impressions from a single source, etc. The notion made in this assertion of uniqueness or individuality is that given a sufficiency of comparative data, a conclusion that the sample in question came from a sole source, to the exclusion of anything or anyone else. This assertion of uniqueness or individuality has come under fire in recent years.2

The rise of statistics in forensic science:

In the past, statistics were used to a limited extent with forensic biology cases. The likelihood of ABO blood groups, A, B, AB and O types were well known. Type AB was the least common and happened in about 4% of the population. The statistical occurrence of polymorphic enzymes, such as phosphoglucomutase or PGM was also well known, and limited the odds further. But DNA evidence, with its exceedingly high likelihood that the sample came from one person out of perhaps a billion changed everything in profound ways. Juries were being given information that for practical purposes placed the defendant in contact with the victim with almost absolute certainty.

DNA evidence, with its use of statistics to describe the significance between evidence sample and a known sample is largely the driving force behind this change in how we express conclusions about the commonality of evidence sources. Prior to the use of statistics in DNA evidence, conclusions were more vague. The use of terms such as consistent, similar to, same as, etc. was rarely challenge to ascertain what the expert meant or how he came to that conclusion.

Another criminalistics concept is the term identification. That term is used to describe the forensic examination of mass-produced items, for example, clothing. No matter how much testing is conducted, source attribution can only be to a commonly made source, perhaps thousands or tens of thousands of garments made from identical material. Other examples are: building materials, paint, glass, fibers, textiles, etc.

There is an overlap between the concepts of identification and individualization. Take, for example a pair of running shoes with a particular tread pattern. Initially, a pair of shoes would have the same pattern as any other new pair. Shoes exhibit marks that are the same as any other shoe of the same size and make. They have class characteristics and may be identified as coming from a common source. With usage and wear, the sole pattern will change. With careful examination the tread may take on unique characteristics and after a time these changes may become unique to a shoe impression left at a crime scene. The challenge is that opinions about similarity and uniqueness are subjective. Different experts may see different markings which allow them to conclude the shoe and the crime scene mark are the same or perhaps unique.

Examiners have opined that inductive reasoning, i.e., observing items of evidence over and over again, allows the expert to conclude that two items are identical. Statistically, this assertion has yet to be proven.3 Over a career, an examiner may conduct thousands of examinations on impression evidence cases. Over time, he may convince himself of the uniqueness of individual items because of those observations. For the present, suggesting that one item is identical to another, to the exclusion of any other has yet to be supported by the larger scientific community (i.e., those outside of forensic science).

On a similar topic, the notion of uniqueness is compounded by other concerns. Observations, which lead an examiner to conclude that evidentiary items are the same may be erroneous. What if the characteristics we are viewing are not significant for some reason? Consider, for example, the analysis of trace elements in bullet lead. This procedure was used to link bullets found at a crime scene to bullets found in possession of a suspect because of a belief that these similarities in the composition of lead used in the manufacture of bullets could be used to prove a common source. The National Research Council4 concluded that the statistics behind an assertion of a common source could not be made. The result was that hundreds and perhaps thousands of cases were undermined.5

Admissibility of scientific evidence:

In 1923, the Federal Court held in the Frye6 case that scientific procedures were valid if there was general agreement within the relevant scientific community that a procedure was valid. That view held until the Daubert7 case was heard in the US Supreme Court. Daubert shifted the admissibility of scientific evidence from general acceptance to reliability. The court outlined how a trial judges, the so-called gatekeeper could determine if scientific evidence was admissible. The Federal Rules of Evidence, Rule 702 Testimony by Expert Witnesses codifies the requirements. These are summarized as follows:

A witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if:

(a) The expert’s scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue. (b) The testimony is based on sufficient facts or data. (c) The testimony is the product of reliable principles and methods. (d) The expert has reliably applied the principles and methods to the facts of the case.

These criteria are the bases of questioning to be considered in reviewing lab reports, discovery requests and cross-examination. Whether the issue is an attempt to exclude evidence from the jury or questioning the government’s expert witness, these issues are the same. Do tests done on the evidence meet standard procedures and are the results reliable?

There are several elements about lab operations you may wish to consider to help you interpret forensic science lab reports. Usually, unless you have some experience with scientific reports you may feel lost. But don’t lose hope. Consider lab testing as a step-by-step process which involves the examiner, the laboratories’ operation, and the testing process. The following are some of the questions that may be asked of an expert witness about work done on a specific test procedure.

Qualification of the examiner:

What was the education and training of the examiner? Does he possess a college degree from an accredited college or university? Does he have an advanced degree? Does any of his academic training have any relationship to the testing he did in the case?

Does the examiner belong to any professional associations? Does s/he attend meetings? Has s/he presented scientific or case related presentations at meetings? Has s/he published in peer-reviewed journals? Has s/he authored any books or chapters in books concerning the material he is testifying about? Is s/he a member of a national body such as federally sponsored Scientific Working Groups?

What was the nature of his/her on-the-job training to conduct an analysis? Did s/he have to pass a competency examination? Are the results of these tests (as well as any other primary documents) available? Does the examiner take periodic proficiency tests? Are they bound or declared? Are there records of the results of the tests?

Does an external body certify the examiner? Is the certification body recognized8 as a legitimate institution? Did the examiner have to take an examination to be certified? Are results available? Are periodic proficiency tests required? Are results available? Has the examiner ever failed a proficiency test and if so, what happened? Did the examiner ever make an error in testing?

What tests were done on the evidence? Often reports on provide a conclusion, e.g. the white powder contained cocaine or the latent printed matched the suspects print. Knowing the nature of the tests done to reach a conclusion is important first step to provide follow up question. But this is only a start. For example, if gas chromatography/mass spectrometry was used to identify a substance, additional questions are suggested: how was the specimen prepared, was equipment working properly, etc.

Quality assurance in laboratory operations:

Does an approved accrediting body accredit the laboratory? Was the laboratory ever sanctioned for not being in compliance with accreditation standards?

Does the laboratory maintain a quality assurance program?

Are the tests used validated? Has the validation been documented? Were protocols for the test used in the case followed? Did the analyst receive training in the test protocols and was he tested to demonstrate that he is able to accurately conduct the tests?

Does the laboratory follow a code of ethics? Are employees thoroughly briefed on this code?

Are Corrective Action Reports issued to include any errors made in prior cases or proficiency tests? What steps are taken to retest cases and retrain the examiner?

Test results:

What do the test results mean? Are there limitations to the test? What factors may cause a positive test result? What factors may cause a negative test result? Did a second qualified examiner review the results? Did the reviewer agree with the primary examiner’s results? Are these documented?

Are case notes available for another expert to review?

Are records maintained on the laboratory instruments used for testing including? Maintenance logs, routine test results of known and blank samples, and instrument repair records? Was the instrument checked before and after the analysis?

Are copies of the procedures for specific tests applicable for the time the test was conducted (procedures may have been modified from time to time)?

Where primary standards are obtained and are record of these standards maintained?

Are records of reagents used kept and are reagent expiration dates noted in reports.

Conclusion:

It is not possible to provide for each and every situation. Laboratory testing can be complicated and for the non-scientists, an unfamiliar language is often in use. Nevertheless, the defense lawyer has a duty to his client to provide him with the best representation possible. Being conversant in forensic science practice, protocols and language is important for that duty to be met.

Up

1  Barry Fisher is the retired crime lab director of the Los Angeles County Sheriff’s Department. You may contact him at: bajfisher@earthlink.net This talk paper is the basis of a talk given at the April, 2013 CLE program of the National Association of Criminal Defense attorneys in Las Vegas, NV. You can also follow Barry on Twitter@barryajfisher
2  Saks, Michael J., and Jonathan J. Koehler. "The individualization fallacy in forensic science evidence." (2007).
3  There is research ongoing to determine the statistical likelihood of of some types of pattern evidence, e.g. fingerprints, but such numerical representations have let to be used widely. See, e.g. PNAS, Accuracy and reliability of forensic latent fingerprint decisions, Bradford T. Ulerya, R. Austin Hicklina, JoAnn Buscaglia, and Maria Antonia Roberts, May 10, 2011 vol. 108 no. 19.
4  National Research Council, Committee on Scientific Assessment of Bullet Lead Elemental Composition Comparison, Forensic Analysis: Weighing Bullet Lead Evidence 1 (2004)
5  See, e.g., Wrongful Convictions and Forensic Science: The Need to Regulate Crime Labs, Paul C. Giannelli, North Carolina Law Review, Vol. 86, p. 163, 2007-2008, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1083735
6  Frye v.United States 293 F. 1013 ( D.C.. Cir 1923)
7  Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993)
8  See the Forensic Specialties Accreditation Board http://thefsab.org/