NCSTL

U.S. House of Representatives NAS Report Hearing

March 10, 2009 - Subcommittee on Technology & Innovation Hearing on Strengthening Forensic Science in the United States: The Role of the National Institute of Standards and Technology Committee on Science

Chairman: David Wu (OR)

Representatives that posed questions: Adrian Smith (NE), Paul Broun (GA), Donna Edwards (MD)

Witness Panel: Mr. Pete Marone, Ms. Carol Henderson, Mr. John Hicks, Mr. Peter Neufeld, and Dr. J.C. Upshaw Downs

Purpose: To review the scientific and technical issues raised by the National Academies (NAS) report Strengthening Forensic Science in the United States: A Path Forward, including those related to accuracy, standards, reliability, validity and the role of the National Institute of Standards and Technology (NIST).

Chairman Wu's Opening Remarks:

Although the NAS Report recommended creating an entirely new department to govern forensics issues and work with NIST, it may be economically prudent to build upon and improve existing federal capabilities rather than trying to create a whole new government structure.
The speaker fully supports the end goal of the report to improve forensic science in the United States.
The television show Crime Scene Investigation has raised public awareness and expectations of the role of forensic science; however its depiction of forensics is far different than the reality. This hearing is thus a "first step" in moving from entertainment to reality.

Representative Smith's Opening Statement:

Forensics is a key factor in our justice system, and its importance has only risen over the past years.
Continued improvement is needed, as suggested by the report.
The core finding of the report, that many forensic disciplines are in need of scientific validation, is serious and requires full and immediate attention, BUT it does not mean that these disciplines are unreliable.

Representative Smith introduced a letter from NDAA into record without objection.

Witnesses introduced and identified. Each witness shared their prepared statements, and then answered questions posed by the subcommittee panel.

Statement of Pete Marone:

The Senate Appropriations Committee, with encouragement from the crime lab community, requested that the National Institute of Justice sponsor a study of forensics because – until this report - the federal government has primarily focused on the single discipline of DNA.
The challenges in forensics that the NAS report identified can be categorized in four classes: resources, research, standardization, and education.

RESOURCES: State and local Crime Labs and the Medical Examiner community have not been receiving the funds they need even though their case loads are exponentially increasing because (1) States are in a fiscal crisis and (2) Federal government has focused overwhelmingly on DNA, which is not the largest caseload.
RESEARCH: Certain forensic disciplines, especially those that are experience-based rather than based on biological or chemical analysis, need further research to better establish the underlying validation for (1) their methodologies and (2) the basis of statements about reliability and precision. Research is also needed on the effects of context and examiner bias.
STANDARDIZATION & EDUCATION: Though most in the forensic science community have started to move towards accreditation, there are still concerns about the lack of mandatory requirements for professional certification and laboratory accreditation, as well as the variability in the way forensic science results are reported in courts.

There are several indications that NIST does not have expertise to play the governance role that forensic science needs. It does not offer the "complete package." The strongest reason for establishing an entirely new independent entity, as called for by the report, is that it would be created solely according to the report's vision, thus avoiding bureaucracy or biases caused by the existing organization's prior knowledge and experience. The issue requires new, fresh thinking.

Statement of Carol Henderson:

The "forensic overhaul" called for by the NAS report requires the collaboration of all stakeholders from scientific, technological and legal fields.
To make the best use of available resources and create sound and lasting systems a three step approach should be used.

FIRST: Immediately address the need for scientific standards with existing Federal resources, specifically those available from NIST. NIST made significant advances in forensics in regards to DNA analysis, the AFIS system, and firearm comparisons.
SECOND: In the interim create an organized, focused program to carefully analyze forensic policy issues, including: moving forensics outside of the law enforcement community, accreditation and certification requirements, and strategic planning for research and education efforts. Note that education efforts should be directed at attorneys and judges, as well as forensic practitioners. Note also that there are few to no papers on forensic science policy, and no forensic think tank akin to that of the Aspen Institute. There needs to be a bridge, not a band aid, built to address these issues.
THIRD: Eventually achieve the long-term goal of creating the National Institute of Forensic Science (NIFS) as envisioned by the NAS Committee, keeping in mind the length of requisite planning and consultative processes.

The forensic community needs to challenge itself and respond to the voices of all its stakeholders, especially the legal population.
There is great hope for the future of forensic science and the NAS report has been welcomed by forensic associations such as the AAFS.
Forensics faces a difficult but achievable goal that can be attained through clear objectives and strategic planning, much like America's race to the moon.
Identifying innovative approaches comes not just from infrastructure but also from a profession's culture, and it is a key strategic issue as is open communication.

Statement of John Hicks:

The recommendations found in the NAS report fall into four categories: (1) methods development and standardization; (2) laboratory accreditation and quality assurance; (3) research and training; and (4) resource needs.
DNA technology has already benefitted from congressionally authorized programs. Non-DNA forensics received similar congressional support with the passage of the Paul Coverdell Forensic Science Improvement Program in 2000. These agendas should receive continued support.
DNA's "robust" underlying research and method validation outpaced that of other forensic disciplines because congressional support allowed scores of scientists and academic researchers to assess, validate, and optimize DNA testing.
There is now concern over the lack of systematic research to validate premises and techniques for forensic disciplines; particularly that rely on pattern recognition such as fingerprints, tool marks and handwriting. Thus such disciplines should be subject to empirical studies, which can then become the basis for appropriate standards.
NIST is particularly well suited for the aforementioned task, as exampled by its work with individual markers used in DNA identification, large data systems such as the Automated Fingerprint Identification System (AFIS) and the National Integrated Ballistics Identification Network (NIBIN), and laboratory reference materials.
Expanding NIST's role is the most effective and efficient way to bring about needed improvements in the forensic science community and to assure appropriate focus in the development of new technology opportunities now and in the future.

Statement of J.C. Upshaw Downs:

This speaker's remarks primarily address the discipline of Forensic Pathology and medicolegal death investigation.
The focus of the entire “status of forensics” comes down to uniformity and best practices.
The National Association of Medical Examiners (NAME) developed and implemented Forensic Autopsy Performance Standards in 2006 so that technical aspects of the performance of the forensic autopsy were consistent from jurisdiction to jurisdiction.
In general, the scientific underpinnings of forensic disciplines are there - but they will benefit from a more formal structured review.
In addition to the obvious impact Forensic Pathology has on the justice system, Medical Examiners have important and sometimes under-recognized duties in public health, medical research, and homeland security/mass disaster preparedness, such the recognition of potentially infectious diseases.
Quality is a goal, not a destination, and to this end the NAS report demands certification and accreditation.
Both Medical Examiners, as physicians, and lab systems should have certain basic credentials – and limiting federal funds to offices that have achieved, or are actively pursuing, accreditation is a good incentive to encourage compliance.
Forensic pathologists are strong proponents of education and research and as physicians are required to comply with continuing education requirements.
Specialty scientific journals present the latest advances in the field, however ongoing research is both sorely needed and underfunded.
The proposal of nationwide Medical Examiner system acknowledges that the mission of medicolegal death investigation has changed over the years and signifies a move towards improving the existing system; however to advance the field in this manner one must also understand underlying issues such as the difference between Coroners and Medical Examiners.
Resources, which can be expensive, are needed to (1) attract new people to the field, (2) train forensic pathologists, (3) solve staffing shortages, and (4) convert to a nationwide Medical Examiner system. However the report did not address "how to pay" for these needs.
Independence is also an important consideration inasmuch as death investigation often falls under the umbrella of law enforcement.
An important distinction should be made between conventional science and forensic science because in the latter case the data must be available for courtroom presentation.
Though it may be tempting to find a quick fix to the issue of oversight for the forensic sciences by placing it under existing entity, such action is not in line with the spirit of the report which recognized that the stated goals will take time and consultation to achieve.
The creation of a new, independent governing entity would elude the dangers of an institutionalized mindset, and could be based on models such as Australia's National Institute of Forensic Science (NIFS).
NIST's continued efforts to improve forensics will remain beneficial, though NIST does lack an established history with regard to the complexities and intricacies of the interaction of law enforcement, legal, and governmental issues so vital to forensics. There is also likely to be a lack of "buy in" from forensic practitioners if NIST were to become more involved in accreditation and certification efforts. Such efforts have been undertaken by others and there is no need to reinvent the well.

Statement of Peter Neufeld:

233 Americans have been exonerated through DNA analysis thanks to the work of the Innocence Project.
Unvalidated / improper forensics contributed to approximately 50% of the aforementioned wrongful convictions, and there is a notable contrast between the careful evolution and thorough regulation of DNA testing, and that of non-DNA forensics which lack similar validation and reproducibility.
Such non-DNA forensic disciplines were generated in the context of law enforcement and include (1) Hair Comparisons, as exampled by the exoneration of Jimmy Bromgard, (2) Bite Mark Comparisons, as exampled by the exoneration of Kennedy Brewer, (3) Fingerprint Comparisons, as exampled by the exoneration of Brandon Mayfield, (4) Shoe Print Comparisons, and (5) Fiber Comparisons, and (6) Other Pattern/Impression Evidence. The Innocence Project agrees with the NAS report regarding what is needed in terms of validation for these disciplines.
Forensic shortcomings have real world consequences and are not limited to one analyst, lab or jurisdiction. The 233 exonerations are just the tip of the iceberg.
Many non-DNA forensics are used solely for investigation, prosecution and conviction in a law enforcement context and thus have not been subjected to the rigors of the scientific process.
The need for reform is urgent since non-DNA forensics continue to be used daily in investigations, prosecutions and convictions across the country despite their potential to mislead police, prosecutors, judges and juries away from the real criminal perpetrators.
The Innocence Project whole-heartedly supports the primary recommendation of the report to create a federal NIFS and suggest that the NIFS should:

focus on research, assessment of validity and reliability, and also quality assurance, accreditation, and certification,
be a non-partisan, independent agency,
coordinate all existing and future federal functions, programs, and research related to forensics,
have mandatory oversight as well as enforcement mechanisms,
be a permanent program in order to ensure ongoing evaluation, review and leadership, and
receive adequate resources from Congress.
The Innocence Project also supports the report's recommendation that NIST would make a sensible partner for setting standards.

Table of cases listing DNA exoneration cases where unvalidated / improper forensic science contributed to the underlying wrongful conviction is attached to this statement.

Questions and Topics of Discussion posed to, and answered by, the witness panel:

Chairman Wu: Is there disagreement among witness about current state of forensic science? This issue speaks to that of whether a practice is "evidence-based." Could the panel please discuss concerns over whether there is a sound scientific basis for many of the forensic testing that we rely on? Do we need to do something more dramatic?

Neufeld: It was the NAS report, which was created by some of the best minds in science, that said such disciplines were lacking, and we can't ignore that.
Downs: The model of evidenced-based medicine is a good one. There is good science behind the testing that we do, we just go ahead of it and never went back to check it because it came through law enforcement and teh courts, which never asked for such basis.

Chairman Wu: What kind of difference would it make if we subjected these forensics to more scientific and analytical rigor rather just have an experience-based basis for them?

Downs: There is not a set number of points of identification required for a fingerprint "match" – that is that has been based on experience in the past. But if we were to apply standards used in medicine we would have a set metric, and that is what we are trying to get to.
Marone: We need additional research. Prior research has been done by the practitioners. Firearms comparisons provide good examples of this. But the studies were not broad enough, weren't statistically analyzed, and didn't answer all the related questions. Such studies can and should be improved upon to get the requested scientific basis. Note that there is no indication that the prior research in these fields is improper; nor is there any indication that improving the studies cannot be done. We can't assume something is empirically wrong just because there is no validation.
Neufeld: (1) Burden to show science has sufficient basis falls on the proponent of the evidence. (2) The standards that have been used in forensics would not work in medicine – you can't have ten pathologists disagree over whether a cell is malignant or benign.

Wu's interjection: In his experience disagreement can occur in medicine; analysts over time become better at analyzing patterns. Though you would not to have 20 different conclusions, you can get, say, 2 very different opinions.

Neufeld continued: (3) there has never been freestanding research of forensics, and we need this to validate old studies and inspire new discoveries.

Representative Smith (NE): What stands out to you as the highest priority technical recommendation from the report? What will it cost?

Marone: There is not one, they are all level. If you can look at what can be achieved first (but note not easiest) the answer may be accreditation and certification. There are already good programs out there that address this. Accreditation in particular needs attention. Then education, and then resources. But we need them all and they can and should be done in parallel. Labs are swamped – and there is no quick fix. As for cost, it is unknown, though we can look at what has happened with DNA for some guidance. Be sure to look specifically at what money made it to the labs. He is hesitant to throw out a figure because then everyone will then run with it. Bottom line: it is large figure.
Henderson: We need a strategic plan to define our priorities. The report does not set forth a specific plan. Research though is critical, in part because forensics is multidisciplinary. We don’t know how much it will cost – but we should look at what is existing and work from there.
Downs: We have to validate pattern evidence because courts are relying on it now (that evidence is in process as we speak).
Hicks: Much of the information is there, it just needs to be reviewed. And we need to acknowledge the history of these forensic disciplines. Many have evolved in police departments, first from investigators and then scientists. NIST can be helpful here in the interim in terms of quickly looking at what data has been developed, identifying gaps, and filling them in.
Neufeld: It's been said there is scientific data out there; it just has not been collected. Using odontology as an example, one can conclude that "it won't be okay" and we need a single agency to coordinate and govern forensics. Once it is up and running then you can prioritize.

Representative Edwards: Would like to hear directly from NIST in the future about their role, and appreciates Neufeld's work. In terms of funding, what is the estimate of the amount of resources already being spent in forensic sciences across federal agencies? In terms of coordinating among these agencies, how can we look at the work ahead and build on our current capacities rather than create a separate federal agency?

Henderson: Forensics is done in / by NIST, NIJ, Dept of Homeland Security, and the DOD. There are many pockets out there, and they don't necessarily communicate well among themselves. We do need to identify what is available now, avoid duplicative efforts, and be cognizant of our economy. Then we need to see what it will take to get people to the table.

Representative Broun: In light of the fact that some say NIST cannot offer the full package, raise your hand if you propose a new national bureau. (Henderson, Marone, Neufeld, Downs – raised their hands. Hicks – did not raise his hand) Of the four that raised their hands, why can't we do this in the private sector or the states / let market forces drive this? Wouldn't this help maintain independence?

Hicks: You could turn it over to the private sector, which is where NIST could come in since they do work with the private sector. And as things develop we may see more areas where we could bring in the private sector.
Neufeld: Agrees that the private sector can play a big role in this, BUT whatever changes do happen need to occur upstream of the courtroom. Lawyers and attorneys don’t do the best job of separating our good and junk science.

Representative Broun: Raise your hand if you have read and studied the Constitution of the United States. (4 or 5 witnesses raised their hands.) Could you show me where in the Constitution it says we need to develop this new bureau?

Henderson: Neufeld may say that we should look in the shadows of a Constitutional amendment to find something that says we have the right to do this to protect people's rights.

Representative Broun: Unless you look at a perverted version of the Constitution you will NOT find authorization to create this new NIFS agency. As a scientist and doctor he is interested in finding the truth. However with a government-based governing body in charge of forensics there may not be the necessary freedom to do what is needed. This should be considered carefully. Also there is a lack of funding, so everyone should seriously consider the existing capabilities and future possible roles of NIST.
Chairman Wu: Considering the constitutional issue, most look to article 1, section 8 to get their authority. This issue of Constitutional authority rises frequently. And one must note there is no textual authority for NASA or the Air force in the Constitution either.
Representative Broun: In response, the government now seems to be working on a perverted view of the Constitution, and national defense is provided for in the Constitution. There is always debate in the scientific community over issues, and that debate is good. However such debate can be hindered by federal government oversight; and this oversight / creation of a new agency is unconstitutional.
Chairman Wu: There is shared concern over whether to create a new forensic agency. But let’s return to the core issue – where is the science behind forensic science? What difference does it make? In sum it seems as if DNA analysis was created independently and then eventually it found a forensic application, while non-DNA forensics was developed in the law enforcement context and – though there is some underlying scientific research – remains partly based on experience. Is this correct?

Hicks: Yes, that is exactly right.

Chairman Wu: What kind of difference would it make if we subjected these forensics to more scientific and analytical rigor rather just have an experience-based basis for them? Neufeld gave a graphic example of this issue by discussing DNA exonerations and the fact that about 100 real criminal perpetrators were loose on the streets while these people were wrongfully incarcerated.

Henderson: Only 2-4% of cases get to trial, so many of these issues go unchallenged.

Chairman Wu: Are most of the problems focused on the pattern evidence? Do you agree that things like toxicology analysis are more "settled" procedures with less doubt? Even where there is "more science" are there additional concerns about accuracy of procedures, certification of tests, and formats?

Yes, it's really the pattern evidence that received a lot of attention in this report, and you can't really get statistics in some of these areas.
Neufeld: Yes, Biggest problems are in the pattern disciplines, but there are other types of problems in forensics such as: lack of national standards for lab reports.
Downs: Reports do need to be understandable and there still needs to be standards for even "sound" sciences.
Henderson: Lawyers have little scientific background and thus previously did not challenge such forensics (fingerprints, toxicology). There are more challenges now as lawyers get educated in this field. For example fingerprints were challenge din a recent robbery case. So these fields that were developed in the context of law enforcement are only now facing request for scientific rigor because lawyers have started to challenge them. The rigor needs to be imposed in "traditionally accepted pattern evidence" – and that's what the report says.

Chairman Wu: Do we have problems when we ask question about whether a match is really a match in terms of fingerprints, etc.? In fields where there are actually guidelines, how many forensic tests actually meet those guidelines?

Hicks: It is experience based. There are problem with getting full evidence, a full fingerprint – that does not always happen. You do need a review / confirmation process, and errors can happen. SWGs do make relevant recommendations for these kinds of matching tests. He is reluctant to make a guess in response to the second question. Again NIST can play a role here.

Chairman Wu: There is a paradigm shift from science, where you don’t make a conclusion until the process has to get to a certain point, over to law, where there is a deadline for decisions.

Representative Smith: Are scientific progress and the criminal justice system keeping up with each other? If we applied today's practices to older cases, would we see progress?

Neufeld: Yes, in terms of cases resolved through DNA. But there are few cases where DNA will come into play. Many of the forensic disciplines that gave rise to the wrongful convictions are still practiced today in the same way that they were practiced 5-15 years ago. There was no initiative to make changes. Hair comparison is offered as an example. So in a way it is sill problem and it is not getting better in the courts.
Hicks: In hair comparison cases there was a disclaimer that it was not an absolute means of identification. Fiber cases – such as the Wayne Williams Atlanta murder case wherein several different fibers were involved - offer similar examples. Forensic evidence is not always found in full at crime scenes. Also forensics can be used to corroborate or invalidate eye witness testimony – it should always be looked at in the context of the whole case. Going back to hair analysis, today they may tend to use it only to identify which hairs would be good candidates for DNA analysis.
Neufeld: The NAS report specifically says there are no studies on the reliability of combining hair and DNA analysis. FBI analysts have offered on hair analysis statistics without basis. But the underlying issue is that if you say something "is more likely than not" you are still making a probabilistic statement, and you should not make such a statement unless you have supporting empirical data. The disclaimer is not enough.
Henderson: There have been cases before where a "match" was declared but nowadays that is not allowed. Going back to the issue of problematic forensic disciplines, there was an article in the Journal of Forensic Sciences (Peterson) on 20+ years of proficiency testing. It showed that in terms of (especially hair and fiber) testing examiners got it wrong 50% of the time.
Marone: Proficiency testing is designed to assess the competency of the specific examiner and the lab's operating process. Exampled in the drug chemist community. Part of the proficiency testing is to figure out why the person came to the wrong conclusion.
Henderson: the 50% incident is in terms of the proficiency testing, not in terms of evidence that made it to court. More proficiency testing is now being done than ever before. Things have improved, people are not perfect, and proficiency testing should be encouraged.
Neufeld: We use proficiency testing because there is no way to know whether the conclusion that evidence at a crime scene connects to an individual is correct, there is no control. Proficiency testing is a substitute. So you would want a national agency to come up with a good program of proficiency testing. There are 4 kinds of such testing now.

*brief webcast malfunction*

Henderson: Immediate action includes using NIST's capacities and evaluating what we need and what is already out there. Then we need interim action including evaluating strategic policy decisions and strategies. Then go t the long term action. It has to be a 3 step plan.
Chairman Wu believes this issue is important, but still doesn't fully grasp the role of bringing scientific vigor to bear on forensics which has been an experience-based field for awhile. For example, how will it impact conviction rates?

Chairman Wu: If you are going to ask for more resources, we want to know how much more? What are the costs, the measurable inputs? We need that before we can take meaningful action.

Marone: We need more people. How many? We don't know yet without assessment. Where are we going to get these people? How are we going to incentivize entering forensics considering there are no scholarship programs? It is a public service. It needs to start at that level.
Downs: In regards to medical examiners, we need to replace lay coroners with qualified, board certified forensic pathologists. The graduation rate is not there yet though, as noted by Chairman Wu. The numbers break down approximately as follows:
- 37 residents graduate in forensic pathology
- 400 practicing active medical examiners full time
- 800 practicing medical examiners are needed
- The cost to get this is $3-$5 per citizen, which equals between 1 and 2 billion dollars, and that excludes infrastructure concerns such as brining facilities up to code / CDC standards and there is also the issue of dealing with about 3,000 counties with coroner systems, and a good start would be helping those coroners improve.

Chairman Wu: Do we have the infrastructure in place to do the amount of research that Henderson recommends?

Henderson: There are some existing programs in universities and labs that can do the research, and if funded we could also look to other "hard science" facilities to perform research in forensics. The capacity is there, but will require money. There has been a movement to support research as seen in stipends that were given to forensic graduate students from the Forensic Science Foundation (they would eventually present their findings in a peer reviewed setting).

Chairman Wu: Would it be constructive to have departments of forensic science, or a federal agency, to handle forensic science - considering that forensics brings together so many sections of science? Would it be a "meaningful" addition? It seems that this is a field tailor-made for a research university structure where multiple disciplines are studies.

Henderson: There are existing institutions that provide funding for forensic research (NIJ, NIST). Perhaps better coordination among entities would work; creating a new entity is not always the best plan.
Neufeld: Echoes recommendation #3 in the NAS report. A coordinator is needed to decide how to tap into finding, etc. There is currently no strategic plan, and we need one. Some feel that NIJ is a poor location for locating forensic research because (1) there is the perception of conflicts of interest related to law enforcement, and (2) history shows that almost all of their prior research money has earmarked / tied to particular institutions. Having a "quarterback" that oversees forensic research would help push certain initiatives such as having the NIH assist forensic pathologists.

Chairman Wu: Does anyone disagree and feel that having a "quarterback" agency is not a good idea?

Hicks: The difficult practical aspects of implementation are a downside. There needs to be some connection between the academic and forensic community. There have been other federal initiatives to support problems in forensics, and perhaps those should continue. Also research can be directed at some of the gaps / issues mentioned earlier, and this can possibly be addressed by existing organizations.

Chairman Wu thanked the hearing's participants.

Hearing was adjourned.

Related Links