Dr. Itiel Dror on Confirmation Bias
Dr. Dror contributed an article to Criminal Investigative Failures. The following is an article by him on confirmation bias with forensic evidence (which is supposed to be bias proof). Not so fast says Dror.
The article originally appeared in the Summer issue of the UK's Police Review (apologies for the formatting errors):
Biased brains
The way police officers and staff examine evidence is always at risk of
being far from impartial. Itiel Dror investigates how unconscious human
bias can inadvertently influence how scientific evidence is interpreted
How is it possible that highly skilled and
professional police forensic experts
make mistakes?
Examination of evidence by forensic examin-
ers, investigations by detectives, or considering
which action to take as police constables, all
have one thing in common: they rely heavily on
human thought processes and the brain.
Thought processes and perception are far
from perfect because the way people internally
understand, interpret, evaluate and judge infor-
mation highly depends on how thought proc-
esses are structured and brain mechanisms. Too
often people overemphasise the role of the in-
formation itself and neglect to understand the
crucial role that the human mind plays in under-
standing and interpreting this information.
People’s brains process information, but they
do not have the resources and capacity to deal
with all the information they receive. Therefore
they have evolved to take ‘short cuts’.
This means prioritising and selectively ex-
amining information, actively and dynamically
processing, and other mechanisms that form
the basis of intelligence. As people become
more experienced and highly skilled, they in-
creasingly develop and rely on these short cuts.
Examining one piece of evidence is used to
guide the search and processing of further in-
formation, piece after piece, in a way that they
all fit together to solve a puzzle. Knowing where
to look, what questions to ask, paying attention
to the important things and knowing where to
find them, is what distinguishes experts from
novices.
However, as one piece of information guides
people’s search and evaluation of subsequent
information, so they can also be led astray.
Once people have a belief or a hunch of what
the data may suggest, a theory or hypothesis,
this has powerful and profound effects on how
they perceive it, the way they process the in-
formation and the mental representations they
form of this data, how they evaluate and inter-
pret this information, and their judgements and
decision making.
Diminished objectivity
These effects can take many different forms and
influence people in a variety of ways. For exam-
ple, confirmation bias is when people notice and
give more weight to information that is consist-
ent and supports certain interpretations and not
others. Conversely, people do not notice, dis-
miss, or give less weight to other information
that does not fit (or even contradicts) the in-
terpretations they unconsciously support. Con-
firmation bias is only one example of the way
people think that diminish experts’ objectivity.
Escalation of commitment and momentum,
conformity and group think, prophecies that
fulfil themselves and wishful thinking are just
a few other psychological and cognitive phe-
nomena where experts unavoidably and uncon-
sciously can lose objectivity and be selective and
biased.
Myself and my research team of David Charl-
ton, Ailsa Peron, Ina Schmitz-Williams and Peter
Fraser-Mackenzie set up to experimentally ex-
amine effects of context on forensic experts. In
a series of studies undertaken over several years
we provided forensic evidence and examined
whether its evaluation by forensic experts was
solely based on the evidence itself.
For example, we would present fingerprints
from a crime scene, and observed if the conclu-
sion by forensic experts on whether they match
depended on if the suspect confessed to the
crime. We consistently found that such contex-
tual information affected the judgement and
decisions made by qualified and experienced
forensic examiners.
In a couple of these studies we presented
identical fingerprints to the same fingerprint ex-
perts, but provided a different external context
for them each time. We found that the context
in which evidence is presented, such as that
described above, can cause the same forensic
examiner to reach conflicting decisions on iden-
tical evidence. Our data and research findings
from these studies suggest that such influences
are most powerful when the quantity and qual-
ity of the evidence is low, and that these effects
occur at a subconscious level without the foren-
sic examiner being aware of them.
Practical evidence
Are confirmation bias and thought process influ-
ences an academic issue existing purely within
the theories of the human mind and brain?
Well, try to say this to Brandon Mayfield, an
Oregon attorney who was arrested for killing
191 people and injuring more than 1,800 in the
March 2004 Madrid train bombings.
Based on a latent fingerprint left at the crime
scene by the real Madrid bomber, Ouhnane
Daoud, FBI fingerprint experts positively identified
Mr Mayfield as the bomber. Even an independ-
ent forensic expert appointed to his defence team
concluded that it is a definite match.
In May that year, after Daoud was identified as
the owner of the fingerprints, the FBI acknowl-
edged the error and partly attributed it to confir-
mation bias. Mr Mayfield was released from cus-
tody and has since recieved an apology from the
federal goverment and was awared USD2 million
in compensation.
The issue of questionable objectivity and bias
when examining evidence within a leading con-
text is not limited to fingerprints or to investiga-
tions in the US.
For instance, CCTV evidence was recently used
in the Old Bailey in the case of Levi Bellfield who
was convicted of the murders of Amelie Dela-
grange in 2004 and Marsha McDonnell in 2003
in southwest London. He was also found guilty of
the attempted murder of Kate Sheedy in 2004.
One piece of evidence in in the attempted
murder charge relied on a CCTV image of a car.
However, there was only a single frame from the
CCTV footage that contained the registration
number, and this was of extremely low quality.
Initial examination of the image by detectives
(with minimal context) was able to conclude very
little information about the number plate.
However, when the image was presented to
forensic experts along with a suspect’s registra-
tion plate (for example, that of the accused), then
the forensic imagery examination of the CCTV
image was conducted within a potentially influ-
encing and biasing context.
The Met was eventually forced to admit in
court that detectives had failed to properly exam-
ine the CCTV footage and four officers were later
formally reprimanded by the force after a review
by the Independent Police Complaints Commis-
sion.
Possible solutions
I believe these examples show that there is no
question that forensic experts and police offic-
ers (like everyone else and like experts in other
domains) are susceptible to bias and other influ-
ences. So what can be done about this? The solu-
tions to this problem, both in the forensic domain
as well as in the larger context of policing, is two-
fold.
The first solution is the development and im-
plementation of best practice in the field. An
example of this is for forensic experts to try to
examine evidence without potentially biasing in-
formation being given to them.
Best practice needs to be scientifically based
and validated by experts in thought processes
and not by forensic experts.
When this is not possible (which does happen
due to operational requirements), the aim should
be to first examine the evidence without the con-
text, clearly documenting the more objective and
independent analysis, and only then to allow the
introduction of the additional contextual and po-
tentially biasing information.
Contextually biasing influences come in many
different forms. Another example of such context
and how it can be managed is the proactive steps
taken by Kevin Kershaw, the head of forensic
services at Greater Manchester Police who is cur-
rently on secondment to the National Policing Im-
provement Agency to work on these issues. He is
actively working to combat this bias and protects
his forensic examiners from being unduly influ-
enced by buffering them from the investigating
detectives.
The second solution is training. There is gener-
ally an alarming lack of training in this area. For
example, in the Levi Bellfield case, both CCTV im-
agery forensic experts stated under cross-exami-
nation in court that they acknowledge the exist-
ence of confirmation bias, but had no training in
this area.
The Fingerprint Society, Hampshire Constabu-
lary, and Greater Manchester Police are examples
of a professional body and forces who have pro-
vided some training in this area.
Forensic evidence is an integral and important
part of policing and the criminal justice system. It
is relied on more and more, and it is vital to make
sure that this, as with other police processes and
decision making, is as professional and objective
as possible.
Understanding the human brain and mind,
and the structure of thought processes, is vital
to ensure the highest quality in judgement and
performance.
Dr Itiel Dror is a senior lecturer in cognitive
neuroscience at the University of Southamp-
ton. He has worked with the US Air Force
and police forces across the world
Web subscribers can read related articles, including:
Surveillance – moving target (PPR, 28 March 2008)
Crime analysts (PR, 7 May 2008)
www.policereview.com
Biased brains
The way police officers and staff examine evidence is always at risk of
being far from impartial. Itiel Dror investigates how unconscious human
bias can inadvertently influence how scientific evidence is interpreted
How is it possible that highly skilled and
professional police forensic experts
make mistakes?
Examination of evidence by forensic examin-
ers, investigations by detectives, or considering
which action to take as police constables, all
have one thing in common: they rely heavily on
human thought processes and the brain.
Thought processes and perception are far
from perfect because the way people internally
understand, interpret, evaluate and judge infor-
mation highly depends on how thought proc-
esses are structured and brain mechanisms. Too
often people overemphasise the role of the in-
formation itself and neglect to understand the
crucial role that the human mind plays in under-
standing and interpreting this information.
People’s brains process information, but they
do not have the resources and capacity to deal
with all the information they receive. Therefore
they have evolved to take ‘short cuts’.
This means prioritising and selectively ex-
amining information, actively and dynamically
processing, and other mechanisms that form
the basis of intelligence. As people become
more experienced and highly skilled, they in-
creasingly develop and rely on these short cuts.
Examining one piece of evidence is used to
guide the search and processing of further in-
formation, piece after piece, in a way that they
all fit together to solve a puzzle. Knowing where
to look, what questions to ask, paying attention
to the important things and knowing where to
find them, is what distinguishes experts from
novices.
However, as one piece of information guides
people’s search and evaluation of subsequent
information, so they can also be led astray.
Once people have a belief or a hunch of what
the data may suggest, a theory or hypothesis,
this has powerful and profound effects on how
they perceive it, the way they process the in-
formation and the mental representations they
form of this data, how they evaluate and inter-
pret this information, and their judgements and
decision making.
Diminished objectivity
These effects can take many different forms and
influence people in a variety of ways. For exam-
ple, confirmation bias is when people notice and
give more weight to information that is consist-
ent and supports certain interpretations and not
others. Conversely, people do not notice, dis-
miss, or give less weight to other information
that does not fit (or even contradicts) the in-
terpretations they unconsciously support. Con-
firmation bias is only one example of the way
people think that diminish experts’ objectivity.
Escalation of commitment and momentum,
conformity and group think, prophecies that
fulfil themselves and wishful thinking are just
a few other psychological and cognitive phe-
nomena where experts unavoidably and uncon-
sciously can lose objectivity and be selective and
biased.
Myself and my research team of David Charl-
ton, Ailsa Peron, Ina Schmitz-Williams and Peter
Fraser-Mackenzie set up to experimentally ex-
amine effects of context on forensic experts. In
a series of studies undertaken over several years
we provided forensic evidence and examined
whether its evaluation by forensic experts was
solely based on the evidence itself.
For example, we would present fingerprints
from a crime scene, and observed if the conclu-
sion by forensic experts on whether they match
depended on if the suspect confessed to the
crime. We consistently found that such contex-
tual information affected the judgement and
decisions made by qualified and experienced
forensic examiners.
In a couple of these studies we presented
identical fingerprints to the same fingerprint ex-
perts, but provided a different external context
for them each time. We found that the context
in which evidence is presented, such as that
described above, can cause the same forensic
examiner to reach conflicting decisions on iden-
tical evidence. Our data and research findings
from these studies suggest that such influences
are most powerful when the quantity and qual-
ity of the evidence is low, and that these effects
occur at a subconscious level without the foren-
sic examiner being aware of them.
Practical evidence
Are confirmation bias and thought process influ-
ences an academic issue existing purely within
the theories of the human mind and brain?
Well, try to say this to Brandon Mayfield, an
Oregon attorney who was arrested for killing
191 people and injuring more than 1,800 in the
March 2004 Madrid train bombings.
Based on a latent fingerprint left at the crime
scene by the real Madrid bomber, Ouhnane
Daoud, FBI fingerprint experts positively identified
Mr Mayfield as the bomber. Even an independ-
ent forensic expert appointed to his defence team
concluded that it is a definite match.
In May that year, after Daoud was identified as
the owner of the fingerprints, the FBI acknowl-
edged the error and partly attributed it to confir-
mation bias. Mr Mayfield was released from cus-
tody and has since recieved an apology from the
federal goverment and was awared USD2 million
in compensation.
The issue of questionable objectivity and bias
when examining evidence within a leading con-
text is not limited to fingerprints or to investiga-
tions in the US.
For instance, CCTV evidence was recently used
in the Old Bailey in the case of Levi Bellfield who
was convicted of the murders of Amelie Dela-
grange in 2004 and Marsha McDonnell in 2003
in southwest London. He was also found guilty of
the attempted murder of Kate Sheedy in 2004.
One piece of evidence in in the attempted
murder charge relied on a CCTV image of a car.
However, there was only a single frame from the
CCTV footage that contained the registration
number, and this was of extremely low quality.
Initial examination of the image by detectives
(with minimal context) was able to conclude very
little information about the number plate.
However, when the image was presented to
forensic experts along with a suspect’s registra-
tion plate (for example, that of the accused), then
the forensic imagery examination of the CCTV
image was conducted within a potentially influ-
encing and biasing context.
The Met was eventually forced to admit in
court that detectives had failed to properly exam-
ine the CCTV footage and four officers were later
formally reprimanded by the force after a review
by the Independent Police Complaints Commis-
sion.
Possible solutions
I believe these examples show that there is no
question that forensic experts and police offic-
ers (like everyone else and like experts in other
domains) are susceptible to bias and other influ-
ences. So what can be done about this? The solu-
tions to this problem, both in the forensic domain
as well as in the larger context of policing, is two-
fold.
The first solution is the development and im-
plementation of best practice in the field. An
example of this is for forensic experts to try to
examine evidence without potentially biasing in-
formation being given to them.
Best practice needs to be scientifically based
and validated by experts in thought processes
and not by forensic experts.
When this is not possible (which does happen
due to operational requirements), the aim should
be to first examine the evidence without the con-
text, clearly documenting the more objective and
independent analysis, and only then to allow the
introduction of the additional contextual and po-
tentially biasing information.
Contextually biasing influences come in many
different forms. Another example of such context
and how it can be managed is the proactive steps
taken by Kevin Kershaw, the head of forensic
services at Greater Manchester Police who is cur-
rently on secondment to the National Policing Im-
provement Agency to work on these issues. He is
actively working to combat this bias and protects
his forensic examiners from being unduly influ-
enced by buffering them from the investigating
detectives.
The second solution is training. There is gener-
ally an alarming lack of training in this area. For
example, in the Levi Bellfield case, both CCTV im-
agery forensic experts stated under cross-exami-
nation in court that they acknowledge the exist-
ence of confirmation bias, but had no training in
this area.
The Fingerprint Society, Hampshire Constabu-
lary, and Greater Manchester Police are examples
of a professional body and forces who have pro-
vided some training in this area.
Forensic evidence is an integral and important
part of policing and the criminal justice system. It
is relied on more and more, and it is vital to make
sure that this, as with other police processes and
decision making, is as professional and objective
as possible.
Understanding the human brain and mind,
and the structure of thought processes, is vital
to ensure the highest quality in judgement and
performance.
Dr Itiel Dror is a senior lecturer in cognitive
neuroscience at the University of Southamp-
ton. He has worked with the US Air Force
and police forces across the world
Web subscribers can read related articles, including:
Surveillance – moving target (PPR, 28 March 2008)
Crime analysts (PR, 7 May 2008)
www.policereview.com
posted by John Allore @ 1/11/2009 09:37:00 AM
0 comments
0 Comments:
Post a Comment
<< Home