Discussion
Assessment of adverse events that occur during therapy to determine
whether they represent an ADR versus the underlying disease of interest
is problematic. The development of tools to provide a more objective
approach rather than relying solely on clinical intuition would permit a
more evidence-based approach to care, notably for patients for whom the
nihilistic approach of simply avoiding the drug may place them at risk
or compromise their care. The Naranjo Scale was developing at the
University of Toronto in the late 1970’s to help address this issue
[14]. This scale was developed and validated in a cohort of patients
undergoing psychotherapy and the various domains on the scale reflect
this and the practice of psychopharmacology in the 1970s. As the
practice of medicine has evolved this has made practical implementation
of the scale more problematic; as an example, while the use of a placebo
(item 6 on the Naranjo Scale) was considered at the time the scale was
developed this would be unlikely to be a realistic option currently
[Fig 1]. This and other elements of the scale have made made the
Naranjo Scale less helpful in a clinical context, and consequently
groups have developed alternate approaches. One such approach, developed
by investigators at the University of Liverpool and Alder Hey Children’s
Hospital, was the Liverpool Causality Assessment, created based on data
from in-patients at Alder Hey [19].
This study was conducted to examine the value of screening suspected ADR
cases prior to any investigations using history and clinical
presentation alone in a ambulatory care setting and to compare the
Naranjo Scale to the Liverpool Causality Assessment. The two published
assessment tools: the older Naranjo Scale and newly developed tool by
the Liverpool group were correlated with well studied diagnostic tests
such as the allergy skin test, LTA, iPTA, RAST and oral challenge
[22-25].
Both assessment tools have a high sensitivity in predicting ADR cases
but poor specificity. Liverpool causality assessment tool (97.2% ±
2.4%) proved to be the more sensitive tool compared to its predecessor,
Naranjo scale (81.2% ± 5.69%). This is likely related to design; the
Naranjo Scale is a summative score of all the questions to determine the
likelihood of ADR while the Liverpool instrument has a flowchart format.
This significantly impacted the score distribution as all cases lost two
points on the Naranjo Scale due to following two reasons. First, a score
of 0 was given for the question addressing whether the reaction
reappeared when a placebo was given; as noted above placebo challenge
was not administered at the LHSC Clinical Pharmacology clinic. Secondly,
our study attempted to retrospectively score charts assuming no
diagnostic tests were performed for all cases, therefore a score of 0
was given for the last question addressing if there were any objective
evidence supporting ADR. This essentially made it impossible for any
cases to score “Definite” on Naranjo Scale as the maximum score any
case can receive was 8/10, which is equivalent to “Probable”. The
placebo question was rejected by the Liverpool group when their tool was
developed as it was not a common practice currently [19]. Moreover,
the Liverpool tool’s flowchart format made it possible for the cases to
achieve “Definite” without having an objective evidence to support
ADR.
The low specificities of the two tools were expected as the authors of
both Naranjo and Liverpool assessment tool designed the questions to
evaluate the sequence of events and reaction response to drug initiation
and/or cessation. The tools were not designed to examine ADR causality
directly or mechanistically but to screen suspected ADR cases in the
clinic setting in a timely manner prior to ordering more expensive, time
consuming diagnostic tests.
Predictive values for both Naranjo and Liverpool tools were similar in
comparison but both tools are more accurate with its negative results
than predicting true positives. Both tools had negative predictive value
of approximately 60% and positive predictive value of just over 30%.
This can be related back to the above explanation for low specificities
observed. The results from either tool are not diagnostically reliable
and should only be used to screen patients in conjunction to clinical
picture to aid in decision making to pursue further investigations.
This is the first study to compare the newer Liverpool Causality
Assessment Tool with its predecessor, Naranjo Scale, independently from
the developers of the two tools and in an ambulatory setting. Our study
population was also independently selected from the cases previously
identified in the papers describing the two tools[14, 19]. The tools
were applied in more than 500 suspected ADR cases and the large sample
size allowed this study to determine the sensitivities and specificities
with narrow 95% confidence intervals.
As previously described by Gallagher et al. , the Liverpool
assessment tool demonstrated a moderate inter-rater reliability with a
global kappa score of 0.48[19]. However in our study, only one rater
scored all charts and the potential for inter-rater difference was not a
concern. By having only one rater, all charts were scored with
consistent interpretation and application of the data.
It should be noted that only a handful number of cases that scored at
the polar ends on both Liverpool and Naranjo scales, either
“Unlikely/Doubtful” or “Definite”. This may have led to skewing of
the negative predictive values as only a limited number of samples were
available for statistical analysis. A future study identifying more
suspected ADR cases that score at the extreme ends would be a useful
addition to our findings, although in the case of the Naranjo Scale
given the items used to scoring findings of Definite are quite uncommon.
The Liverpool Causality Assessment tool has superior sensitivity
compared to the Naranjo Scale. The Liverpool tool has an observed
sensitivity of 97.2% ± 2.4% which suggests that it can be a useful
screening tool in the clinic setting. Nonetheless its specificity and
predictive values are low and should not be used as the sole method of
measuring the likelihood of ADR in suspected cases. This reserves
clinicians to consider the overall clinical impression and use the tool
as a supplementation when making ADR diagnosis or decisions to pursue
further investigations. Other diagnostic tests should be performed for
understanding the mechanism of ADR or to prove direct causality between
a drug and symptoms. Liverpool Causality Assessment Tool is useful for
screening ADR prior to pursuing diagnostic investigations, such as
during initial patient encounter.
The utility of ADR screening tools such as the Liverpool causality
assessment tool or Naranjo scale has not been verified prospectively in
an ambulatory or primary care setting. As well, the value of these tools
perceived by clinicians using them in clinical settings should be
evaluated by collecting data after applying the tools when suspecting
ADRs in patients suggesting that more research is needed to refine
causality instruments for the evaluation of potential ADRs.