Johns Hopkins Nursing Evidence-Based Practice
Appendix D
Evidence Level and Quality Guide
© 2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
Evidence Levels
Quality Ratings
Level I
Experimental study, randomized controlled trial
(RCT)
Explanatory mixed method design that includes
only a level I quaNtitative study
Systematic review of RCTs, with or without meta-
analysis
QuaNtitative Studies
A High quality: Consistent, generalizable results; sufficient sample size for the study design; adequate
control; definitive conclusions; consistent recommendations based on comprehensive literature review that
includes thorough reference to scientific evidence.
B Good quality: Reasonably consistent results; sufficient sample size for the study design; some control,
fairly definitive conclusions; reasonably consistent recommendations based on fairly comprehensive
literature review that includes some reference to scientific evidence.
C Low quality or major flaws: Little evidence with inconsistent results; insufficient sample size for the
study design; conclusions cannot be drawn.
QuaLitative Studies
No commonly agreed-on principles exist for judging the quality of quaLitative studies. It is a subjective
process based on the extent to which study data contributes to synthesis and how much information is known
about the researchers’ efforts to meet the appraisal criteria.
For meta-synthesis, there is preliminary agreement that quality assessments of individual studies should be
made before synthesis to screen out poor-quality studies
1
.
A/B High/Good quality is used for single studies and meta-syntheses
2
.
The report discusses efforts to enhance or evaluate the quality of the data and the overall inquiry in
sufficient detail; and it describes the specific techniques used to enhance the quality of the inquiry.
Evidence of some or all of the following is found in the report:
• Transparency: Describes how information was documented to justify decisions, how data were
reviewed by others, and how themes and categories were formulated.
• Diligence: Reads and rereads data to check interpretations; seeks opportunity to find multiple
sources to corroborate evidence.
• Verification: The process of checking, confirming, and ensuring methodologic coherence.
• Self-reflection and scrutiny: Being continuously aware of how a researcher’s experiences,
background, or prejudices might shape and bias analysis and interpretations.
• Participant-driven inquiry: Participants shape the scope and breadth of questions; analysis and
interpretation give voice to those who participated.
• Insightful interpretation: Data and knowledge are linked in meaningful ways to relevant literature.
C Low quality studies contribute little to the overall review of findings and have few, if any, of the features
listed for high/good quality.
Level II
Quasi-experimental study
Explanatory mixed method design that includes
only a level II quaNtitative study
Systematic review of a combination of RCTs and
quasi-experimental studies, or quasi-
experimental studies only, with or without meta-
analysis
Level III
Nonexperimental study
Systematic review of a combination of RCTs,
quasi-experimental and nonexperimental studies,
or nonexperimental studies only, with or without
meta-analysis
Exploratory, convergent, or multiphasic mixed
methods studies
Explanatory mixed method design that includes
only a level III quaNtitative study
QuaLitative study Meta-synthesis
Johns Hopkins Nursing Evidence-Based Practice
Appendix D
Evidence Level and Quality Guide
© 2017 The Johns Hopkins Hospital/ Johns Hopkins University School of Nursing
1 https://www.york.ac.uk/crd/SysRev/!SSL!/WebHelp/6_4_ASSESSMENT_OF_QUALITATIVE_RESEARCH.htm
2 Adapted from Polit & Beck (2017).
Evidence Levels
Quality Ratings
Level IV
Opinion of respected authorities and/or
nationally recognized expert committees or
consensus panels based on scientific evidence
Includes:
• Clinical practice guidelines
•
Consensus panels/position statements
A High quality: Material officially sponsored by a professional, public, or private organization or a government
agency; documentation of a systematic literature search strategy; consistent results with sufficient numbers of
well-designed studies; criteria-based evaluation of overall scientific strength and quality of included studies and
definitive conclusions; national expertise clearly evident; developed or revised within the past five years
B Good quality: Material officially sponsored by a professional, public, or private organization or a government
agency; reasonably thorough and appropriate systematic literature search strategy; reasonably consistent
results, sufficient numbers of well-designed studies; evaluation of strengths and limitations of included studies
with fairly definitive conclusions; national expertise clearly evident; developed or revised within the past five
years
C Low quality or major flaws: Material not sponsored by an official organization or agency; undefined, poorly
defined, or limited literature search strategy; no evaluation of strengths and limitations of included studies,
insufficient evidence with inconsistent results, conclusions cannot be drawn; not revised within the past five
years
Level V
Based on experiential and nonresearch evidence
Includes:
• Integrative reviews
• Literature reviews
• Quality improvement, program, or financial
evaluation
• Case reports
•
Opinion of nationally recognized expert(s)
based on experiential evidence
Organizational Experience (quality improvement, program or financial evaluation)
A High quality: Clear aims and objectives; consistent results across multiple settings; formal quality
improvement, financial, or program evaluation methods used; definitive conclusions; consistent
recommendations with thorough reference to scientific evidence
B Good quality: Clear aims and objectives; consistent results in a single setting; formal quality improvement,
financial, or program evaluation methods used; reasonably consistent recommendations with some reference to
scientific evidence
C Low quality or major flaws: Unclear or missing aims and objectives; inconsistent results; poorly defined
quality improvement, financial, or program evaluation methods; recommendations cannot be made
Integrative Review, Literature Review, Expert Opinion, Case Report, Community Standard,
Clinician Experience, Consumer Preference
A High quality: Expertise is clearly evident; draws definitive conclusions; provides scientific rationale; thought
leader(s) in the field
B Good quality: Expertise appears to be credible; draws fairly definitive conclusions; provides logical argument
for opinions
C Low quality or major flaws: Expertise is not discernable or is dubious; conclusions cannot be drawn
