Evaluating Nursing Research 

Objectives:  
	After the lecture and discussion, the student will be able to:
–	Identify the purpose of conducting a critique
–	Describe criteria of each step of the critique process
–	Discuss strengths and weaknesses of the report	

•	What is most important to communicate to the profession?
–	Depends on:
•	Guidelines provided by the journal
•	Stylist presentation 

Purpose of Evaluation
•	Constructive evaluation of scientific merit
–	Objective, unbiased, impartial appraisal
•	Strengths, limitations (weaknesses)
•	Worth of findings for clinical application to nursing practice (meaning and significance)
–	Leads to promoting sound evidence

CONDUCTING A RESEARCH CRITIQUE
•	Read and critique the entire study
•	Examine the organization and presentation
•	Examine the significance of the problem studied for nursing practice
•	Identify the strengths and weaknesses

CONDUCTING A RESEARCH CRITIQUE
•	Provide specific examples of strengths and weaknesses
•	Provide a rationale for critique
•	Suggest modifications for future studies
•	Discuss feasibility of replication
•	Discuss the usefulness of the findings for practice

SKILLS NEED
•	Critical reading skills for understanding
–	Preliminary – read abstract and skim
•	Look up definitions
–	Comprehension – researcher’s intent
•	Core variables, concepts (does article make sense)
–	Analysis – evaluate for scientific merit (use critiquing criteria)	
–	Synthesis – summarizing
•	How useful is the study for practice

COMPREHENSION
•	Understand terms and concepts
•	Identify steps in research process

•	Review abstract
•	Read entire study
•	Examine references
ANALYSIS
•	Critique of the logical links connecting one study element with another
–	Problem/purpose
–	Review of literature
–	Hypothesis/research question(s)
–	Design
–	Methods
–	Data Analysis
SYNTHESIS
•	Examine the meaning of the findings
•	Examine the significance of the findings
•	Summary of the qualities of the study

The problem statement and purpose
•	What is the problem?
•	What is the purpose?
–	Does the problem or purpose statement express a relationship between two or more variables?
•	Is the population specified?
•	What is the significance of the problem (to nursing, clinical care)

Review of the literature
•	What are the concepts (independent and dependent variables)?
•	Is there a conceptual/theoretical framework or model?
•	Are relationships between variables explicit? Are the relationship between variables and framework explicit?
•	Are sources primary or secondary?
•	Are they current (past 5 years)?
ROL
•	Are conceptual/operational definitions presented?
•	Is a summary of current knowledge provided?
•	Are the gaps or conflicts identified?

Hypothesis/question
•	What hypothesis(es) or questions are stated?
–	Are they appropriately stated?
•	Are the independent/dependent variables, research variables stated?
•	If a hypothesis, is it in the form of a research or null hypothesis?
•	Is a direction stated (positive, negative, etc.)

Hypothesis
•	Is the hypothesis testable as stated?

Sample
•	How was sample selected? 
•	What is the size? Is power analysis included? Are characteristics of sample stated?
•	Is method stated? Is is appropriate to the design?
•	Does sample reflect population?
•	To what population can the findings be generalized?

Research design
•	What type of design is stated?
–	Experimental, non-experimental, quasi-experimental
–	Is the rationale stated?
•	Does the study involved at treatment/intervention? If so, is it described?

Design
•	Is pilot data included?

Validity
•	Internal – are threats to internal validity addressed?  What might these be?
•	External – are threats to external validity addressed?  What might these be?
•	Consider extraneous, antecedent, intervening variables

Methods
•	What are the data-collection methods?
•	Are the procedures described?

Instruments
•	What type of instrument is used?
–	Physiologic
–	Observation
–	Interview
–	Questionnaire
–	Qualitative: descriptive
•	Is/are the instrument(s) well described (author given, etc.)
•	What is the level of measurement of each instrument?

Reliability/validity
•	Is reliability (type) reported?
•	What is the level of reliability? Is it acceptable?
•	Is validity (type) reported?

Analysis of data
•	What statistical analyses are reported?
•	What the purpose of each analysis described? (description, examination of differences, relationships, etc.)
•	Were the statistics (descriptive, inferential) appropriate to the level of measurement for each variable?

Analysis
•	Is the level of significance set for the study?
–	If so, what is it?
•	Are tables/figures, graphs used? If so, do they:
–	Supplement and economize the text
–	Have precise titles and headings
–	Do not report the text

Results/Conclusions
•	If hypothesis(es) testing was done, was/were they supported or not supported?
•	Are results interpreted in the context of the problem/purpose/study framework?
•	Were unexpected findings listed?
•	Are findings consistent with previous research?
•	Are limitations/weaknesses/problems identified by author(s)?
•	Is the relevance for nursing identified?

Conclusions
•	Are generalizations made?  If so, are they within the scope of findings or beyond?
•	Are recommendations for future research stated or implied?
•	Is a summary provided?  If so, is it a synthesis of findings and recommendations?

Legal/ethical
•	Was institutional review board approval sought and received?
•	Is there reference to informed consent?
•	How have the rights of subjects been protected?

Application/utilization
•	Does the study appear valid?
•	What are the missing elements of the study?
–	Do strengths outweigh weaknesses? 
•	How are findings applicable to nursing practice?
•	Is is possible to replicate the study based on the information provided?

Application
•	Does the title accurately reflect the study?
•	Are there other studies with similar/dissimilar findings?

CRITIQUE OF QUALITATIVE RESEARCH

•	Is the phenomenon of interest clearly identified?
•	Why is it important to study the phenomenon from a qualitative perspective?
•	Are the philosophical underpinnings described?

•	Is the purpose explicit?
•	Is the significance to nursing implicit or explicitly stated?
•	Is the method (process) for data collection described?
–	If so, it is appropriate to address the phenomenon?

•	Is the sampling procedure described (purposive)?
•	Is the data collection focused on the human experience?
•	Are data collection strategies given (interview, field notes, observation)?
•	Is protection of human rights addressed?

•	Is data saturation described?
•	What analytic procedures were used?
•	Are credibility, auditability, fittingness described?
•	Are the findings presented within a context?
•	Is the reader able to apprehend the essence of the experience?

•	Do the conclusions reflect the study findings?
•	Is the significance of the study to nursing made explicit?

Summary
•	Purpose is to examine the quality of the research report and the potential usefulness of findings for application to nursing practice

Main areas for critique
•	Problem and purpose
•	Review of literature
–	Theory/conceptual framework
–	Definitions of variables/concepts
•	Hypotheses/research questions
•	Sample
•	Research design

Main areas
–	Internal validity
–	External validity
•	Research approach
–	Methods
–	Legal-ethical issues
–	Instruments
–	Reliability/validity

Main areas
•	Analysis of data
•	Conclusions/recommendations/implications
•	Application to nursing