Related Information
Assessing Research Quality
The quality of social science and policy research can vary dramatically. Research Connections accepts all research and related documents that are disseminated in the field, without judging the quality of their design, methods, findings and general content. It is essential, therefore, that consumers of Research Connections research evaluate the quality of these studies.
Key Questions to Ask
Was the research peer reviewed?
Peer reviewed research studies have already been evaluated by experienced researchers with relevant expertise. Peer-reviewed research is usually of high quality. A research consumer, however, should still critically evaluate the study's methodology and conclusions. The Research Connections Web site indicates whether a study has been peer reviewed.
Can a study's quality be evaluated with the information provided?
Every study should include a description of the population of interest, an explanation of the process used to select study subjects, definitions of key variables and concepts, descriptive statistics for main variables, and a description of the analytic techniques. Research Connections users should be cautious when drawing conclusions from studies that do not provide sufficient information about these key research components.
Are there any potential threats to the study's validity?
A valid study answers research questions in a scientifically rigorous manner. Threats to a study's validity are found in three areas:
Internal Validity
To determine whether a research study has internal validity, a research consumer should ask whether changes in the outcome could be attributed to alternative explanations, which are not explored in the study. For example, a study may show that a new curriculum preceded a significant increase in children's reading comprehension. The study must rule out alternative explanations for the increase in reading comprehension, such as a new teacher, in order to attribute the increase in reading comprehension to the new curriculum. Studies that specifically explain how alternative explanations were ruled out are more likely to have internal validity.External Validity
To assess whether a study has external validity, a research consumer should ask whether the findings apply to individuals whose place, times, and circumstances differ from those of study participants. A study's external validity is closely related to the generalizability of the findings. For example, a research study shows that a new curriculum improved reading comprehension of third-grade children in Iowa. As a research consumer, you want to ask whether this new curriculum may also be effective with third graders in New York or with children in other elementary grades. Studies that randomly select participants from the most diverse and representative populations are more likely to have external validity.Construct Validity
To assess whether a study has construct validity, a research consumer should ask whether the study has adequately measured the key concepts in the study. For example, a study of reading comprehension should present convincing evidence that reading tests do indeed measure reading comprehension. Studies that use measures that have been independently validated in prior studies are more likely to have construct validity.
Research Assessment Tools
The purpose of the quantitative and qualitative research assessment tools is to provide Research Connections users with a quick and simple means to evaluate the quality of research studies on our site. The Research Connections research assessment tools describe the information that should be available in study reports and the key features of high quality study design.
Quantitative Research Assessment Tool (PDF 46K)
Qualitative Research Assessment Tool (PDF 62K)
Additional Resources for Assessing Research
Research Connections provides links to the following resources as a convenience. The inclusion of any link does not imply endorsement in any way. Research Connections does not control the content of information found in any of the research organization links and Research Connections is not responsible for the contents or the privacy practices of any linked site or any link contained in a linked site. If you feel that information found via a link from www.childcareresearch.org is offensive or counterproductive to the mission of Research Connections, please contact webmaster@childcareresearch.org.
Online Resources
Early childhood program evaluations: A decision-maker's guide
Research Connections Web Site
Early childhood assessment: Why, what, and how?
Research Connections Web Site
Understanding Validity
William M.K. Trochim,
Research Methods Knowledge Base
Understanding and Evaluating Education Research
Education
Commission of the States (ECS) and Mid-continent Research for Education and Learning (McREL)
Offline Resources
Black, T. R. 2000. Understanding Social Science Research. Thousand Oaks, CA: Sage Publications.
Girden, E. R. 2001. Evaluating Research Articles: From Start to Finish. Thousand Oaks, CA: Sage.
Stern, P. C. 1979. Evaluating Social Science Research. New York: Oxford University Press.
 |
 |
 |
 |
 |