Procedures Handbook. Version 4.0. What Works Clearinghouse".

Download or read book Procedures Handbook. Version 4.0. What Works Clearinghouse". written by What Works Clearinghouse (ED) and published by . This book was released on 2017 with total page 76 pages. Available in PDF, EPUB and Kindle. Book excerpt: The What Works Clearinghouse (WWC) systematic review process is the basis of many of its products, enabling the WWC to use consistent, objective, and transparent standards and procedures in its reviews, while also ensuring comprehensive coverage of the relevant literature. The WWC systematic review process consists of five steps: (1) Developing the review protocol; (2) Identifying relevant literature; (3) Screening studies; (4) Reviewing studies; and (5) Reporting on findings. A separate "What Works Clearinghouse Standards Handbook" (ED577036) describes Step 4, including the standards used by the WWC to review studies and assign one of the following three study ratings indicating the credibility of evidence from the study: "Meets WWC Design Standards Without Reservations," "Meets WWC Design Standards With Reservations," or "Does Not Meet WWC Design Standards." Taken together, these two documents replace the single document used since March 2014, the "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)." Figure I.1 shows how the steps of the WWC systematic review process are divided between the two Handbooks. In general, this new "Procedures Handbook" contains the same procedures that were included in the "Procedures and Standards Handbook (Version 3.0)." However, in addition to changes to the organization, the following substantive updates were made: (1) The "Handbook" includes additional information that can be specified in review protocols. This additional information may be needed because of updates to the "WWC Standards Handbook" or to take into account methodological considerations that may vary by topic area; (2) The "Handbook" includes additional clarification of procedures. The additional clarification of the procedures is intended to support consistency across reviews, and includes new discussion of how the WWC defines studies and conducts its correction for multiple comparisons within studies; (3) The "Handbook" includes updated formulas for calculating statistical significance of findings. The formula for continuous outcomes includes a new small sample size adjustment, and a separate formula is provided for dichotomous outcomes; and (4) The "Handbook" includes procedures for reporting findings from randomized controlled trials that present complier average causal effects. A new appendix describes how the WWC reports findings and statistical significance from studies with complier average causal effect estimates to examine the effects of intervention participation rather than intervention assignment. The remainder of the document is organized as follows: Chapter II describes the steps that the WWC uses to develop a review protocol. Chapter III describes how the WWC identifies the relevant literature. Chapter IV describes the screening process to determine if a study is eligible for review, and Chapter V describes the procedures used to review eligible studies. Chapter VI describes how the WWC summarizes evidence of effectiveness. Organizational procedures used by the WWC to ensure an independent, systematic, and objective review are described in the appendices. As the WWC uses and applies the procedures in this "Procedures Handbook," reviewers may occasionally need additional guidance. If necessary, the WWC will produce guidance documents for reviewers to provide clarification and interpretation of procedures and support consistency across reviews. This WWC reviewer guidance will clarify how these procedures should be implemented in situations where the current "Procedures Handbook" is not sufficiently specific to ensure consistent reviews. As the WWC continues to refine and develop procedures, the "Procedures Handbook" will be revised to reflect these changes. [See ED544775 to review the previous version of this guide. ].