Cross-site Evaluation Methodology
The cross-site evaluation, being led by RTI International and supported by the Merck Foundation, seeks to answer a comprehensive set of questions related to the Alliance initiative:
- How well did the programs engage their intended audiences?
- How well did the programs build on success, and what characteristics account for success?
- To what extent were the sites able to implement their planned programs?
- Were the programs able to affect important clinical and patient-reported outcomes?
- Do programs plan and implement for sustainability?
The evaluation is using a multi-method approach. It employs and draws on the strengths of both quantitative (e.g. survey research and clinical data analysis) and qualitative (e.g. semistructured and telephone interviews) techniques.
To conduct the evaluation RTI is providing technical assistance and obtaining data from the five grantees. The technical assistance includes working with the grantees to obtain needed IRB and HIPAA clearances, decide on patient-reported and cost measures, manage data collection, and help with other matters related to launching the evaluation.
RTI is conducting a qualitative process evaluation of program implementation. This involves in-person site visits, telephone interviews, and document reviews to determine what the grantees did and how they did it. RTI is also collecting available data related to patients’ level of engagement with the enhanced care processes offered. These data will enable direct assessment of how many patients are reached by the initiative, and the extent to which the initiative is implemented, and the success of various interventions.
RTI is also compiling and analyzing quantitative clinical and patient-reported data that the grantees collect.
- Clinical data include: HbA1c levels, blood pressure, lipid levels, height and weight to calculate BMI, and smoking status. It will be used to assess effects of the interventions on the key clinical indicators related to diabetes.
- Patient-reported data, in the form of survey measures of diabetes competence, quality of life, resources and supports for self-management, and diabetes self-care behaviors, aim to capture patients’ self-reported experiences to help explain changes in clinical outcome measures.
Finally, RTI will assess the cost-effectiveness of the interventions and work with the grantees to determine whether health care utilization data are feasible to collect and analyze.
Although the project has not concluded, early results are being analyzed and will be presented at national conferences. Abstracts for such presentations follow.