Informing Evaluation Practice through Research on Evaluation

This dissertation advances research on evaluation (RoE) through a trio of studies focusing on the role of context and the innovative use of Linguistic Inquiry and Word Count (LIWC) software in formative evaluation in a qualitative research project. The initial study maps out how evaluation context dimensions—evaluator, stakeholder, organizational/program, and historical/political—affect evaluation, providing a nuanced understanding of these impacts. Subsequent research demonstrates LIWC's potential to monitor and formatively evaluate interviewer effects in data collection using LIWC's summary variable (authenticity and emotional tone), revealing that interviewer-interviewee demographic alignment has no significant effect in this specific qualitative research's data collection process. The final paper broadens LIWC's application, employing all built-in variables to pinpoint linguistic indicators of data richness, thereby refining data collection techniques. Together, these investigations shed light on contextual influences in RoE and validate LIWC as a pivotal tool for evaluators to assess evaluation context and provide strategies to evaluate qualitative data collection efforts ethically and efficiently, advocating for informed and adaptive evaluation practices to enhance research quality.
Key Words: Research on evaluation (RoE), evaluation context, Linguistic Inquiry and Word Count (LIWC), formative evaluation, interviewer effect, data collection, data richness