Statistical models for multilevel data with “Don’t know” category: implication for program evaluation
| dc.contributor.author | Huang, Beili | |
| dc.contributor.examiningcommittee | Shaw, Souradet (Community Health Sciences) | en_US |
| dc.contributor.examiningcommittee | Chen, Jieying (Business Administration) | en_US |
| dc.contributor.supervisor | Jiang, Depeng | |
| dc.date.accessioned | 2023-05-24T13:29:52Z | |
| dc.date.available | 2023-05-24T13:29:52Z | |
| dc.date.copyright | 2023-05-23 | |
| dc.date.issued | 2023-05-23 | |
| dc.date.submitted | 2023-05-19T23:05:39Z | en_US |
| dc.date.submitted | 2023-05-23T18:28:07Z | en_US |
| dc.degree.discipline | Community Health Sciences | en_US |
| dc.degree.level | Master of Science (M.Sc.) | en_US |
| dc.description.abstract | Background: “Don’t know (DK)” category has been increasingly used in surveys of longitudinal research. This creates unique challenges in data analysis and program evaluation. Strategies applying missing data methods may lead to biased and inaccurate estimations and lose valuable information. Objectives: (i) To illustrate advantages of the proposed two-part mixed effects model over other methods for longitudinal outcomes with DKs through simulation; (ii) to apply the proposed model to a mental health program (Project 11) to evaluate the program effects on participants’ awareness and level of connectedness. Methods: We applied a two-part mixed effects model for longitudinal outcome containing DKs. A simulation study was designed to illustrate the advantages of our proposed model over other methods, where different conditions including sample size, DK proportion, correlation strength between DKs and non-DK responses were considered under different DK mechanisms. We also compared the proposed model with other approaches by applying them to a mental health program (Project 11). In the data analysis of Project 11, we further extended the two-part model to account for within-cluster correlations among students within schools and to explore gender differences in program effects. Results: The proposed two-part mixed effects model outperformed other methods (i.e., CCA, SI, and ML) in estimating both intervention and random effects under all DK mechanisms. In contrast, methods disregarding DKs as missing experienced issues in at least some scenarios. Application of the two-part model to Project 11 data suggested significant intervention effects on improving the connectedness among boys (β ̂_11: -0.071, p = 0.049), whereas no significant improvements were observed among girls. Significant correlations were also found between the likelihood of DKs and connectedness level at both student level and school level. Conclusions: The proposed two-part mixed effects model is highly recommended for analyzing data with DK responses, based on the results of both simulations and empirical data analysis. Missing data techniques should be avoided due to potentially biased and/or imprecise estimates and the loss of information conveyed by DK responses. | en_US |
| dc.description.note | October 2023 | en_US |
| dc.identifier.uri | http://hdl.handle.net/1993/37351 | |
| dc.language.iso | eng | en_US |
| dc.rights | open access | en_US |
| dc.subject | “Don’t know (DK)” response | en_US |
| dc.subject | Two-part mixed effects model | en_US |
| dc.subject | Longitudinal data | en_US |
| dc.subject | Connectedness | en_US |
| dc.subject | Mental health | en_US |
| dc.subject | Program evaluation | en_US |
| dc.title | Statistical models for multilevel data with “Don’t know” category: implication for program evaluation | en_US |
| dc.type | master thesis | en_US |
| local.subject.manitoba | yes | en_US |
| oaire.awardTitle | The VADA Program (Visual and Automated Disease Analytics Graduate Training Program) | en_US |
| project.funder.identifier | https://doi.org/10.13039/501100000038 | en_US |
| project.funder.name | Natural Sciences and Engineering Research Council of Canada | en_US |