Background: From early in the COVID-19 pandemic, it was clear that vaccines would be necessary to stop the spread and severity of the virus. Many jurisdictions established COVID-vaccine-specific websites or web content to help inform their citizens about the vaccines. Although some criteria exist for analyzing health-related communication, there are few clear evaluative frameworks specifically for analyzing websites and even fewer specifically for vaccine information.

Objective: Based on existing criteria and best practices in public health communication, my study aims to develop more comprehensive criteria for analyzing official, public-facing, public health/government websites about vaccination and then apply those criteria to select COVID-19 vaccine websites.

Methods: I use content analysis methodology to develop vaccine website criteria and to evaluate websites. Using pre-existing frameworks and evaluative tools, incorporating current best practices in risk communication, and consulting with experts in the field, I developed a concise set of criteria for vaccine websites. I used these criteria to evaluate seven websites from Canada and the United States.

Results: From my analysis, I identified ten criteria (functionality, accessibility, authorship, purpose, funding source, privacy policy, content quality, currency, unbiased, references) and organized them into three themes (usable, transparent, and helpful) and three tiers of website navigation. My application of these criteria against seven websites showed that individual websites scored well in a few criteria. However, there is much room for improvement, particularly in ensuring that information is unbiased, being transparent about funding sources, and ensuring that websites are accessible.

Discussion and Conclusion: Fifteen recommendations are provided to support public health organizations in communicating with the public to help them make informed decisions to stay safe and healthy. These recommendations and the criteria on which they are based are intertwined and inextricable from their greater context. Consequently, these recommendations would be most effective when combined with other risk communication interventions.