Challenges to Adolescent HPV Vaccination and Implementation of Evidence-Based Interventions to Promote Vaccine Uptake During the COVID-19 Pandemic: “HPV Is Probably Not at the Top of Our List”

Introduction The COVID-19 pandemic has prevented many adolescents from receiving their vaccines, including the human papillomavirus (HPV) vaccine, on time. However, little is known about the impact of the pandemic on implementation of clinic-level evidence-based interventions (EBIs) that help to improve HPV vaccine uptake. In this qualitative study, we explored the pandemic’s impact on EBI implementation and HPV vaccine delivery. Methods During August–November 2020, we interviewed clinic managers in a rural, midwestern state about their experiences implementing EBIs for HPV vaccination during the COVID-19 pandemic. We used a multipronged sampling approach with both stratified and purposive sampling to recruit participants from Vaccines for Children clinics. We then conducted a thematic analysis of transcripts. Results In interviews (N = 18), 2 primary themes emerged: decreased opportunities for HPV vaccination and disruption to HPV-related implementation work. Most participants reported decreases in opportunities to vaccinate caused by structural changes in how they delivered care (eg, switched to telehealth visits) and patient fear of exposure to COVID-19. Disruptions to EBI implementation were primarily due to logistical challenges (eg, decreases in staffing) and shifting priorities. Conclusion During the pandemic, clinics struggled to provide routine care, and as a result, many adolescents missed HPV vaccinations. To ensure these adolescents do not fall behind on this vaccine series, providers and researchers will need to recommit to EBI implementation and use existing strategies to promote vaccination. In the long term, improvements are needed to make EBI implementation more resilient to ensure that progress does not come to a halt in future pandemic events.


Introduction
The COVID-19 pandemic has prevented many adolescents from receiving their vaccines, including the human papillomavirus (HPV) vaccine, on time. However, little is known about the impact of the pandemic on implementation of clinic-level evidencebased interventions (EBIs) that help to improve HPV vaccine uptake. In this qualitative study, we explored the pandemic's impact on EBI implementation and HPV vaccine delivery.

Methods
During August-November 2020, we interviewed clinic managers in a rural, midwestern state about their experiences implementing EBIs for HPV vaccination during the COVID-19 pandemic. We used a multipronged sampling approach with both stratified and purposive sampling to recruit participants from Vaccines for Children clinics. We then conducted a thematic analysis of transcripts.

Results
In interviews (N = 18), 2 primary themes emerged: decreased opportunities for HPV vaccination and disruption to HPV-related implementation work. Most participants reported decreases in opportunities to vaccinate caused by structural changes in how they delivered care (eg, switched to telehealth visits) and patient fear of exposure to COVID-19. Disruptions to EBI implementation were primarily due to logistical challenges (eg, decreases in staffing) and shifting priorities.

Conclusion
During the pandemic, clinics struggled to provide routine care, and as a result, many adolescents missed HPV vaccinations. To ensure these adolescents do not fall behind on this vaccine series, providers and researchers will need to recommit to EBI implementation and use existing strategies to promote vaccination. In the long term, improvements are needed to make EBI implementa-Introduction Since March 2020, the pandemic caused by SARS-CoV-2 has affected nearly all aspects of daily life, particularly in the ways in which health care is provided. Because of fear of coming into clinics and the preference for telehealth appointments, one area that has been especially affected is pediatric and adolescent immunization (1,2). Before the pandemic, data from the 2019 National Immunization Survey-Teen showed that only 54% of US adolescents aged 13 to 17 were up to date with the human papillomavirus (HPV) vaccine series (3). An analysis to estimate the impact of the pandemic on HPV vaccination found that vaccination rates were 75% lower during the pandemic compared with prior periods, and statistical modeling showed that these lower rates could lead to increases in the incidence of genital warts, cervical intraepithelial neoplasia, and other HPV-related cancers if adolescents do not catch up on the HPV vaccine (4). Adolescence is a critical period for HPV prevention. The Advisory Committee for Immunization Practices recommends administration of 2 doses of HPV vaccine for children and adolescents aged 9 to 14 years (5). Initiating administration early in adolescence is linked to higher rates of on-time completion (6) and increased effectiveness (7,8). Therefore, understanding the extent to which the pandemic has affected adolescents' ability to get vaccinated is imperative.
Challenges in vaccinating adolescents during the last decade have led researchers and quality improvement (QI) staff to develop evidence-based interventions (EBIs) and strategies to assist clinical staff in increasing HPV vaccination. For example, commonly used EBIs include reminder/recall systems, standing orders for HPV vaccination, and provider assessment and feedback (9,10). Because of the complexity of these EBIs, a substantial amount of work happens "behind the scenes" in clinics. These implementation efforts to increase HPV vaccination rates (9-11) are often conducted independently or in collaboration with academic and community partners and led by administrative, non-patient-facing staff.
Nearly 2 years into the pandemic, little is known about the impact of the pandemic on implementation of these EBIs to encourage vaccinations or about the experience of clinics that continued to provide routine care during the pandemic. Our attention now needs to turn to these lesser studied impacts on clinic practices that have implications for future health outcomes of adolescents. The aim of this qualitative study was to explore the experiences of clinics that continued to provide routine care during the pandemic and the impact of the pandemic on ongoing implementation efforts to promote HPV vaccination.

Methods
This study was part of a larger project using the Consolidated Framework for Implementation Research (CFIR; https:// cfirguide.org/) to understand barriers and facilitators to EBI implementation focused on HPV vaccination in clinics integrated or affiliated with large health care systems in Iowa. However, only results related to the impact of COVID-19 are reported here. We conducted semistructured interviews with clinic managers or administrators working in Vaccines for Children (VFC) clinics in large health care systems in Iowa from August through November 2020. The University of Iowa Review Board determined that this study did not meet the criteria for human subjects research. All participants were provided with information about the study and its purpose, compensation, the voluntary nature of their participation, and the researcher's contact information. We offered a $25 gift card to all participants to thank them for their time.
We used multipronged sampling that included stratified sampling of VFC clinics in Iowa. First, researchers examined the list of VFC clinics in the state (N = 594) and excluded clinics that were either not pediatric/family practice clinics or not integrated or affiliated with a larger health care system, resulting in a final list of 305 clinics that met inclusion criteria. We stratified clinics by congressional district and rurality; a random sample of clinics (n = 5) was drawn from each stratum (n = 8); we repeated this process, ultimately recruiting from a sample of 80 clinics and completing 9 interviews. Up to 6 attempts were made to contact the clinic manager at each clinic, either by email or telephone. Common reasons for refusal were lack of time due to COVID-19 or not currently having a staff member in an administrative or management position. When this approach did not achieve thematic saturation in interviews, directed recruitment efforts were made through professional networks. Throughout the interview process transcripts were reviewed for thematic saturation, and recruitment ended when it was determined saturation had been reached.
We adapted the interview guide from the CFIR (12). In addition to questions to address the CFIR constructs, we included questions to address the impact of COVID-19 on HPV vaccination delivery and on implementation of EBIs for HPV vaccination. Before the interview, we sent all participants a brief survey to collect demographic information about them and their clinic. All interviews were conducted by the first author (G.R.) via telephone and audio recorded. A third-party service was used for verbatim transcription.
We generated frequencies and appropriate descriptive statistics for survey items capturing information on participant and clinic characteristics. To analyze the interviews, we conducted thematic analysis (13) that explored the impact of COVID-19. In the first round of coding, we used Nvivo version 12 (QSR International) to code transcripts and created a code to identify information on the pandemic ("impact of COVID-19"). This process was completed by the first author and a trained student in a master of public health program. We then analyzed the data coded under "impact of COVID-19" using a thematic analysis approach (13) to identify themes and subthemes.

Results
We completed interviews with 18 individuals; interviews ranged from 19 to 50 minutes, averaging 32 minutes. Of the 18 participants, 8 were aged 27 to 39, all were women, 14 worked in a rural clinic, and 12 worked in general practice or family medicine clinics (Table 1).
Two primary themes emerged in all interviews under the parent code of impact of COVID-19: decrease in HPV vaccination and routine care and impact of the pandemic on implementation work ( Table 2). In a minority of interviews, a third theme was also identified: patient safety improvements.

Decreased opportunities for HPV vaccination and routine care
Overall reduction in in-person clinic visits due to the pandemic posed tremendous challenges to the clinics in delivering HPV vaccinations. Participants spoke about 2 main challenges in being able to vaccinate patients. First, all clinics had to implement new protocols to safely treat patients, which included reducing overall patient volume and switching to telehealth visits, both of which resulted in fewer adolescents being seen in person. To reduce patient volume, many participants reported that a respiratory clinic was established to physically separate sick patients from well patients, and this meant less time and capacity to see well patients. As one participant described, "[R]ight now, the biggest priority in our clinic is the sick clinic; we're doing COVID testing in our sick clinic. That's our biggest area here" (Interview-8). Other participants noted that shifts to virtual visits presented numerous challenges to HPV vaccination. In the first place, these changes required time to implement, and in the second place, specific to HPV vaccination, it meant that "when they're pushing virtual visits, it's not going to get them into the door to get those vaccines done" (Interview-11).
The second main challenge was fear of COVID-19 exposure. Most participants reflected on the fear among their patient population of coming into clinics for preventive care such as vaccinations and being exposed to the virus. Although none of the participants had calculated exact numbers for the reduction in vaccinations, nearly all identified patient fear of COVID-19 as a barrier to vaccinating adolescents. As one participant summarized, "[F]or quite a while in the spring, people were really reluctant to come to the doctor's office, because they felt like we're a hotbed of disease" (Interview-10). Parents and guardians were not the only ones afraid of attending routine care visits; the clinics themselves shut down for a certain period, and schools made allowances about attendance at sports physicals in recognition that parents may not want to bring their children into clinics. For example, one participant reported that "in April [2020] . . . [the clinic] essentially cancelled every other visit except zero-to five-[year-old patients]" (Interview-3). Another participant spoke about the ruling that schools allowed prior years' sports physicals to count for school entry, so they "missed some opportunity this year getting in [their] normal amounts" of those visits (Interview-9). Together, the shifts made in clinics to provide patient safety and the hesitancy among parents to take their children into clinics translated into a reduction in HPV vaccinations.

Disruption to ongoing EBI implementation work
We identified 2 subthemes: disruptions due to logistical challenges and disruptions due to shifts in priorities during the pandemic. All participants noted that the COVID-19 pandemic had a negative impact on their ability to carry out ongoing projects and implementation of EBIs to increase HPV vaccination rates. Several participants spoke about previous efforts with QI teams for HPV vaccination projects. As one clinic manager described, "[I]t's just kind of fizzled off with . . . COVID" (Interview-14). Others reflected on projects that were ongoing with external partners that had been forced to halt because of the pandemic. For example, one participant spoke about a project with a pharmaceutical company to implement new strategies to promote HPV vaccination that they had started, but with the pandemic "all those meetings and such came to an end because [they] couldn't meet in person anymore" (Interview-6). Another clinic manager spoke about a school outreach program to promote HPV and other adolescent vaccines in which their clinic usually participates during the spring. However, she reflected that "school wasn't in session at that time. So, we missed that opportunity" (Interview-5). Finally, several participants noted that because of the pandemic, pharmaceutical representatives who are usually allowed into the clinic to provide education were not able to come and that the regularly scheduled state immunization conference was cancelled. These participants reported that these education opportunities are the primary way staff and providers learn about updates to HPV vaccination and best practices and motivate staff to implement EBIs. These interruptions were due to both logistical challenges and shifts in priorities that were necessitated by the pandemic. For example, early in the pandemic, many participants noted that because of shut-downs in routine care services, non-patient-facing staff were furloughed while multiple providers were out of the office due to mandatory quarantines resulting from COVID-19 exposure or infection. As one participant noted, "When we [were] short-staffed . . . that changed a lot of different workflows" (Interview-2), which meant increased time was spent to create new workflows, detracting from time available for other projects. In addition to creating staff shortages, the pandemic also disrupted regular communication between participants and others working on EBI implementation or HPV vaccine promotion. For example, one participant noted that their QI team had been redirected to focus on COVID and so "prior to COVID [they] were meeting once a month. But since COVID, [they've] gotten pulled into more of the COVID-related areas" (Interview-8). Another clinic manager noted that all her team communication had to switch to an online format and "there are some challenges of not having that person in front of you to talk to" (Interview-5) and that this lack of in-person communication had a particularly negative impact on implementation work. Related to fears of COVID-19 transmission, one participant noted that the clinic had to pull all educational materials from waiting areas because it didn't "want to lay them out for the patient to pick up" (Interview-4).
Beyond these logistical challenges, the inability to work on EBI implementation and projects related to HPV vaccination was primarily due to the shift in priorities. As one participant summarized it, "[R]ight now with COVID, I would be amiss if I didn't say HPV [is] probably not at the top of our list. We're trying to make sure that people are staying healthy" (Interview-17). Many participants spoke about the challenges of maintaining safety protocols and the extra work that came along with that, and how these challenges had led to a reduced focus on HPV vaccination overall. Another participant noted that these shifts were not just occurring in clinics but also that "COVID as a whole has changed our health organization. Looking at how do we bring people in safely has been a huge thing" (Interview-1). This need to focus on patient safety, above all else, has been necessary, but all participants who spoke about these shifts said that it meant they have not had time to focus on EBI implementation for HPV vaccination.

Patient safety improvements for infection control made during the COVID-19 pandemic
Finally, in discussing implementation of EBIs for HPV vaccination, several participants spoke more generally about how the pandemic has changed health care delivery and lessons learned for the future. These participants spoke about some positive changes that have resulted from the need to be more creative about health care delivery and patient safety. They reflected that there are likely to be some permanent changes to health care delivery for their clinics and health care systems that may have implications for how EBIs are implemented. Several reported that the health care systems their clinics are affiliated with had created special respiratory clinics -designed to control infection -to care for COVID-19 patients. One reported that "we plan on keeping the respiratory clinic going forever. With all our respiratory stuff, it just makes sense, really" (Interview-16). Another common change was creating separate entrances or times for well and sick patients to be seen. One participant said they "had to reinvent the wheel as far as what keeps people safe, and how [they] still operate and get things that keep people healthy, without giving them the opportunity to catch something" (Interview-1). Because of the attention they devoted to these efforts, this participant spoke with her team about continuing with these changes throughout respiratory syncytial virus (RSV) season and indicated that many of these changes make "the most sense to keep the most majority of the people healthy coming in" (Interview-1). Implications of these kinds of permanent changes are not yet known, although one participant noted that in her clinic this change had resulted in fewer staff members being available for routine preventive care in the short term.

Discussion
Results from these interviews highlight a unique perspective on the impact of the COVID-19 pandemic on adolescent HPV vaccination, that of administrative clinic staff, most of whom work in rural areas. Participants in this study, while not directly involved in health care delivery, manage much of the work that happens behind the scenes to ensure patients receive the care they need. Across interviews, the impact of the pandemic on not only adolescent HPV vaccination but on all health care delivery and related EBI implementation work was evident. Interviews focused on implementation of EBIs for HPV vaccination specifically, but many of the barriers reported in relation to this area also applied to other areas. Data from 2020 identified sharp decreases in adolescent and pediatric vaccination (1,2) as well as well-child visits (14) that have likely persisted into 2021. Now, with the authorization of the COVID-19 vaccine for both adolescents and children, HPV vaccination may not be at the forefront of parents' or clinicians' priorities. Clinics will need to recommit to or expand their HPV vaccination efforts to ensure adolescents are vaccinated on time.
In circumstances without the added pressure of the pandemic, clinics face challenges implementing existing EBIs to encourage HPV vaccination uptake, such as lack of staff to implement EBIs or QI initiatives, lack of knowledge about which EBIs to implement (9), competing priorities, the need for more staff training, and limited PREVENTING CHRONIC DISEASE resources (15). With the added stresses of the pandemic, these challenges have been compounded, and many participants reported that because of the need to address pandemic-related issues, HPV vaccination had fallen lower on the list of priorities. This has meant that ongoing QI efforts and EBI implementation to improve vaccination rates were often halted. By necessity, the response to the pandemic has been reactive, rather than proactive, which means that processes that were already in place were not prioritized during the pandemic. To overcome this, clinic staff should refocus on implementing strategies known to work to promote HPV vaccination (eg, reminder/recall, strong provider recommendation) (16) and have been effective in increasing rates for other vaccines (17,18).
Although these interviews focused on challenges presented by the pandemic, several participants spoke about some of their unexpected findings from their efforts to continue providing health care. These participants spoke about how the pandemic was a learning opportunity in keeping patients safe during large-scale outbreaks and noted that they would continue some of their precautions in the future to deal with other infectious diseases. Although the COVID-19 pandemic has had an overwhelmingly negative impact on health care, valuable lessons have been learned about how to continue delivering primary care. However, the time spent to make these changes was at the expense of other ongoing work. For example, many participants spoke about the shift in priorities and the time that was needed to create processes for telehealth visits. Future research should focus on identifying best practices that have been developed during the pandemic to support not only future pandemic responses but potentially also dealing with typical influenza seasons.
Although the topical focus of these interviews was HPV vaccination, results highlight challenges that have likely been present in all implementation work during the pandemic. There have been calls from the implementation science community to use implementation science to address COVID-19 (19,20), but less attention has been paid to how to address the fact that so much of the ongoing implementation work came to a halt during the pandemic. At this juncture, the pandemic is likely far from over, and history shows us that other pandemics and epidemics will occur. Implementation science researchers need to create resilient and sustainable EBIs and implementation processes that are not as vulnerable to emergency situations. This could mean focusing on implementing practices that could be more sustainable in emergency situations, for example, ensuring systems are in place to use reminder/ recall messaging. Sustainability has long been a challenge for implementation science, and many implementation studies lack an explicit definition of what sustainability means in practice (21,22), making it even harder for researchers and practitioners alike to focus on best practices in this area. The current situation and data from these interviews highlight that researchers must renew their focus on resiliency and sustainability for EBI implementation.
Our study has several strengths. The primary strength was the use of qualitative methods to gather detailed and descriptive information from a relatively understudied perspective during the COVID-19 pandemic, namely clinic managers working in rural settings. Interviews allowed for detailed and nuanced data from clinic managers about the challenges presented by the pandemic to HPV vaccine delivery and EBI implementation. Additionally, more than three-quarters of participants worked in rural clinics, providing another often-understudied perspective on health care delivery and implementation.
This study also has several limitations, primarily related to the timing of these interviews. When interview recruitment began in August 2020, COVID-19 cases were relatively low in Iowa, but by mid-November cases had risen again; therefore, participants may have had different perspectives on the impact of COVID on their work and organizations. However, despite these limitations, these results offer critical insights into this issue, and future research could focus on understanding perspectives from clinic managers in other geographic areas.
In summary, pre-existing low rates of HPV vaccination coupled with the impact of the pandemic threaten to leave adolescents unprotected against HPV and with increased susceptibility to HPVrelated cancers. Our results have short-and long-term implications for both practitioners and researchers working in the fields of adolescent health, HPV vaccination, and implementation science. In the short term, a renewed commitment to EBI implementation for adolescent HPV vaccination is needed to ensure that those who are eligible now as well as those who may have missed doses over the past 2 years are vaccinated. Research conducted before the pandemic found that clinics do not always use EBIs and, when they do, there are significant challenges in implementing them (9,14). These challenges have been exacerbated by the pandemic, and both researchers and clinic staff will need to expend even more effort in this area. For example, for clinics that previously did not have reminder recall systems in place, these systems could be one way to identify all undervaccinated adolescents. However, implementing these new systems may require substantial effort given that the pandemic has taken priority and those involved may need to work even harder to obtain staff buy-in and leadership support. In the long term, the implementation science community needs to create more resilient and sustainable EBIs that can be easily implemented in health care systems. Doing so will help protect PREVENTING CHRONIC DISEASE  Shift in priorities I think prior to COVID happening, we were putting plans in place on how to increase immunizations, whether that's signage in the rooms and just communicating with patients and parents about why this is beneficial for your child or for yourself. And then obviously COVID, and that kind of just threw everything quality improvement out the window, while you're trying to focus on, how the heck are we going to do this? (I-2) A lot has changed prior to COVID and now just trying to ensure that we adhere to all the new guidelines and make sure that we have staff, that we protect our staff as well with PPE and all of those things that are constantly changing, as we're entering into the search plan. I would say that's probably [leadership's] main focus right now. (I-15) Abbreviations: EBI, evidence-based intervention; HPV, human papillomavirus; I, interview; PPE, personal protective equipment; QI, quality improvement.