An Open-Label Phase II Clinical Trial Assessing the Safety, Feasibility, Efficacy and Immunological Correlates of Heterologous Prime-Boost With pBI-11 (IM) and TA-HPV (IM) Combined With Pembrolizumab as Treatment for Patients With Advanced, PD-L1 CPS≥1, hrHPV+ Oropharyngeal Cancer

Inclusion Criteria: * Signed and dated written informed consent * Male or female \>= 18 years of age on the day of signing informed consent * Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1 * Having been diagnosed with R/M p16+ PD-L1 CPS \>= 1 OPC not previously treated for R/M disease. They must be eligible for, and planning to start therapy with pembrolizumab, according to standard of care * hrHPV(+) status (staining with p16 is adequate) and PD-L1 expression (CPS≥1) in tumor based on validated testing methods performed on FFPE tumor tissue (needle core biopsy or resection; fine needle aspiration/biopsy \[FNA\] cell blocks acceptable if with adequate tissue) at local labs or VUMC labs. \[This biopsy tissue will also be used for the pre-treatment tissue research correlate studies.\] For patients with neither existing tumor hrHPV and PD-L1 CPS test results nor adequate archived tissue available for PD-L1 and hrHPV testing, a biopsy performed during screening is necessary to obtain diagnostic tissue. If no tissue is available for PD-L1 and hrHPV testing on either archived or newly obtained tissue, the patient cannot be enrolled. * Evaluable tumor burden (measurable and/or non-measurable tumor lesion\[s\]) which can be followed by computed tomography (CT) scan or magnetic resonance imaging (MRI), based on Immune-Modified Response Evaluation Criteria in Solid Tumors (iRECIST) as assessed by the local site investigator/radiology. Tumors that are biopsied (for diagnosis and research use) will not be considered in the iRECIST version (v)1.1 assessment. However, if patient has only one evaluable tumor, patient may still be eligible for participation * Absolute neutrophil count (ANC) \>= 1,000/uL (resulted =\< 28 days prior to first dose of protocol-indicated treatment) * CD4 T cell count \> 200/uL (resulted =\< 28 days prior to first dose of protocol-indicated treatment) * Platelets \>= 75,000/uL (resulted =\< 28 days prior to first dose of protocol-indicated treatment) * Hemoglobin \>= 7.0 g/dL (resulted =\< 28 days prior to first dose of protocol-indicated treatment) * Estimated glomerular filtration rate (eGFR) \>= 45 mL/min (as calculated by the Cockcroft-Gault Formula or calculated/measured by an alternative established institutional standard consistently applied across participants at the site) (resulted =\< 28 days prior to first dose of protocol-indicated treatment) * Total bilirubin =\< 1.5 times institutional upper limit of normal (ULN), or direct bilirubin =\< ULN for participants with total bilirubin \> 1.5 x ULN (resulted =\< 28 days prior to first dose of protocol-indicated treatment) * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\]) and alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 times institutional ULN (resulted =\< 28 days prior to first dose of protocol-indicated treatment) * Calcium =\< 11.5 mg/dL or =\< 2.9 mmol/L; in patients with albumin outside the normal range, calcium (corrected for albumin) must be =\< 11.5 mg/dL or =\< 2.9 mmol/L (resulted =\< 28 days prior to first dose of protocol-indicated treatment) * International normalized ratio (INR) or prothrombin time (PT) =\< 1.5 times institutional ULN; except for patients receiving anticoagulant therapy as long as PT in such patients per investigator judgment is within therapeutic range of intended use of anticoagulants (resulted =\< 28 days prior to first dose of protocol-indicated treatment) * Activated partial thromboplastin time (aPTT) preferred or partial thromboplastin time (PTT) =\< 1.5 times institutional ULN; except for patients receiving anticoagulant therapy as long as PTT in such patients per investigator judgment is within therapeutic range of intended use of anticoagulants (resulted =\< 28 days prior to first dose of protocol-indicated treatment) * Patients must not be breastfeeding and further agree to not breastfeed during study treatment; and for at least 120 days after patient's final dose of heterologous vaccination and pembrolizumab * A woman of childbearing potential (WOCBP) must have a negative serum or urine pregnancy test during screening within 28 days prior to receiving first dose of protocol-indicated treatment, and must agree to follow instructions for using acceptable contraception from the time of signing consent, and until at least 180 days after her final dose of heterologous vaccination and pembrolizumab. In order to be considered not of childbearing potential, women under the age of 62 must have a documented serum follicle stimulating hormone (FSH) level more than 40 mIU/mL (or within local laboratory reference range for postmenopausal women * Patients must refrain from donating blood or sperm throughout the duration of study treatment followed by at least 120 days after patients' final dose of heterologous vaccination or pembrolizumab * A patient able to father children who is sexually active with a WOCBP must agree to follow instructions for using acceptable contraception, from the time of signing consent, and until at least 120 days after his final dose of heterologous vaccination and pembrolizumab Exclusion Criteria: * Disease that can be treated with curative intent as assessed by patient's study physician. If patient in this situation wishes for treatment with palliative intent then they may enroll * Multiple primary oropharyngeal (OP) tumors * Primary cancer not originated from the oropharynx * Has received any therapy for R/M disease. Therapy in the setting of curative intent is allowed * Patient is pregnant or breastfeeding * Prior prophylactic or therapeutic vaccination with any human papillomavirus (HPV) antigen except L1 \* Note: previous receipt of the Gardasil (registered trademark) vaccine (including Gardasil 9 \[registered trademark\]) or the Cervarix (registered trademark) vaccine does not exclude * Has received a live vaccine within 30 days prior to first dose of study treatment * Examples of prohibited live vaccines include, but are not limited to measles, mumps, rubella, varicella (chickenpox), zoster (shingles; Zostavax \[registered trademark\] \[not including Shingrix (registered trademark)\]), yellow fever, rabies, bacillus Calmette-Guerin (BCG), and typhoid (oral) vaccine * Seasonal influenza vaccines for injection are generally killed virus vaccines and are allowed; however, intranasal influenza vaccines (e.g., FluMist \[registered trademark\] Quadrivalent) are live attenuated vaccines and are not allowed. The three United States (US) FDA-approved coronavirus disease 2019 (COVID-19) vaccines (by Pfizer BioNTech, Moderna, and Janssen) and AstraZeneca COVID-19 vaccine are based on messenger (m)RNA or replication-deficient adenoviral vectors and are allowed * Is currently participating in or, within 4 weeks prior to first dose of study treatment, has participated in a study of an investigational agent or has used an investigational device * Note: Participants who have entered the follow-up phase of an investigational study may participate as long as it has been 4 weeks after the last dose of the previous investigational agent * Patients with persistent toxicities of =\< grade 3 (National Cancer Institute \[NCI\]-Common Terminology Criteria for Adverse Events \[CTCAE\] v5.0) will be excluded * Diagnosis of immunodeficiency * Known additional malignancy that is non-localized or progressing or has required active treatment within the past 3 years prior to first dose of study treatment \* Note: Participants with usual basal cell carcinoma of the skin, squamous cell carcinoma of the skin, or carcinoma in situ (for example, in situ cervical cancer or breast carcinoma, superficial bladder cancer, or non-invasive intraductal carcinoma of the prostate) that have undergone potentially curative therapy are not excluded * Radiographically detectable (even if asymptomatic and/or previously treated) central nervous system metastases and/or carcinomatous meningitis as assessed by local site investigator and radiology review * Recipient of previous allogeneic tissue/solid organ transplant * Known severe hypersensitivity (\>= grade 3) to pembrolizumab and/or any of its excipients * History of myocarditis or pericarditis or other known clinically significant underlying heart disease (e.g., cardiomyopathy, congestive heart failure, symptomatic arrhythmia not controlled by medication, unstable angina, history of acute myocardial infarction) * Uncontrolled intercurrent illness including, but not limited to ongoing or active infection, sepsis, or psychiatric illness/social situations that would limit compliance with study requirements * Active or chronic infection with any of the following (with testing for all three conditions required during screening for this study): * Human immunodeficiency virus (HIV) * Hepatitis C virus (HCV) * Hepatitis B virus (HBV) * Active autoimmune disease with immunodeficiency as a clinical component (e.g., rheumatoid arthritis, systemic lupus erythematosus \[SLE\], ulcerative colitis, Crohn's disease, multiple sclerosis \[MS\], ankylosing spondylitis) * Within 60 days prior to initiating first dose of protocol-indicated treatment, patient has received therapy with immunosuppressive drugs such as cyclosporine, adrenocorticotropic hormone (ACTH), alkylating agents, antimetabolites, radiation, tumor necrosis factor (TNF) inhibitors, or systemic corticosteroids * Recognized immunodeficiency diseases including cellular immunodeficiencies; hypo-gammaglobulinemia or dys-gammaglobulinemia; or acquired, hereditary, or congenital immunodeficiencies * Patients and their close social, sexual, or domestic contacts may not have non-healed wounds or active exfoliative skin conditions, such as eczema, burns, impetigo, varicella-zoster virus infection, herpes simplex virus infection, severe acne, severe diaper dermatitis with extensive areas of denuded skin, psoriasis, lichen planus, or Darier disease (keratosis follicularis) * History and presence of atopic dermatitis * Patients and their close social, sexual, or domestic contacts may not have known conditions associated with immunosuppression, such as HIV/acquired immunodeficiency syndrome (AIDS), leukemia, lymphoma, generalized malignancy, solid organ transplant recipient, or hematopoietic stem cell transplant recipient \< 24 months post-transplant OR \>= 24 months, but who have graft-versus-host disease or disease relapse, or be of less than 1 year of age * Found at screening visit to have severe arrhythmias (e.g., atrial fibrillations), sinus bradycardia (\< 50 beats per minute \[BPM\]), sinus tachycardia (\> 100 BPM) via resting electrocardiography (EKG). Patients with controlled arrhythmias followed by a cardiologist are eligible * Patients with active interstitial lung disease (ILD)/pneumonitis or a history of ILD/pneumonitis requiring treatment with systemic steroids. Hypoxemia (oxygen saturation \[SpO2\] \< 92% for 5 min or longer) by finger pulse oximetry corrected by oxygen supplementation is allowed