Phase II Study of Hypofractionated Radiation Therapy to Augment Immune Response in Patients With Metastatic GastroIntestinal Malignancies Progressing on Immune Therapy (ARM-GI)

Inclusion Criteria: 1. Patients must have a histologically, cytologically, or radiographically confirmed metastatic gastrointestinal (GI) malignancy (esophageal, gastroesophageal, gastric, small intestine, hepatocellular, pancreaticobiliary, colorectal, or anal cancer). 2. Patients must be receiving immunotherapy (checkpoint inhibitor or CTLA4 inhibitor) with overall response of progressive disease by RECIST criteria. 3. Patients must have at least two metastases which are individually progressing as per RECIST criteria, one of which can be safely unirradiated as adjudicated by the treating radiation oncologist (e.g. lesions for which small increases in dimensions are unlikely to precipitate significant symptoms). 4. Patients must have 1-5 sites of disease meeting standard-of-care indications for palliative radiation therapy as adjudicated by the treating radiation oncologist. For example: * Symptomatic disease causing pain, bleeding, dyspnea, dysphagia, or nausea * At-risk for neurologic, respiratory, cardiovascular, gastrointestinal, musculoskeletal, or hepatobiliary compromise 5. Evaluation by a radiation oncologist within 28 days of study registration. 6. Must have adequate organ function to administer radiation therapy and immunotherapy as per standard of care. 7. Age \>= 18 years. 8. Life expectancy exceeding 6 months. 9. Eastern Cooperative Oncology Group (ECOG) 0-2 or Karnofsky performance status \>= 50. 10. Radiation therapy is known to be teratogenic and therefore women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control) prior to study entry and for the duration of radiation therapy. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study and for 3 months after completion of radiation therapy. Contraception requirements during the follow-up period of 6 months will be according to standard of care for immunotherapy administration. a. If a woman is of child-bearing potential, a negative pregnancy test within 28 days prior to study enrollment is required. 11. Ability to understand a written informed consent document, and the willingness to sign it. Exclusion Criteria: 1. Enrollment on immunotherapy clinical trial for which radiation therapy is not permitted. 2. Administration of radiation therapy within 4 weeks prior to study enrollment. 3. Treatment with systemic corticosteroids or other immunosuppressive medications which would significantly diminish the effect of immunotherapy as judged by the treating physician. 4. Radiation therapy is contraindicated as adjudicated by the radiation oncologist.