Phase I/II Study of CDK4/6 Inhibition With Abemaciclib to Upregulate PSMA Expression Prior to 177Lu-PSMA-617 Treatment in Patients With Metastatic Castrate Resistant Prostate Cancer (mCRPC) Previously Treated With Novel Hormonal Agents and Chemotherapy

Inclusion Criteria: 1. Participants must have histologically or cytologically confirmed prostate cancer. Either fresh biopsy or archival tissue can be used for confirmation. 2. Age \>= 18 years. 3. Patients must have metastatic castration resistant prostate cancer (mCRPC) with progression based on Prostate Cancer Working Group 3 (PCWG3) criteria. 4. Patients must have adenocarcinoma histology. 5. Prior treatment with at least one novel hormonal agents (NHA) such as abiraterone acetate, enzalutamide, apalutamide, darolutamide etc. 6. Patients must have prior orchiectomy and/or ongoing androgen-deprivation therapy and a castrate level of serum testosterone (\< 50 ng/dL or \< 1.7 nmol/L) 7. Patients must have a 68Ga-PSMA-11 PET scan with at least one PSMA-positive lesions (maximum standardized uptake value \[SUVmax\] greater than SUVmax of liver) as determined by nuclear medicine review prior to start of lead-in treatment with abemaciclib 8. Patients must have Eastern Cooperative Oncology Group (ECOG) performance status (PS) of 0-2 9. Patients must have life expectancy of \> 6 months 10. Patients must have adequate organ function as outlined below and bone marrow reserve * White blood cell (WBC) \> 2.5 * Absolute neutrophil count (ANC) \> 1.5 * Hemoglobin (Hgb) \>= 8.0 \[Note- Participants may receive erythrocyte transfusions to achieve this hemoglobin level at the discretion of the investigator. Initial treatment must not begin earlier than the day after the erythrocyte transfusion\] * Platelets (Plt) \>= 100 x 10\^9/Liter (100,000/Microliter) * Total bilirubin =\< 1.5 x the institutional upper limit of normal (ULN). For patients with known Gilbert's Syndrome =\< 2 ULN and direct bilirubin within normal limits is permitted * Aspartate aminotransferase (AST)(serum glutamic oxaloacetic transaminase (SGOT)) =\< 3 X institutional upper limit of normal (=\< 5.0 ULN for patients with liver metastases) * Alanine aminotransferase (ALT)(serum glutamic pyruvic transaminase (SGPT)) =\< 3 X institutional upper limit of normal (=\< 5.0 ULN for patients with liver metastases) * Creatinine =\< 1.5 x within institutional upper limit of normal OR creatinine clearance glomerular filtration rate (GFR) \>= 30 mL/min/1.73 m, calculated using the Cockcroft-Gault equation. 11. Patient must be able to swallow oral medications 12. Patients must have the ability to understand a written informed consent document, and the willingness to sign it 13. Human immunodeficiency virus (HIV)-infected individuals on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial 14. For participants with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated 15. Individuals with a history of hepatitis C virus (HCV) infection must have been treated and cured. For individuals with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load 16. Individuals with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial 17. Patients with reproductive potential must agree to use effective contraception and to not donate sperm during the study and for at least 2 months following the last dose of study treatment. Effective method of contraception means male condom with spermicide, female condom with spermicide, diaphragm with spermicide, cervical sponge, or cervical cap with spermicide Exclusion Criteria: 1. Patients with small cell or neuroendocrine carcinoma histology. 2. Patients with a super scan seen in the baseline bone scan. Super scan refers to a bone scan with diffusely increased skeletal radioisotope uptake relative to soft tissue 3. Patients with prior treatment with CDK4/6 inhibitors 4. Patients with prior treatment with PSMA-targeted radioligand therapy. Patients with previous treatment with PSMA targeting therapies (Such as Chimeric antigen receptor T cells (CAR-T) or Bi-specific T-cell engagers (BiTEs) are eligible. 5. Patients treated with Radium-223 within 6 weeks prior to study entry. 6. Any systemic anti-cancer therapy within 3 weeks of study entry 7. Patients who have experienced significant radiation-related adverse events (AEs) from prior radiation treatment (\>= grade 3) or have experienced persistent radiation-related AEs that have not resolved by the time of study randomization 8. Patients with a history of central nervous system (CNS) metastases are ineligible unless they have received prior therapy (surgery, radiation therapy (RT), gamma knife) are asymptomatic, and not receiving corticosteroids for this indication. Head imaging is not required 9. Patients with symptoms of cord compression or impending cord compression 10. Patients with concurrent serious medical conditions as determined by primary investigator 11. Patients with other significant malignancies that are expected to alter life expectancy or interfere with disease assessment. Patients with adequately treated skin cancer, non-muscle-invasive bladder cancer and patients with prior history of malignancy who have been disease free for more than 2 years are eligible. Patients with history of in-situ/early stage melanoma will not be excluded 12. Patients who have not recovered from adverse events due to prior anti-cancer therapy to =\< grade 1 or baseline (other than alopecia or peripheral neuropathy) 13. Patients with serious and/or uncontrolled preexisting medical condition(s) that, in the judgment of the investigator, would preclude participation in this study (for example, interstitial lung disease, severe dyspnea at rest or requiring oxygen therapy, history of major surgical resection involving the stomach or small bowel, or preexisting Crohn's disease or ulcerative colitis or a preexisting chronic condition resulting in baseline grade 2 or higher diarrhea) 14. The patient has active systemic bacterial infection (requiring intravenous (IV) antibiotics at time of initiating study treatment), fungal infection, or detectable viral infection (such as known human immunodeficiency virus positivity or with known active hepatitis B or C (for example, hepatitis B surface antigen positive). Screening is not required for enrollment 15. The patient has a personal history of any of the following conditions: syncope of cardiovascular etiology, ventricular arrhythmia of pathological origin (including, but not limited to, ventricular tachycardia and ventricular fibrillation), or sudden cardiac arrest. 16. Patients currently receiving any other investigational therapeutic agents.