Virus infections pose a continuous threat to human health and can result in millions of deaths per year. SARS-CoV-2 infection has been linked to the high-affinity high-density lipoprotein (HDL) receptor scavenger receptor class B, type 1 (SR-B1). Mechanisms by which SR-B1 supports SARS-CoV-2 infection and replication, as well as the breadth of viruses that exploit this receptor, are incompletely defined. In evaluating the role of SR-B1 in the biology of infection with SARS-CoV-2, influenza A virus, and vesicular stomatitis virus, we show that SR-B1 chemical inhibition or knockout adversely affects infection for these viruses. Inhibiting SR-B1 results in lack of acidification in the endolysosomal compartment and entrapment of SARS-CoV-2 in endosomal-lysosomal vesicles. These findings together indicate that SR-B1, and possibly HDL, is critical for successful SARS-CoV-2 trafficking through a pH-dependent vesicular entry pathway. Our work provides insights into how SR-B1 can impact viral infection in human lung cells.