INTRODUCTION: Emergency medical services (EMS) dispatchers have made efforts to determine whether patients are high risk for coronavirus disease 2019 (COVID-19) so that appropriate personal protective equipment (PPE) can be donned. A screening tool is valuable as the healthcare community balances protection of medical personnel and conservation of PPE. There is little existing literature on the efficacy of prehospital COVID-19 screening tools. The objective of this study was to determine the positive and negative predictive value of an emergency infectious disease surveillance tool for detecting COVID-19 patients and the impact of positive screening on PPE usage. METHODS: This study was a retrospective chart review of prehospital care reports and hospital electronic health records. We abstracted records for all 911 calls to an urban EMS from March 1-July 31, 2020 that had a documented positive screen for COVID-19 and/or had a positive COVID-19 test. The dispatch screen solicited information regarding travel, sick contacts, and high-risk symptoms. We reviewed charts to determine dispatch-screening results, the outcome of patients' COVID-19 testing, and documentation of crew fidelity to PPE guidelines. RESULTS: The sample size was 263. The rate of positive COVID-19 tests for all-comers in the state of Massachusetts was 2.0%. The dispatch screen had a sensitivity of 74.9% (confidence interval [CI], 69.21-80.03) and a specificity of 67.7% (CI, 66.91-68.50). The positive predictive value was 4.5% (CI, 4.17-4.80), and the negative predictive value was 99.3% (CI, 99.09-99.40). The most common symptom that triggered a positive screen was shortness of breath (51.5% of calls). The most common high-risk population identified was skilled nursing facility patients (19.5%), but most positive tests did not belong to a high-risk population (58.1%). The EMS personnel were documented as wearing full PPE for the patient in 55.7% of encounters, not wearing PPE in 8.0% of encounters, and not documented in 27.9% of encounters. CONCLUSION: This dispatch-screening questionnaire has a high negative predictive value but moderate sensitivity and therefore should be used with some caution to guide EMS crews in their PPE usage. Clinical judgment is still essential and may supersede screening status.