piRNAs function as guardians of the genome by silencing non-self nucleic acids and transposable elements in animals. Many piRNA factors are enriched in perinuclear germ granules, but whether their localization is required for piRNA biogenesis or function is not known. Here we show that GLH/VASA helicase mutants exhibit defects in forming perinuclear condensates containing PIWI and other small RNA cofactors. These mutant animals produce largely normal levels of piRNA but are defective in triggering piRNA silencing. Strikingly, while many piRNA targets are activated in GLH mutants, we observe that hundreds of endogenous genes are aberrantly silenced by piRNAs. This defect in self versus non-self recognition is also observed in other mutants where perinuclear germ granules are disrupted. Together, our results argue that perinuclear germ granules function critically to promote the fidelity of piRNA-based transcriptome surveillance in C. elegans and preserve self versus non-self distinction.