Recombinant adeno-associated viruses (AAVs) are quickly becoming the preferred viral vector for viral gene delivery for the treatment of a wide variety of genetic disorders. However, since their use in a clinical trial targeting hemophilia B patients 10 years ago, immune responses to the AAV capsid appear to have hampered some of the early clinical gene transfer efficacy. Indeed, AAV-based gene transfer has been shown to reactivate capsid-specific memory T cells, which have correlated with a decline in AAV-transduced tissue in some patients. Importantly, clinical trials have also shown that this reactivation can be quelled by administering time-course taper of glucocorticoid steroids before or after dosing. More recently, two clinical studies have shown that AAV gene transfer is not only able to induce a deleterious immune response, but also can result in the initiation of a tolerance to the AAV capsid mediated by regulatory T cells and exhausted T cells. This article reviews clinical trials describing immune responses to AAV, as well as the mechanisms responsible for immune tolerance in chronic infections and how it could apply to AAV-based gene transfer. A better understanding of both cytotoxic and tolerogenic immune responses to recombinant AAV will lead to safer gene transfer protocols in patients.