Zero-inflated count outcomes arise quite often in research and practice. Parametric models such as the zero-inflated Poisson and zero-inflated negative binomial are widely used to model such responses. However, interpretations of those models focus on the at-risk subpopulation of a two-component population mixture and fail to provide direct inference about marginal effects for the overall population. Recently, new approaches have been proposed to facilitate such marginal inferences for count responses with excess zeros. However, they are likelihood based and impose strong assumptions on data distributions. In this paper, we propose a new distribution-free, or semiparametric, alternative to provide robust inference for marginal effects when population mixtures are defined by zero-inflated count outcomes. The proposed method also applies to longitudinal studies with missing data following the general missing at random mechanism. The proposed approach is illustrated with both simulated and real study data.