Hepatic gluconeogenesis is required for maintaining blood glucose homeostasis; yet, in diabetes mellitus, this process is unrestrained and is a major contributor to fasting hyperglycemia. To date, the impacts of chromatin modifying enzymes and chromatin landscape on gluconeogenesis are poorly understood. Through catalyzing the removal of methyl groups from specific lysine residues in the histone tail, histone demethylases modulate chromatin structure and, hence, gene expression. Here we perform an RNA interference screen against the known histone demethylases and identify a histone H3 lysine 36 (H3K36) demethylase, Jhdm1a, as a key negative regulator of gluconeogenic gene expression. In vivo, silencing of Jhdm1a promotes liver glucose synthesis, while its exogenous expression reduces blood glucose level. Importantly, the regulation of gluconeogenesis by Jhdm1a requires its demethylation activity. Mechanistically, we find that Jhdm1a regulates the expression of a major gluconeogenic regulator, C/EBPalpha. This is achieved, at least in part, by its USF1-dependent association with the C/EBPalpha promoter and its subsequent demethylation of dimethylated H3K36 on the C/EBPalpha locus. Our work provides compelling evidence that links histone demethylation to transcriptional regulation of gluconeogenesis and has important implications for the treatment of diabetes.