Genetic and epigenetic inheritance through mitosis is critical for dividing cells to maintain their state. This process occurs in the context of large-scale re-organization of chromosome conformation during prophase leading to the formation of mitotic chromosomes, and during the reformation of the interphase nucleus during telophase and early G1. This review highlights how recent studies over the last 5 years employing chromosome conformation capture combined with classical models of chromosome organization based on decades of microscopic observations, are providing new insights into the three-dimensional organization of chromatin inside the interphase nucleus and within mitotic chromosomes. One striking observation is that interphase genome organization displays cell type-specific features that are related to cell type-specific gene expression, whereas mitotic chromosome folding appears universal and tissue invariant. This raises the question of whether or not there is a need for an epigenetic memory for genome folding. Herein, the two different folding states of mammalian genomes are reviewed and then models are discussed wherein instructions for cell type-specific genome folding are locally encoded in the linear genome and transmitted through mitosis, e.g., as open chromatin sites with or without continuous binding of transcription factors. In the next cell cycle these instructions are used to re-assemble protein complexes on regulatory elements which then drive three-dimensional folding of the genome from the bottom up through local action and self-assembly into higher order levels of cell type-specific organization. In this model, no explicit epigenetic memory for cell type-specific chromosome folding is required.