Structural maintenance of chromosomes (SMC) protein complexes are key determinants of chromosome conformation. Using Hi-C and polymer modelling, we study how cohesin and condensin, two deeply conserved SMC complexes, organize chromosomes in the budding yeast Saccharomyces cerevisiae. The canonical role of cohesin is to co-align sister chromatids, while condensin generally compacts mitotic chromosomes. We find strikingly different roles for the two complexes in budding yeast mitosis. First, cohesin is responsible for compacting mitotic chromosome arms, independently of sister chromatid cohesion. Polymer simulations demonstrate that this role can be fully accounted for through cis-looping of chromatin. Second, condensin is generally dispensable for compaction along chromosome arms. Instead, it plays a targeted role compacting the rDNA proximal regions and promoting resolution of peri-centromeric regions. Our results argue that the conserved mechanism of SMC complexes is to form chromatin loops and that distinct SMC-dependent looping activities are selectively deployed to appropriately compact chromosomes.