After spending several years with the post-punk outfit Sidecar Kisses, vocalist/guitarist Ritzy Bryan and bassist Rhydian Dafydd left the lineup and launched the Joy Formidable, drawing heavily from shoegaze and noisy alt-rock to create their new group's sound. Formed in North Wales in 2007, the band later resettled in London, where original drummer Justin Stahley was replaced by Matt Thomas. The Joy Formidable began building a buzz in 2008, making appearances at several British festivals and earning high marks from publications like The Guardian, which recommended the band’s "darkly joyous soft-loud racket."

After releasing several singles, the group issued the mini-album A Balloon Called Moaning in early 2009. Tours with the Editors and Passion Pit followed, and the Joy Formidable issued a live album, First You Have to Get Mad, before signing with Black Bell Records, an independent label run by Passion Pit member Ayad Al Adhamy. With Black Bell’s support, A Balloon Called Moaning was released stateside in 2010. The Joy Formidable toured throughout the year and kicked off 2011 with the international release of The Big Roar, their full-length studio debut.