We Bought a Kazoo formed in January 2017 when Oli Harris bought a kazoo from a local music store. The kazoo wasn't used in the final recording of their first single, The Ballad of Mickey Purchase.