According to tradition, a Norman warrior, Walterus le Brun, arrived in Scotland in 1073 and with him came the origins of the Brown family.
Originally afforded status due to their links to the Royal house of Dunkeld, the Brouns would go on to construct their first residence at Colstoun in 1270, believed to have been a small, square tower which would be added to extensively in later years.
After the emigration of the clan chief, the tenth Baronet, Sir William, to Australia, the clan would begin a permanent association with the continent which would continue even after his grandson, Sir Lionel, died in 1995 without an heir. This is because the chiefship would pass to his cousin, Sir William Windsor Broun, who served during the Second world War in New Guinea and Borneo and would later work for Universal Films Australia. He was succeeded in 2007 by his nephew, Sir Wayne Broun, who returned home to Colstoun House in 2009 for the year of the Scottish Homecoming.