The manor of Charlton was held at the time of the Domesday Survey by the Church of Malmesbury. There was land for 13 ploughs and a mill rendering 15s. The church was standing at Charlton in the late 12th century and in 1248 it was called a chapel, served by Malmesbury (VCH XIV, 48-50).