The Domesday survey recorded 3 holdings in Middleton in 1086. Richard, son of Count Gilbert of Brionne held 1½ hides and 30 acres, which was in the hands of 13 sokemen in 1086. A second holding of 1½ hides and 28 acres was held by Gilbert the Priest, who claimed to hold it as the queen’s gift. This was held by 9 sokemen of Earl Aefgar in 1066. Finally 15 acres were appropriated by Robert Malet in 1086.
By the reign of Edward III the manor was held by a family called Theobald, who later took the name of Sudbury from their residence in that town, Simon Sudbury, Bishop of London (1361-75) and Archbishop of Canterbury (1375-81) founded the College of St Gregory in Sudbury in 1373 and endowed it with his manor of Middleton, where it remained until the Dissolution.