In Cnut's reign, lands in Waltham were held by Tofig the Proud, who founded the church and granted lands to it. After his death in 1042 his lands were forfeited to King Edward the Confessor, who granted them to Harold son of Godwin. Harold then held a manor of 40 hides in Waltham, 3 of which he granted to the college he had established at the church. After his defeat and death at Hastings, his land passed to King William, who granted it c.1075 to Walcher, Bishop of Durham, in order to provide the bishop with a home near London. Later sources show that Walcher also acquired 2½ of the 3 hides held by the college in Northland, and it seems probable that he established some degree of control over the abbey itself. In 1086 the Bishop of Durham was William of St Calais, and his tenants at Waltham included 2 sokemen holding 5 hides, another 4 holding 2 hides and ½ a virgate, the church of the Holy Cross holding ½ a hide, William of Warenne holding 1½ hides less 15 acres and Ranulf, brother of Ilger holding 30 acres. There were also 12 houses and a gate in London (Aldgate) that belonged to the bishop as part of this manor. According to VCH, the history of the sokemen's lands cannot be traced after 1086.
A dispute between William II and the Bishop of Durham led to the Durham lands being in the king's hands between 1088 and 1091, and while the see of Durham was vacant in 1096, King William granted Waltham and its adjoining lands to the college, but the manor appears to have remained in royal hands. Beginning with Henry I's first wife, Maud, Waltham appears to have been granted successively to Queens of England until the time of Eleanor of Aquitaine. This arrangement ceased around 1163-64, when it was farmed by a series of tenants, then in 1177 Henry II refounded the college and increased its endowments. In 1189 the abbot and canons paid King Richard I 300 marks for a grant of the entire manor, to be held at an annual rent of £60. This arrangement continued, and the manor remained in the possession of the abbey until the Dissolution.