In 1086 East Shefford was held by Aiulf the Sheriff and was assessed at 5 hides, but under the Confessor, when it was held by Beorhtic, it was assessed at 10 hides. It had passed to the crown by the beginning of the 12thc, and was granted to Payn Peverell by Henry I. Payn gave the manor, along with his daughter Maud, to Hugh, son of Fulbert de Dover, but when Hugh died childless after 1170 East Shefford passed to Maud's sister Alice, wife of Hamon Peche, and subsequently to her son Gilbert about 1194. He was to forfeit it under King John.
The church served the Manor House as well as the village, and the advowson is first mentioned in 1222-23 when Henry de Boxworth released it to the Prior of Barnwell. It was subsequently to return to the Lords of the Manor, however, and in particular the Fettiplace family, first recorded at East Shefford in the person of Thomas Fettiplace in 1413. He m. Beatrice, a member of the royal family of Portugal, and their tomb is in the S chapel. The Fettiplace connection ended with Thomas's great-grandson John, and his wife Dorothy (also buried in the church). The Manor House was demolished in 1871.