In the C10th, both Upper and Lower Tadmarton formed a royal estate. In 956 King Edwy is reputed to have granted 20 hides to two of his thegns, but these soon came into the possession of Abingdon Abbey, who retained them until the Dissolution. After the Conquest, Abbot Aethelm was induced to grant them to Robert d'Oilly, Sheriff of Oxford. In 1086 the estate was still around 20 hides. In Henry I's reign, Simon le Despenser exchanged lands that he held of the abbey in Berkshire for Tadmarton manor and 3 hides at Garsington. Simon was a nephew of Reynold, Abbot of Abingdon. There was continual trouble between the Despensers and the abbey until the end of the C13th, although the abbey had secured bulls from the Pope in 1146 and 1152 confirming its possession.
The church was attached to the manor throughout the relevant period. Tadmarton was a manor of Abingdon Abbey , but for much of the C12th it was held of the Abbey by the Despenser family. The advowson was in the hands of the abbey until it was surrendered to the crown in 1538 (VCH).