The value of Elmstone to the lord of the manor, which at the time of the Domesday Book was St Augustine's Abbey in Canterbury, was £2 in 1066, £0.5 c. 1070 and £3 in 1086. By the time of Henry III the manor had passed to the Leyburne family. In 1367 it passed to the crown and in 1388, Richard II passed it to Chiltern Langley Priory in Hertfordshire.
The church remained an unappropriated rectory, valued at £10 in the 1291 Taxatio. Its dedication is unrecorded in medieval sources and the church remains to this day without a recognised dedication.