The Domesday Survey does not mention a church at Milton Ernest, but records that land was held here by Miles Crispin, Hugh de Beauchamp, Nigel de Aubigny, Walter of Flanders, Hugh de Grandmesnil's wife and the King's Reeves, Beadles and Almsmen.
VCH records that the advowson was in the hands of the de Aubigny family from an early date. (VCH,148). Cecelia de Aubigny granted the church to Beaulieu Priory, which had been founded by her son, Robert de Aubigny c.1140-46. In 1275 the church was given to Beaulieu by the Bishop of Gravesend.