Lilley was held by Leodgifu from Earl Harold in 1066, and in 1086 the manor, assessed at 5 hides, was held by Geoffrey de Bec. 2 hides were in demesne, and the rest was held from him. The recorded population of 19 villans, 6 bordars, 4 cottars, 6 slaves and a priest suggests a total population of some 200 people, making Lilley a substantial settlement.
By the early 13thc the manor was in the tenure of William Malet of Gerardville, but in 1204 it was granted by King John to Matthew de Lilley.