Prittlewell was held in 1086 by Swein of Essex in demesne as a manor of 7½ hides. The record also notes pasture for 12 pigs, pasture for 200 sheep, and a church. The priory is first noted in a confirmation of the possessions of Lewes Priory (Sussex) by Ralph, Archbishop of Canterbury dated 1121, in which it is said to have been granted to Lewes by Robert, son of Swein. This suggests a foundation date between 1086 and 1121. Robert granted to the priory the church of Prittlewell with the chapels of Sutton and Eastwood, and the tithes of the hamlet of Milton. Archbishop Becket,confirmed these gifts, and also the churches of Rayleigh, Thundersley, the two Shoeburys, Canewdon, Wickford, Stoke, Clavering and Langley: a generous endowment for a house of this size.
At the Dissolution it passed to Thomas Audley, and at his death ten years later it was bought by Sir Richard Rich, owner of large estates in Essex, including his residence of Lees Priory. Prittlewell was apparently rented out to tenants during Rich's ownership.