Castlethorpe was part of the manor of Hanslope which, in 1066 was held by King Edward's housecarl Healfdene, and at the time of the Domesday Survey, was held by Winemar the Fleming. It was assessed at 10 hides, 5 of which were in demesne, and was home to 36 villans, 11 bordars and 8 slaves. There was a mill, meadow for 11 ploughs and woodlnd sufficient for 1000 pigs.
Winemar's lands passed at his death to Michael de Hanslope, and when he died the Honour of Hanslope and the manor passed to William Mauduit, married to Hanslope's daughter Maud. The manor largely remained in the possession of the Mauduits until 1215, when Robert Mauduit joined the wrong side in the war against King John. The manor was eventually returned to Robert and passed to his son William , whose own son, also William succeeded to the earldom of Warwick through his mother. Hanslope remained in this line throughout the medieval period.
The church was originally the parish church of the manor of Hanslope, and St James,Hanslope a chapel. The situation was reversed by a licence granted by Bishop Grosse teste (1235-53) and Castlethorpe church became a chapel of Hanslope.