The bailiff of the abbey of Séez in Normandy once lived at Bailiffscourt, explaining the presence of a small 13thc. chapel on the site. The present house and outbuildings were erected by Lord Moyne in 1935, to a design by Aymas Phillips. The main house has a courtyard plan on an intimate scale, and is built in a late medieval style, incorporating many imported medieval features. The doorway of the thatched guest-house nearby incorporates several carved 12thc. stones, reportedly retrieved from the walls of a Georgian farmhouse on the Bailiffscourt site. The implication is that the stones had been reused in the walls of that building.

Aslet states that the doorway to the guesthouse was 'reconstructed from fragments found in the walls of the demolished farmhouse'. The farmhouse was Georgian.

The stones are very similar to those on the outer order jambs of the S transept doorway at nearby Clymping. They may have been removed from Clymping during a programme of restoration and reused in the farmhouse at Bailiffscourt.