Behind the hotel is a dry-stone wall constructed of loose pieces of Barnack limestone, discovered in the garden of 35a Southgate St, adjacent to the hotel. The site is close to that of the former White Hart public house, in the grounds of which once stood St Botolph's Chapel (see History). The stones form a retaining wall for a raised garden.

The stones have sometimes been connected with St Botolph's chapel, which formerly stood in a nearby location. The history of the chapel is supplied in its own site report.

The most interesting feature (now lost) is the waterleaf capital. No dimensions were available but it is clearly very small - perhaps window-sized. This should be dated to the 1170s or '80s. Voussoirs 1 and 3 with their fat angle rolls are likely to date from earlier in the 12thc than voussoir2, with its more complex mouldings and curved rolls.

The fact that the stone is a valuable limestone from a distant quarry in the Soke of Peterborough makes it unlikely that it was originally intended for the humble chapel of St Botolph. The illustration that we have shows a small, simple building, while the stones described here point to a number of large or intricate features. In fact it seems beyond doubt that these are stones from the dissolved abbey, perhaps transferred to St Botolph's Chapel for use in repairs after the Dissolution.