Nothing of the abbey church remains in situ, and the surviving abbey buildings all postdate the 12thc. In the Abbey Gardens, where the church once stood, is an artificial ruin constructed of fragments from the site. Reset in the S wall of this are the voussoirs described below.

The abbey was founded in 675, and the church rebuilt after its destruction by the Danes by Bishop Aethelwold between 954 and 963. It is known to have been rebuilt again, largely under Abbots Rainald (1084-97) and Fabritius (1100-17), but Fabritius's church was not dedicated until 1239.

The voussoirs formed part of an arch of considerable size. The best estimate, which is necessarily extremely speculative, is of the order of 8-9 m inner diameter, quite large enough for the main arcade of the abbey church, or even for a crossing arch.