The area around St Peter's is historically one of the most interesting in the town. On College Street stood the Augustinian Priory of St Peter and St Paul until 1527, when Cardinal Wolsey founded his Cardinal College of St Mary on the site. It was not completed, but a gateway survives. St Peter's Street itself runs S from the town centre and boasts a good collection of timber-framed shops and houses. St Peter's is at the southern end of the street, at an intersection of the inner ring road. Beyond it to the S are derelict waterfront warehouses standing on the dockside. Its present position is by no means attractive, therefore, but work is under way on the regeneration of the waterfront, and St Peter's is likely to benefit from them. It was made redundant in 1979 along with three other town centre churches, and the Ipswich Historic Churches Trust was founded at the same time to ensure their maintenance and preservation. In 1981 these four churches, St Lawrence, St Peter, St Clement and St Stephen, were passed to the Borough Council by the Church Commissioners for a nominal sum and then offered to the Ipswich Historic Churches Trust on long leases. The intention was that the Trust would undertake repairs and find appropriate new uses.

The church is largely 14th -15thc., with a four-bay aisled nave with 14thc. arcades and aisle windows. It has N and S doorways, the S under a 15thc. porch. The N clerestory is 14thc. with sexfoils, the S 15thc.. The chancel is early 15thc. and Scott extended the aisles alongside it in 1881 and he was also responsible for the great E window. A surpassingly unattractive vestry has been added at the E end, reached through the S chancel aisle. The 15thc. W tower is of knapped flint with diagonal buttresses and a polygonal N stair turret. It has a battlemented parapet with flushwork ornament. The N aisle windows are all boarded up, and other windows are protected by wire mesh grilles except those in the porch, which are broken. The church is home to an important Tournai marble font.

The Austin Canons' Priory of SS Peter and Paul was established in the reign of Henry II (1154-89), probably towards the end, traditionally by the ancestors of Thomas de Lacy and his wife, Alice. Little is known of its early days, but by 1291, the taxation roll shows it to have possessed the Ipswich churches of St Peter, St Nicholas and St Clement, and the rectories of Cretingham, Wherstead, and part of Swineland. In 1344, Thomas de Lacy and his wife, Alice obtained a licence to alienate land and the advowson of the church at Duxford (Cambs) to the priory. The priory was marked for suppression in 1527, when Wolsey formulated his design for a Cardinal's College, but this was never built and the site of the monastery was granted to Thomas Alvard, a gentleman-usher of the King's Chamber.

This former parish church is now in the care of the Ipswich Historic Churches Trust.

The so-called Tournai marble is a dense carboniferous limestone quarried on the banks of the river Scheldt near Tournai and either exported as freestone for decorative carving (as at Lewes Priory (Sussex)) or worked nearby and the products, mostly fonts, exported. This vigorous industry extended from the 12thc. to the end of the 15thc. An English group of seven Tournai School fonts was established by Allen and Kitchen in 1894 articles. They were: St Mary Bourne, East Meon, St Michael’s Southampton and Winchester Cathedral (all Hants), Lincoln Cathedral and Thornton Curtis (both Lincs), and St Peter's Ipswich (Suffolk). Also in 1894, part of the bowl of an eighth font was discovered at Ipswich and is now in Ipswich Museum. Since that date, three other fonts have been attributed to the group: Boulge, Romsey Abbey (Hants) and Iffley (Oxon) - see Eden (1909) and Dunning (1944). More recent scholars, notably Drake (2002) and Anderson (1999) have cast doubt on these attributions. According to Drake, the Romsey Abbey font was said to have been destroyedc.1850 during a restoration, but there is no other evidence that it ever existed. The Iffley font is of black limestone but is uneven in shape, undecorated and unlike other fonts in the Tournai group. As for the Boulge font; Drake asserts that the finish of the bowl is too smooth for decoration to have been chiselled off it (as suggested by Eden), and points out that the bowl is too tall for its width, in comparison with genuine Tournai School products. Lions are a common motif on Tournai fonts, being found on 15 out eighty or so known examples throughout Europe (Drake, 53). An attempt to explain the peculiarity of their arrangement here, with one central lion facing left where the others face right, was made by Blatchly and Haward, who proposed that the scheme was to avoid having a central lion with its hindquarters towards the altar in the original arrangement. This seems untenable to the present author: if this kind of thing was important to the patron, he would surely have arranged that none of the lions presented a posterior view. The use of shafts supporting a horizontal lintel to articulate the face and mark the angles is also found in the font at Vermand (Aisne) in northern France. It is normally dated late in the reign of Henry II i.e.c.1170-90, and was almost certainly the original font of the Priory of St Peter and St Paul.