Withersdale is nearly 3 miles E of Harleston and a mile and a half from the river Waveney, which marks the border with Norfolk. The church stands alongside the B1123 and the moated hall site, with a medieval farmhouse, is 500 yards (457 metres) to the S. The rest of the village has migrated W along the road towards Harleston, forming the settlement of Withersdale Street.

The church is a two-cell building, largely of flint. The nave is tall with a wooden belfry on the W gable. A blocked N doorway, converted into a window, indicates a 12thc. date, but the S doorway, pointed and chamfered, and small pointed lancets in the lateral walls suggest a major 13thc. remodelling. Lateral two-light windows were added at the E end of the nave in the 15thc. Inside there is a N rood stair. There is no chancel arch and the chancel, slightly lower than the nave, has a 13thc. S doorway and plain pointed lancets in the side walls at the W end. Further E the lateral windows have Y-tracery, suggesting that it was lengthened c.1300. The E wall is 18thc., of brick part-rendered with a window with wooden glazing bars. An oblong W window, bargeboards at E and W and on the S porch, and a mortar render on the S side combine to give the church a domestic look. The highlight for the student of Romanesque sculpture is a damaged but elaborately carved font.

In 1316 the manor was vested in Oliva de Ingham, who died seised of it in 1343. In 1347 a fine was levied on it and the advowson by Alexander, parson of Horham church and others against Alexander de Walsham and Margaret his wife. Shortly after this the manor was acquired by Sir William Jermy, who died in 1385, from which time it passed in the same course as the manor of Metfield, remaining in the Jermy family until the extinction of the male line in 1652.

Benefice of Fressingfield, Mendham, Metfield, Weybread and Withersdale.

There is no close local parallel for the font decoration. The cut-down font at St Mary's, Naughton has intersecting arches, and the more elaborate font at SS Mary and Lawrence, Great Bricett has varied arcaded designs on its four faces, including trefoil and intersecting arches, but these two are some 30 miles away to the SW. The font at All Saints, Great Thurlow has arcaded faces and hollowed out or shafted angles, but is not really similar and is at the far SW of the county. Dating is problematic. The finely carved and accurately laid out trefoil-headed pointed arches of the S face suggest a date after 1200, whereas there is nothing on the other faces that could not have been carved in the 1140s or '50s. It is possible that the font was remodelledc.1200 or later (Pevsner suggests c.1300), but if so it is surprising that the opportunity was not taken to complete the carving of the N face at the same time.