Located at the southern edge of the village, SS. Peter and Paul is primarily a 13^th century parish church consisting of a nave, N and S aisles, chancel and clerestory; 14^th-century W tower; vestry added W of the porch in 1982. The baptismal font is from the 12^th century.

In the 11^th c. the village name, Castretone, was derived from the existence of a Roman camp NE of the church. Though neither Little Casterton nor Great Casterton are mentioned as distinct villages in 1086, Domesday Book does list Casterton in two entries. One entry is for land belonging to the king and the other for land belonging to David. Under both entries a priest is noted among the residents.

The E face of the font has two characteristics that set it apart from the other three faces: its divergence in an overall, consistent design motif and the deeper scoring of the face into four quadrants. Does this perhaps suggest that the E face was the starting point of production as the artisan worked out this elaborate, abstract geometric motif? The geometric motif here links this font to three other fonts in neighboring Leicestershire, those at St. John the Baptist, Hungarton, St. John the Baptist, Rothley, and St. Mary Magdalene, Pickleton. While the Hungarton font is square and seems to be a clear copy of the Great Casterton font, the two at Rothley and Pickleton, though they carry the same geometric, lozenge motif, are round drum fonts with no roll mold edging.