The church was to a large extent rebuilt in the 17th and 18th centuries, but some of the 12th-c wall still survives, including 2 blocked windows in the E wall and a blocked window in the N wall of the chancel. There is also a later blocked doorway in the S wall of the chancel. This appears to have replaced another doorway (now a window) west of it. The original plan of the church consisted of a rectangular nave and smaller rectangular chancel at the east end. A loose stone with carving on one side is kept inside the church. The interior of the church was remodelled in the 20thc.

David I granted the manor of Smailholm to David Oliford, who then in 1160 granted a carucate of land there to the monks of Dryburgh. In the 1170s, Smailholm church was dependent on Earlston, but not long afterwards gained parochial status. Walter Olifard (d. 1242) gave the church at Smailholm to Coldingham Priory. In Pope Urban III’s confirmation charter for Durham (1185-87) the church at ‘Smalhom’ is listed as one of Durham’s possessions, which would have been the result of Coldingham being a cell of Durham. The church at Smailholm was one of the many churches consecrated by David de Bernham, bishop of St Andrews, this undertaken in April 1243. In 1288, Pope Nicholas IV confirmed that David Olifard (presumably a later descendant) had given the patronage to the chapter of Glasgow. But at the end of the 13thc the church passed to the Moray family, lords of Bothwell, who contested this grant. In 1292/3, the chapter of Glasgow conceded its rights in exchange for the church at Walston. The patronage then passed, in 1362, to the earls of Douglas through marriage. They retained it until Archibald, 4th Earl of Douglas, granted the church to Dryburgh. This was confirmed by the pope in 1420. In 1451 the baronies of Smailholm, Hawick, Bedrule and Spouston were erected into a free regality for William, Earl of Douglas.

There was also a hospital within the manor of Smailholm, the property of the abbey of Dryburgh.

Not much of the original detailing survives, except for the blocked windows. The sills on those of the E wall may originally have been the tops of other windows, unless carved at a later date as a decorative feature. The one surviving medieval stone with carved decoration is kept inside the church. This is undated but may be late 12th or 13thc. It seems to have been carved for a corner but its precise use is uncertain.