The round tower stands in the SW section of the churchyard, isolated from the church. It is built of coursed dressed-ashlars on both the interior and exterior faces, with a change in stone type observable above the lower section. The only significant elements are on the exterior: the raised N doorway and the four window openings of the highest level. There are no stringcourses on the exterior face, except for that at the top of the structure, which appears to be of later date. Only a few small plain openings light the the lower interior levels. Gordon stated in his published itinerary of 1726 that the interior of the tower was 'hollow' like a 'deep draw well'. In a letter to his sister in 1760, Pococke reported that the tower had seven stories and that the internal 'floors extend to rather more than three quarters of the circle, the rope of the bell coming down by the open space'. With no stairway built into the interior, it seems that ladders were the only means of ascending the various levels. The N doorway, which inclines slightly towards the top is heavily weathered and shows no distinctive decoration, while the upper window openings have been covered with some sort of protective coating. In the Statistical Accounts of 1794, it was stated that the church, which stood a short distance NE of the tower, 'is remarkable for nothing but its antiquity' and that there was no record of when it was built. However, in Pococke's letter of 1760 he wrote that the church was small and had a door of 'the plainest Saxon architecture' (i.e. round-headed). He wrote that it was also believed that the 'great church' had been built to the NE of this, but no longer existed. The church which was still existent at that time was taken down in 1802 and a new church built north of it, apparently using stones from the earlier church. In 1821, excavations of the base of the tower discovered a skeleton and green urn buried inside the tower, below which were flagstones and more skeletons. Brash drew a section of the tower for his article of 1862, which showed six floors built onto the interior stone stringcourses, separating the tower into five main areas. There were, as well, a basement level and top parapet. Butler (1893) reported that the Earl of Home had introduced a spiral staircase into the tower so that visitors could climb to the top, and that some repair work had been undertaken. This would have required the removal of the previous floorings. The present metal spiral staircase fills the entire interior space. New stones from various repairs are evident in the stonework.

Abernethy appears to have been a religious site at least as far back as 6th century, with its church dedicated to St Bridget/Brigid. Three stones with early medieval decoration have been found within the churchyard over time. King Necton is said to have founded Abernethy in the 5th century, but there is some uncertainly about what exactly this means. For a period of time from the late 9th century until the second half of the 10th century, Abernethy appears to have served as the centre of a bishopric. Culdees had arrvied in Abernethy by the late 11th century (and probably earlier), but the actual date when they were established at Abernethy is unrecorded. Malcolm III, king of Scotland, met with William the Conqueror at Abernethy in 1072, and among the witnesses to a grant of 1093 X 1107 by Ethelred, son of King Malcolm III, are listed: Malnethte filii Beollani sacerdotum de Abyrnethyn’ and ‘Berbeadh rectoris scolarum de Abyrnethyn’ (from Early Scottish Charters, no. XIV). Sometime between 1189 and 1199 the church of Abernethy was granted to Arbroath Abbey, with the exception of those tithes that pertained to the Culdees. In 1239/40, the altarage was ceded to the bishop of Dunblane. Eventually, in 1272/3, Abernethy was made a priory of Canons Regular. Bagimond’s Roll, in the 1270s, lists the church of Abernethy for the Abbot of Arbroath as 6 marcs, while the priory of Abernethy is listed separately as 33 shillings and 4 pence for the first year. For the second year, the lists are shown as 6 marcs 4 shillings, and 6 marcs respectively. The vicarage was joined to the episcopal mensa of Dunblane about 1425, and sometime before 1464 (between 1427 and 1465) it had become a prebend, with no further mention of Canons Regular to be found after this. Following the Reformation, the church seems to have become no more than a parsonage.

In the Old Statistical Accounts (1794), William Duncan, minister of the church at Abernethy, stated that the Picts were responsible for the construction of the round tower, a suggestion made in many of the early references to the tower. In the New Statistical Accounts of 1845, David Duncan, minister, was uncertain and referred to more recent speculation and theories. He also noted that a skeleton and several human skulls had been found buried within the tower. In 1848 (Muir, Descriptive Notices, p. xxx), the north doorway was said to have been 'extensively modernised' and was described as a 'plain round-headed doorway with rectangularly-recessed jambs'. The upper windows were described as having 'two round-headed orders, the inner one square-edged, ... the outer one moulded with a quirked edge-roll, and supported in each jamb by a cylindrical shaft with base and capital of decided Norman character and date'. Brach, in his article published in 1862, wrote that he felt that the tower was of two dates, that the original tower must have become ruinous in the 11th or 12th centuries, which caused the tower to be partially rebuilt, at which time the doorway and upper windows were enlarged. MacGibbon and Ross stated that the tower 'shows the impress of the Norman influence in its details (vol. 2, p. 178), while Radford (1942) suggested a date 'probably after 1050'. Eric Fernie (1986) proposed a date around 1100, which modern scholars generally accept.

It is possible that during the medieval period there were two churches within the present churchyard, one for the parish and one for the clergy, but there is a lack of factual information to confirm this. Foundations of other buildings have certainly been found during various excavations.