Although the cathedral and the episcopal palace were probably the most elaborate buildings on the site, it was the castle that was at the heart of Old Sarum and the reason for the complex’s existence in that location. The original building was a motte and bailey castle on an earthwork. The castle, which was always intended as a temporary measure, was originally held by the King, but by c.1130 it had passed to Bishop Roger who replaced many of the early, purely-defensive structures with a substantial, fortified house, the ‘Great Tower’ and Herlewin's Tower in the northern half of the inner bailey. The Great Tower was first mentioned in 1130-1 and by the 13th century it was probably three storied.

The house had four ranges enclosing a rectangular, paved, inner courtyard. The major chambers were on the first floor, with the Great Hall occupying the west range and the Great Chamber the north one. The eastern end of the south range was occupied by St Margaret’s Chapel on the ground floor with St Nicholas’ chapel above. At the north end of the east range there was a ‘kitchen tower’ and an east turret holding further garderobes. Although Roger’s house was substantial and elaborate in plan, it does not seem to have been embellished with the same level of sculptural detail as the cathedral. Where sections of the walling survive, the remnants show that they were reinforced with pilaster buttresses and in the excavations fragments of stone shingles and ornate, red and green glazed ridge-tiles were discovered. The published reports suggest that the doors and windows were decorated with chevron, while it was surmised that the upper windows were subdivided by stone shafts with spiral grooves and other patterns. The chimney, which is the most elaborate surviving feature from the Great Tower, was discovered during the 1910 excavations and illustrated in the following year's report. The Great Tower and the adjacent narrow Postern Gate, as well as the east gate, are the only significant, but ruinous, structures surviving from this period.

Many fragments in the English Heritage stone store and the Salisbury and South Wiltshire Museum may originate from the early 12th century fortified house that succeeded the early castle. The most striking feature was the large chimney, which has been reconstructed in the museum (museum entry 80). For details of these fragments refer to the Salisbury and South Wiltshire Museum entry.

For a history of the site of Old Sarum, see the entry for Old Sarum Cathedral.

The cathedral at Old Sarum was abandoned in the 13th century, but the castle remained in use though there are no indications of new, major building works. In 1247 a wheelhouse was built above the well, and the hall, the gaol and two kitchens in the castle were repaired. In 1315 £60 of repairs were carried out followed in 1337-8 by repairs to the towers and walls, particularly the Great Tower. In 1366 a detailed indenture was compiled concerning the repair of the Courtyard House. Although some repairs were being undertaken the castle was still decaying. By 1307-8 the hall, chamber, kitchen and bakehouse had ceased to be used. On 29 November 1399 a commission was appointed 'to inquire into divers wastes, dilapidations, and destructions committed in the King's Castle of Old Sarum.' The introduction of artillery rendered the castle militarily ineffective and by 1446-7 it was described as 'now fallen into decay'. In 1514 Thomas Crompton, Groom of the Chamber, was granted the right to use the castle for building materials.