The Domesday Book records that in 1086 the manor was held by Hervey. By the end of the 12thc it was a possession of Queen Berengaria but in 1204 its ownership passed to Hugh V of La Marche and his wife Isabella of Angoulême.
The hospital was first mentioned in 1195 when its prior Hubert Claud was recorded as the tenant of some lands in Ditchampton and Bemerton. A chapel in the hospital complex was probably dedicated in 1217. In 1388 the ruinous conditions of the building led John Waltham, Bishop of Salisbury, to announce a 40-days indulgence in favour of the hospital.