Four manors were recorded in Harlow in 1086. The first was a manor, originally of 1½ hides, that was held by Bury St Edmunds abbeybefore and after the Conquest. To this manor, KIng William added 3 hides that had belonged to 5 free men in King Edward's time. Half a hide that Beorhtmaer, a free man, held before the Conquest was held in 1086 by Geoffrey from Count Eustace, and Turgis held a manor of 1 hide and 3 virgates from Eudo the Steward that Godwine held in 1066. Finally 1 hide in Harlow was held as a Berewick of Roydon in 1086. The tenant in chief was Ranulf, brother of Ilger and Richard held it from him.
The relevant manor for Harlowbury was clearly that belonging to Bury St Edmunds, known as Harlow or Harlowbury manor, which remained in the abbey's hands until its Dissolution in 1539.