A small motte covered in trees with a bailey of about a quarter of an acre is on the eastern edge of town; no sign of habitation has been found (Frankland 1948/9). The church is at the opposite end of the main street from the motte.
In Sedbergh, 3 carucates had been held by Earl Tostig (VCH II, 207).
Sedbergh was a vill of Whittington-in-Amounderness at Domesday and belonged to the king, afterwards it belonged to the Mowbrays (VCH Yorkshire II, 37; Frankland 1938, 293).