The lord of the manor had his hall, which was the administrative centre for the manor, and he cultivated a home farm which was set out around the hall and contained within hedged fields. In the manor of Halesworth, the Hall once stood behind the Church of St.Mary somewhere on or near the site of the present Church Farm, whose lands would have been part of his demesne lands of the manor. Many lords of the manor such as Halesworth, owned windmills, had the right of choosing the parson, of having stocks, pillories and even gallows.

While much of the land was in open fields, other was in the form of pasture, hay-fields, marshes, fens, commons and woods. All land transactions took place in the manor court, and peasants might hold land in exchange for rent and labour as they worked part of their time on the lords demesne lands. His work might consist of ploughing, seeding, harvesting, in the scattering of manure, weeding, mowing the lord's meadow, cutting and binding hay, reaping, threshing grain, carrying grain, beans & peas, making malt, digging or clearing ditches, making hurdles or hedging and thatching.

The Halesworth peasant had to take his corn to the lord's mill to be ground into flour. There was one mill recorded in 1086, but by the 19th century there were eight known to have existed. They were known as Gothic Mill (1837), Broadway Mill (c.1844), two mills in Pound Lane both built 1743 and demolished 1900 and 1905. Pound Lane is better known to us as London Road, although it was also later called Pound Street. This is where the pound stood, a place where stray animals could be placed until they were fetched by their owners. One of the cottages which are in the group leading up to the DayCentre and Waveney District Offices is still called Pound Cottage. Another mill was Calver's Mill, which was built in 1788 and was demolished in 1942, while lastly, there was Mill Hill windmill, built in 1788 which stood in Mill Hill Street, a lane which runs from the corner of Rectory Street to meet Wissett Road.

The villager also had to pay a sum of money called a 'fine' when a house or land was transferred from father to son - also called 'tallage' - and on the death of a villager another 'heriot' or fine was expected to be paid. In the early days he might also have to pay a sum of money as 'gersum' on the marriage of a daughter, or even be forced to pay 'childwyte' if his daughter had a child out of marriage.

During the 12th century, waste land was being cultivated with the woods also being felled and scrubland cleared. By the end of the 12th century there was little land not in use.