When Richard Abrincis succeeded as Earl of Chester, he also succeeded to the Manor of Halesworth. William the Conqueror was his great uncle and in time he married Maud, the daughter of Adela (William I 's daughter) and Stephen of Blois, - so was the sister of the future King Stephen. The present King, Henry I, had a son Prince William, who in 1120 boarded a fine new vessel called the 'White Ship', with a large number of barons and nobles. With the royal party were Richard and Maud, the lord and lady of the Manor of Halesworth, and all in good spirits as they set out for France. Suddenly the ship struck a rock and a gaping hole was torn in the hull and the ship sank in minutes. The only survivor being a servant who swam to safety to tell the tale.

In Halesworth, the sudden death by drowning of Richard and Maud must have come as a great shock, and we don't know quite what happened to the succession of the Halesworth Manor. However, some thirty years later, in 1150, the new lord of the Manor, aptly named Thomas de Halesworth (Thomas of Halesworth) died, leaving his estates, including the Halesworth Manor to his daughter Rose de Halesworth. We also know that Rose married Reginald de Argentein and started a family which was to control the future of Halesworth for almost three hundred years.

When Henry I died, it was his wish that his daughter Maud should succeed him as Queen of England, but a group of powerful barons were not happy with a queen as ruler, so sent an invitation to Stephen, Henry's nephew, to claim the throne and become the next King of England. The people of Halesworth were most probably very happy with this solution, as this Stephen invited to be king, was the brother of their late Lady of the Manor, and so might look favourably on the town.

In making his next move, Stephen was helped by Hugh Bigod, the Earl of Norfolk, for he perjured himself by saying that Henry I on his deathbed had changed his mind and favoured Stephen instead of his daughter Matilda. So after much discussion Stephen was crowned in 1135. Hugh Bigod also had local connections, not only by owning Bungay Castle, but his predecessor Roger Bungay had taken over the 15 acre manor in Halesworth owned by the Saxon Gunner in 1066. The family is commemorated by the naming of Bigod Close which is off Lansbury Road in Halesworth.

Hugh Bigod was soon causing trouble for King Stephen by revolting against his rule and seizing Norwich Castle. Stephen came in person to lead an army into East Anglia to recapture Norwich Castle and also sent troops to take Bungay Castle for the Crown. As Geoffrey de Mandeville had also rebelled and captured Ely, Stephen built a chain of motte and bailey castles across East Anglia from Suffolk to Huntingdonshire in order to contain the rebels.

Maud landed in Sussex in 1139 to make a claim for the throne, so Hugh Bigod quickly changed sides and was found in command of the right wing of Stephen's forces at the Battle of Lincoln in 1141. In 1148 Matilda finally gave up her claim and left England to Stephen, with the provision that her son Henry would succeed on Stephen's death.

With peace declared, Hugh Bigod quickly resumed his rebellion against Stephen, capturing Ipswich Castle in 1153. Stephen and his son Prince Eustace came to Suffolk to besiege Ipswich, but Eustace caught a fever and died in the army camp. Although Ipswich was captured by Stephen, he died the next year in 1154, and Duke Henry came to England to be crowned as Henry ll.

He came to restore order and help the country get over 20 years of war, but he was soon fighting Hugh Bigod, who was a plotter who could not be trusted. He was powerful in East Anglia, as he had castles at Framlingham, Bungay, Walton and Thetford. Over the years Henry ll kept up a running battle against him until in 1174 Henry besieged him in his castle at Framlingham and Hugh Bigod surrendered. Bungay castle was next, defended by 500 men. The king's sappers dug a mine under its defences and parts of the wall collapsed and Bungay Castle was left devastated. The King's engineers destroyed Framlingham Castle, filled in the moat, and also destroyed Ipswich Castle to stop it falling into rebel hands. Hugh Bigod was fined 1,000 marks and his powerbase was destroyed, just as the King desired.