In the early 13th century, Bernard de Casnac, the lord of the manor, held much power in the region. However, he was also a supporter of the Cathar faith, also known as Albigensian, and showed extreme cruelty towards Catholics. In 1214, the renowned knight Simon de Montfort led a crusade against the Albigensians. Because of Bernard de Casnac's actions, he captured the Castelnaud castle and held it for a couple of months. Nevertheless, in 1215, Bernard de Casnac regained possession of his property, only to see it burned down a few months later under the order of the Archbishop of Bordeaux. Of this fire only a few stones remained intact. Then, the castle was reconstructed, but in 1337, the Hundred Years' War erupted between France and England. During this conflict, the castle sided with the English. It thrived as a significant power in Périgord, much like Beynac Castle, its neighbouring rival. The barons of Beynac were supporters of the French. Over the course of just over a century, the castle changed hands seven times.
After the French emerged victorious in the war, the Caumonts reclaimed their property and the castle was rebuilt once again. Castelnaud continued to serve as the central hub of the seigneury and adapted to the military practices of the time, as evidenced by the artillery tower constructed in 1520.
In the 16th century, the Caumonts embraced the Protestant faith. Geoffroy de Vivans, a Huguenot captain born at Castelnaud castle, fiercely defended the town and fought against the Catholics, earning a fearsome reputation throughout the region. Because of this, Castelnaud remained untouched during the Wars of Religion. In the 17th and 18th centuries, the Caumonts enjoyed great prosperity and received numerous prestigious honours, gradually forgetting about their old castle in Périgord, which was too distant from the royal court. Following the French Revolution, the site fell into disrepair, overrun by vegetation and becoming a neglected relic.
The castle was even used as a quarry, which didn’t help improve matters. In 1832, when river transportation and the development of the village of Castelnaud necessitated the construction of a port slipway, masons found it more convenient to dismantle stones from the southern section of the castle rather than cutting them to size. In 1966, the castle was classified as a Historic Monument upon the request of its new owners, Philippe and Véronique Rossillon.
Today, the castle houses the Museum of War in the Middle Ages. Inside there are reconstructed war machines adorning the bastion of the castle. This museum generally showcases a variety of weaponry, including spears, swords, crossbows, and much more.