The castle was built on a former outpost of the Roman era that dominated the Via Tiburtina Valeria. Its history is characterized by disputes between different families and for centuries was the property of the Colonna family. For a short period of about 11 years, Pope Boniface VIII gave the Riofreddo fief to the Orsini but on his death Riofreddo returned to the Colonna who became extinct in 1459. Around the middle of the sixteenth century, the Riofreddo castle was sold to Del Drago and Marchesato. The decline of the castle occurred quite recently in the 1960s.