The leasing of the site for the fort was managed in 1631 by a displeased former employer of the Dutch West India Company called Arent Groote. Arent Groote had persuaded Chief of the Fanti state to allow exclusive trade rights to the British. Ambro Braffo (chief of the Fanti) then built a lodge to solidify their claim to the area.
In 1640, the lodge was burnt down and the blame was on the Dutch. A strong new fort with a three storey apartment was then built by the British. The Royal African Company became owners of the fort in 1661. The fort then became the headquarters of English Gold Coast activities.
In 1665, Dutch Admiral Michiel Adriaensz De Ruyter captured the fort, reconstructed it and renamed it as Fort Amsterdam. In 1811, the people of Anomabu (who were English allies) destroyed the fort. The first slave prison on the Gold Coast was believed to be in the fort's hollow southeast bastion. The fort’s opening hours are 9:00am to 4:30pm.