The Robert Stuart House was built in 1817 as the "Agent's House," housing the resident agent of the American Fur Company, which was at that time Ramsay Crooks. In addition the building housed other agents and clerks. The Agency House was part of a four-building complex constructed to house the American Fur Company's offices. The other three buildings were a clerk's quarters (now demolished), a warehouse built in 1810 (now the Community Hall), and a trading post (subsequently altered, but later restored).
Also in 1817, Robert Stuart arrived on the island as Crooks' assistant. Upon his arrival, Stuart was housed as a guest in the Agent's House. In 1820, Crooks moved on an Stuart was appointed to succeed him as resident agent, a position he held for the next 14 years. Because of Stuart's national prominence and his lengthy association with the Agent's House, the building is nominally referred to as the Robert Stuart House.
The 1820s and 1830s were boom years for the American Fur Company's operation on Mackinac Island; in 1822 more than three million dollars of furs were cleared through the Mackinac Island operation. Given his prominence, it was natural that Robert Stuart's house served as the social center of the island during this time. However, the fur trade began to decline in the 1830s, and in 1835 Stuart moved on to Detroit.
As the fur trade declined, Mackinac Island became a resort community. The Robert Stuart House was used as a boardinghouse in the years before and during the Civil War. In 1871, the entire American Fur Company complex was purchased by James F. Cable and the three main buildings - the Agent's House, warehouse, and clerk's quarters - were linked with palisades and turned into the premiere island hotel of the time, the John Jacob Astor House. It remained the social center of the island until the construction of the Grand Hotel.
In 1900, the complex was sold to the city of Mackinac Island. It was operated as a hotel until 1929, but in 1941 the building was separated back into individual units. The Robert Stuart House is now known as the Stuart House City Museum.