The unassuming and slightly dilapidated structure behind the grand Bayly House at 207 High Street in Cambridge could be quite a historically significant little building. According to information found in the Maryland Inventory of Historic Properties, the Bayly House is probably the oldest dwelling in Cambridge, dating from the 1740s. But what has the attention of historians, architects and archaeologists now is the small structure in the backyard. They are working to document its history and verify whether it was once used as slave quarters. If confirmed, it would be a rare finding for an urban setting.

It is believed that the house was moved from Oxford, Maryland, and assembled in the 1750s on the site where the Dorchester County Courthouse currently is located. Shortly afterwards, it was moved across High Street to its present location to allow the Courthouse to be built. In the rear of the house are several outbuildings, one of which could be a slave cabin.

Architectural Historian Stephen Delsordo has evaluated the building and has stated that it was constructed prior to the middle of the 19th century by reusing material from older buildings, along with new building material. And according to Bayly family tradition, the outbuilding served as quarters for the house servants. In addition, Kate Larson, historian and Harriet Tubman scholar, has confirmed that the John Caille family (the original owners of the house) and the Bayly family, particularly Josiah Bayly and his son Dr. Alexander Hamilton Bayly, owned slaves. The property was purchased by the prominent Bayley family around 1830, and occupied by Dr. Bayley (1814-1892) and his descendants until 2003.

In addition to the small building described as a slave cabin, there is an attached kitchen and smokehouse.