New Jersey Local History: Stephen Smith House and Underground Railroad at Cape May

Stephen Smith was the original owner and builder of a summerhouse in Cape May, New Jersey.
Although born in the late eighteenth century, Smith was a nineteenth century philanthropist. During his lifetime, he was said to be the wealthiest Black American in the nation. He established the first home in the United States for aged and indigent Colored people. This home is still in existence today.
An energetic businessman, Mr. Smith had a coal mine, stone quarry and lumber yards in Philadelphia and Columbia, Pennsylvania. His business activities led to many real estate holdings. He established Smith, Whipple & Co. with his cousin William Whipper. Stephen Smith was a devoted church and family man; married to Harriet Lee, they lived in a stone mansion on Lombard Street in Philadelphia, Pennsylvania. The vacation house was built by him in Cape May, New Jersey using material from his lumber yard. This house is still standing except for the carriage house, kitchen and fireplace room once visible in the rear of the building. Not having children of his own, he welcomed the offspring of his wife’s secretary and friends; namely the Bascoms and Harlans. His secretary Anna Vidal, wife of Ulysses B. Vidal, brought their three children, Etienne, Marie and Anna Clorise to Cape May. Etienne engraved his name in a small glass windowpane with his diamond ring at this house. In later years Marie would point out the pane and recall the prank, for Etienne had long since emigrated to France. Stephen Smith was called “Daddy” Smith by the children and he relished the time consuming carriage ride to Cape May with them. Portraits of Stephen and Harriet were painted by the noted Black artist Edward Stridom and are today the property of the Pennsylvania Historical Society. An avid abolitionist and agent in the Underground Railroad, Mr. Smith was active in politics and church affairs. Records of the AME church attest to his generosity.
Stephen Smith was born October 13, 1795 in Columbia, PA. At the age of nine he was purchased by an officer who was a Revolutionary War general named Thomas Boude. His mother escaped from her owner, found her son, and was taken in by Boudes. Weeks later her mistress (owner) followed and demanded her property. The Boudes’ refused and were supported by the townspeople who believed in aiding a fugitive. Raised by the General Thomas Boude, in time, Stephen purchased his freedom. Before the general died, he set Stephen up in the lumber business. Stephen earned the respect and confidence of the people of Columbia, PA and they supported his endeavors. At age 21, he was inspired by the eminent minister, Richard Allen, founder of the AME church, and at age 31 became licensed to preach. He was a Teller in the election of every Bishop since Richard Allen until his death in 1873. Smith built a public hall in Philadelphia for the use of the “People of Color”; but this was destroyed by fire in the riots of 1842. Stephen Smith was one of the Signers of Frederick Douglas’ Men of Color to Arms appeal during the early part of the Civil War calling on Black Americans to join the Union Army.