When a good man lends himself to the advocacy of slavery, he must, at least for a time, feel himself to be any where but at home, amongst his new thoughts, doctrines, and modes of reasoning.

A staunch abolitionist he was a member of the Secret Six who financially supported John Brown's raid at Harpers Ferry West Virginia. Besides making substantial donations of both land and money to the African-American community in North Elba New York he was involved in the Temperance Movement and later in life the colonization movement. Smith a significant financial contributor to the Liberty Party and the Republican Party throughout his life spent much time and money working towards social progress in the nineteenth century United States.