It is the safeguard of the strongest that he lives under a government which is obliged to respect the voice of the weakest.

They returned to Philadelphia where their family was among the black elite. In 1828 Purvis' older brother William died of tuberculosis which resulted in Robert and Joseph's having increased shares of the estate. His father was the English immigrant William Purvis.

He was born in Charleston South Carolina educated at Amherst College and lived most of his life in Philadelphia. Purvis and his brothers were three-quarters European by ancestry and inherited considerable wealth from their native English father.