I thought this article was interesting and helps show how difficult to weigh up real history can be.
William Ellison Jr., born April Ellison (c. April 1790 – December 5, 1861), was a U.S. cotton gin maker and blacksmith in South Carolina, and former black slave who achieved considerable success in business before the American Civil War. He eventually became a major planter and one of the medium property owners, and the wealthiest black property owner in the state. According to the 1860 census (in which his surname was listed as “Ellerson”), he owned 53 black slaves, making him the largest of the 171 black slaveholders in South Carolina. He held 40 slaves at his death and more than 1,000 acres (400 ha) of land. From 1830-1865 he and his sons were the only free blacks in Sumter County, South Carolina to own slaves. The county was largely devoted to cotton plantations, and the majority population was slaves.
Ellison and his sons were among a number of successful free people of color in the antebellum years, but Ellison was particularly outstanding. His master had passed on social capital by apprenticing him to learn a valuable artisan trade as a cotton-gin maker, at which Ellison made a success. He took a wife at the age of 21. After buying his own freedom when he was 26, a few years later Ellison purchased his wife and their children, to protect them from sales as slaves. The Act of 1820 made it more difficult for slaveholders to make personal manumissions, but Ellison gained freedom for his sons and a quasi-freedom for his surviving daughter. During the American Civil War, Ellison and his sons supported the Confederate States of America and gave the government substantial donations and aid. A grandson fought informally with the regular Confederate Army and survived the war.