During the Memphis race riots in 1866 Mary's father was shot in the head and left for dead. He survived the attack and eventually became a successful businessman. He speculated in the property market and was considered to be the wealthiest black man in the South.
Mary was an outstanding student and after graduating from Oberlin College, Ohio, in 1884, she taught at a black secondary school in Washington and at Wilberforce College in Ohio. Through her father, Mary met Frederick Douglass and Booker T. Washington. She was especially close to Douglass