The first African-American first family to move into the White House

By Susan Roesgen and Aaron Cooper
CNN

WASHINGTON (CNN) -- In January, President-elect Barack Obama and his family will make history, becoming the first African-American first family to move into the White House -- a house with a history of slavery. In fact, the legacy of American presidents owning slaves goes all the way back to George Washington.

A wood engraving of handcuffed and shackled slaves passing the U.S. Capitol, depicts a scene circa 1819.

Twelve American presidents owned slaves and eight of them, starting with Washington, owned slaves while they lived in the White House. Almost from the very start, slaves were a common sight in the executive mansion. A list of construction workers building the White House in 1795 includes five slaves - named Tom, Peter, Ben, Harry and Daniel -- all put to work as carpenters. Other slaves worked as masons in the government quarries, cutting the stone for early government buildings, including the White House and U.S. Capitol. According to records kept by the White House Historical Association, slaves often worked seven days a week -- even in the hot and humid Washington summers.

In 1800, John Adams was the first president to live in the White House, moving in before it was finished. Adams was a staunch opponent of slavery, and kept no slaves. Future presidents, however, didn't follow his lead. Thomas Jefferson, who succeeded Adams, wrote that slavery was an "assemblage of horrors" and yet he brought his slaves with him. Early presidents were expected to pay their household expenses themselves, and many who came from the so-called "slave states" simply brought their slaves with them.

Martin Van Buren, William Henry Harrison, Andrew Johnson and Ulysses S. Grant all owned slaves but not during their time in office. James Madison, Jefferson's successor, held slaves all of his life including while he was in office. During the war of 1812 Madison's slaves helped remove material from the White House shortly before the British burned the building. Michelle Obama uncovers slaves in her family

In 1865 one of Madison's former slaves, Paul Jennings, wrote the first White House memoir: "A Colored Man's Reminiscences of Life in the White House." In the book, Jennings called Madison "one of the best men that ever lived" and said Madison "never would strike a slave, although he had over one hundred; neither would he allow an overseer to do it."

There were other presidents who treated their slaves less kindly.

James Monroe, Andrew Jackson, John Tyler, James K. Polk and Zachary Taylor all owned slaves while they were in office. The last of these, President Taylor, said owning slaves was a Constitutional right and he said slave-owners like himself would "appeal to the sword if necessary" to keep them. The Civil War, of course, put that opinion to the test.

Now, the Obamas are moving into the White House.

"The apple cart has been turned over here when you have the Obamas -- the first African-American couple -- now actually management and you are having in some cases white Americans serving them," says presidential historian Doug Brinkley.

Michelle Obama learned this year that one of her great-great grandfathers was a slave who worked on a rice plantation in South Carolina. She says finding that part of her past uncovered both shame and pride and what she calls the tangled history of this country.

For many, the historic election on November 4 marked a new beginning.

Though Michelle Obama's ancestors had to come through the ordeal of slavery, "Her children are sleeping in the room of presidents," said Brinkley. "It's a very great and hopeful sign."