
Grant selected a cabinet mainly composed of his friends and supporters, rather than seeking to appease the various factions in the Republican Party. He chose his friend Elihu B. Washburne as Secretary of State. (Washburne served only twelve days before resigning over claims of ill-health). Grant then appointed Hamilton Fish, a conservative New York statesman, as Secretary of State. Fish would be Grant's most successful appointment. Grant appointed his former fellow Union Officer John A. Rawlins as Secretary of War. Rawlins had been Grant's principal aide when Grant was commanding in the civil war. One of his tasks was to keep Grant sober. Grant promoted Sherman to his own former post as Commanding General. Rawlins died in office in 1869 and Grant appointed William W. Belknap as his replacement. Both Rawlins and Belknap and had limited Sherman's military authority and Grant refused to contradict them. This put a strain on Grant's relationship with Sherman.
By 1870, all of the former Confederate states were successfully restored into the United States. Grant lobbied Congress to pass the Fifteenth Amendment, a constitutional amendment intended to guarantee that no state could prevent someone from voting based on race. Grant believed that its passage would secure the rights of freed slaves. To enforce the new amendment, Grant relied on the army and the newly created Justice Department. In 1870, Grant had signed a bill establishing the Department of Justice. The department was created in part to see that federal laws were enforced in the South, where state courts and prosecutors were unwilling to do so. Previously, the Attorney-Ggeneral had only been only a legal adviser to the president. Now the A-G led a cabinet department dedicated to enforcing federal law, with a Solicitor General to assist him.
Grant was angry at this rise in terror and violence by the Ku Klux Klan and other groups. With Grant's encouragement, Congress passed the Enforcement Acts in 1870 and 1871. These Acts made it a federal offense to deprive any person of his civil rights and allowed the president to use the army to enforce the laws. In May 1871, Grant ordered federal troops to assist US Marshals in arresting Klansmen. In October of that year, Grant suspended the writ of habeas corpus in part of South Carolina and sent federal troops to enforce the law there. With Grant's aggressive enforcement, the Klan's power collapsed, and by 1872, elections in the South saw African Americans voting in record numbers.
In 1872, Grant signed the Amnesty Act, which restored political rights to former Confederates. But the spotlight was taken off of Grant's efforts to help African-Americans, due to scandals in Washington involving members of Grant's administration. This made it harder for Grant to find support for his enforcement policies. After the collapse of the Klan in 1872, southern whites formed armed groups such as the Red Shirts in South Carolina and the White League. Unlike the Ku Klux Klan, these groups were not secret. Like the Klan however they used violence and intimidation against African Americans in order to take control of state governments away from the Republicans. The economic Panic of 1873 and the ensuing depression made many in the North grew less concerned with reconstructing the South.
By 1875, Democratic politicians known as "Redeemers" retook control of all but three Southern states. Violence against African-Americans escalated again. Grant signed the Civil Rights Act of 1875, which expanded federal law enforcement by prohibiting discrimination on account of race in public accommodations, public transportation, and jury service. But the law was rarely enforced and so it did not stop the rise of white supremacist forces in the South.
In the election of 1876, Grant chose not to seek a third term in office, following the precedent set by George Washington. In the election the remaining three Republican governments in the South fell to the Democrat "Redeemers". The election results left the question of who would be President in doubt. The issue was decided by what became known as the Compromise of 1877, in which Republican Rutherford Hayes became President, and US troops were removed from the south. This marked the end of Reconstruction.
After leaving the Presidency, Grant embarked on a world tour where he was well received. The countries he visited included England, Ireland, India, Burma, Siam, Vietnam, China and Japan. In 1880 Grant changed his mind about a third term in the White House, but he was unable to secure the Republican nomination, and James Garfield was selected as a compromise candidate.
Grant's son Ulysses Jr. (known as "Buck") had gone into business with an unscrupulous partner named Ferdinand Ward. Ward swindled a number of investors after using Grant's name to secure their investment. Although he had no legal responsibility for the debt, Grant helped his son repay the investors by selling off most of his Civil War memorabilia. The matter left Grant financially destitute.
Grant had forfeited his military pension when he assumed the presidency, but Congress restored him to the rank of General of the Army with full retirement pay in March 1885. Around the same time, Grant learned that he was suffering from throat cancer, and he was concerned about how his family would make ends meet after he was gone. Grant wrote several articles on his Civil War campaigns for The Century Magazine at $500 each. The articles were critically acclaimed, and editor Robert Underwood Johnson suggested that Grant write a book of memoirs, as Sherman and others had successfully done. Century offered Grant a book contract with a 10% royalty, but Grant's friend, Mark Twain, made his own offer to Grant for his memoirs, proposing a 75% royalty. Grant accepted Twain's offer.
Grant worked on his memoirs at his home in New York City, and then from a cottage on the slopes of Mount McGregor. The deadline he was working on was his own lifeline. He finished the book shortly before he died on July 23, 1885. The book, entitled Personal Memoirs of Ulysses S. Grant, was a huge success. Julia Grant received about $450,000.

Grant was 63 years old at the time of his death. President Grover Cleveland ordered a thirty-day nationwide period of mourning. After private services, Grant's body was placed on a funeral train and traveled via West Point to New York City, where a quarter of a million people viewed it in the two days prior to the funeral. Tens of thousands of men, many veterans from the Union Army, marched with Grant's casket to Riverside Park on the west side of New York. His pallbearers included Union generals Sherman and Sheridan, Confederate generals Simon Bolivar Buckner and Joseph E. Johnston, Union Admiral David Dixon Porter, and John A. Logan. His body was laid to rest in Riverside Park, first in a temporary tomb, and later in a sarcophagus at General Grant National Memorial, better known as "Grant's Tomb". Above the entrance to the building, etched in marble, is Grant's 1868 campaign slogan: "Let us have peace."
