As President, the most pressing issue on Johnson's desk was reconstruction. Johnson had to lead on the issue of how the confederate states would be brought back into the union. He had said that he intended to follow the policies of his predecessor, Abraham Lincoln, who had promised to "bind up the nation's wounds" in a benevolent manner. The Republican-controlled Congress had other ideas. It began enacting legislation to guarantee the rights of former slaves. Johnson's focus was on pardoning former Confederate officials. He was less concerned about giving any rights to freed slaves in the southern states. Johnson's policies angered the Radical Republicans in Congress and gradually alienated the moderates. By 1866, Congress had gathered enough unified support against Johnson to enact the first override of a Presidential veto in over twenty years. They passed a bill that continued the Freedmen's Bureau. Johnson also managed to alienate his own cabinet, three members of which resigned in 1866.
The mid-term elections were approaching and many saw them as a referendum on the Johnson presidency. Johnson had a reputation as a very good stump speaker, and he developed a strategy to put that talent to use to that end he decided that he would make a political speaking tour, something that was highly unprecedented for a sitting President at the time. He brought an entourage with him that included his two supporters in the cabinet, Secretary of State William Seward and Navy Secretary Gideon Welles. In order to attract bigger crowds, he commanded some of the more prominent Civil War military heroes to accompany him on the tour. These included David Farragut, George Custer, and Ulysses S. Grant. Grant was, at that time, the most admired man in the country. He would have these heroes stand next to him while he spoke, giving the impression to the audience that he had their support. As good military men, they knew how to follow orders and would not contradict their commander-in-chief.
The tour lasted for 18 days. Johnson and his group made stops in Baltimore, Maryland; Philadelphia, Pennsylvania; New York City, West Point, Albany, Auburn, Niagara Falls, and Buffalo, New York; Cleveland and Toledo, Ohio; Detroit, Michigan; Chicago, Springfield, and Alton, Illinois; St. Louis, Missouri; Indianapolis, Indiana; Louisville, Kentucky; Cincinnati and Columbus, Ohio; and Pittsburgh and Harrisburg, Pennsylvania, as well as short stops in smaller towns between.
Johnson's critics looked upon his tour as something beneath the dignity of the office Johnson held, because Presidents had traditionally not undertaken political campaigning in the past. Johnson's advisors recommended that he give only carefully prepared speeches, but Johnson thought he knew better. As he had often done on the campaign trail, he would prepare a rough outline of speaking notes and speak spontaneously around the outline. This turned out not to be a wise strategy. Johnson's fiery oratory offended a larger segment of his audience than it pleased with its plea for generosity to the defeated Confederates.
At first Johnson was enthusiastically received, particularly in Baltimore, Philadelphia, and New York. On these stops he delivered a speech that thanked his audience for its welcome, paid tribute to the army and navy, and declared that he still favored the preservation of the Union of the states. He would begin with the same line usually, "Fellow citizens, it is not for the purpose of making speeches that I now appear before you", a line that would generally garner a laugh. He would then give a brief biography of himself, recounting his rise from being a humble tailor to the presidency. He would compare himself to Jesus Christ and explained that like the Savior, he, too, liked to pardon repentant sinners. He would blame Congress, and especially Thaddeus Stevens and the radicals, whom he said still wanted to break up the Union, something he was trying to prevent.
The press nonetheless gave him positive coverage throughout the first leg of the tour. But as Johnson entered the Radical Republican strongholds of the Midwest, he began facing much more hostile crowds, many of which were organized by Republican leaders in those towns. It was Johnson's stop in Cleveland on September 3 that was especially memorable on the tour. There the crowd included mobs of hecklers, many of them plants by the Radical Republicans, who goaded Johnson into engaging them. When one of them yelled "Hang Jeff Davis!" in Cleveland, Johnson angrily replied, "Why don't you hang Thad Stevens and Wendell Phillips?" When he left the balcony from which he had spoken, reporters overheard an advisor telling Johnson to maintain his dignity. Johnson angrily replied "I don't care about my dignity". The remark was printed in newspapers across the nation, abruptly ending the tour's favorable press.
At later appearances in southern Michigan did not go well. On September 7, Johnson spoke in Chicago, but Illinois governor Richard J. Oglesby refused to attend the speech. The Chicago city council also boycotted the event. In Chicago Johnson changed his strategy and delivered only a short pre-written speech. Two days layer, in St. Louis on September 9, Johnson got into another argument with a heckler, after Johnson accused Radical Republicans of deliberately inciting the deadly New Orleans Riot that summer. Once again Johnson compared himself to Jesus, and called the Republicans in Congress his betrayers. The next day in Indianapolis, the crowd was so hostile and loud that Johnson was unable to speak at all. After he left, violence and gunfire broke out in the streets between Johnson supporters and opponents, resulting in the death of one man. At other points in Kentucky, Ohio, and Pennsylvania, spectators drowned out Johnson with calls for Grant, who refused to speak, and for "Three cheers for Congress!"
On September 14 in Johnstown, Pennsylvania, a temporary platform built beside the railroad tracks for the president's appearance gave way, sending hundreds of people standing on the platform into a drained canal 20 feet below. Johnson attempted to halt the train and use it for triage for the injured, but he was advised by engineers that the train could not wait due to conflicting train traffic. Some of the presidential party left the train to assist the victims, while Johnson and the rest of the party continued onto Harrisburg. Johnson's opponents spun the appearances to look as if Johnson had callously abandoned the scene. This was not accurate and in fact Johnson later donated $500 ($8,318 in 2016 dollars) to assist the victims.
The press was generally very critical of Johnson for his appearances and speeches. The New York Herald had once been the most supportive newspaper for Johnson, but it was also critical of the president. In an editorial, the newspaper wrote: "It is mortifying to see a man occupying the lofty position of President of the United States descend from that position and join issue with those who are dragging their garments in the muddy gutters of political vituperation." Cartoonist Thomas Nast created three large illustrations lampooning Johnson.
Johnson's Republican opponents criticized Johnson for the failure of his tour. Thaddeus Stevens gave a speech about the tour, calling it "the remarkable circus that traveled through the country that cut outside the circle and entered into street brawls with common blackguards." Radical Republicans also began spreading rumors that Johnson had been drunk at several appearances, like he had been at his inauguration as vice president the year before. Though untrue, reporters and political opponents continued to spread the rumors. Although he never publicly criticized Johnson, General Ulysses Grant said, in a letter to his wife Julia, that he found the experience embarrassing, both for Johnson and for Grant himself for having to be next to the president on much of the tour.
Some of Johnson's supporters were also critical of the tour. Former Georgia Governor Herschel V. Johnson wrote that the President had sacrificed "the moral power of his position, and done great damage to the cause of Constitutional Reorganization." Senator James Doolittle of Wisconsin saidd that the tour had "cost Johnson one million northern voters."
By the time he returned to Washington from the speaking tour, Johnson had even less support in the North than he had started with. His only remaining allies in Congress were southern Democrats. These were mostly former rebels. The Republican party won a landslide victory in the congressional elections, and the new Congress took control of Reconstruction from the White House with the Reconstruction Acts of 1867. Johnson fought bitterly with the new Congress for the control of the nation's domestic policy. But the Republicans' vastly increased congressional voting bloc gave them sufficient votes to attempt impeachment of Johnson, first unsuccessfully in 1867 and again successfully in 1868. The tenth of 11 articles charging that the President "did make and declare, with a loud voice certain intemperate, inflammatory, and scandalous harangues, and therein utter loud threats and bitter menaces, as well against Congress as the laws of the United States duly enacted thereby, amid the cries, jeers and laughter of the multitudes then assembled in hearing." The impeachment managers chose not to bring this article to a vote in the Senate, but it was clearly drafted as a reference to the Swing Around the Circle Tour.
The Republicans captured the White House in 1868 and maintained control of it until 1885. The Swing Around the Circle began a long series of political defeats that crippled Johnson, the Democratic Party and the presidency for years to come.