It wasn't until 1892, when Cleveland was elected to his second term as president, that Gray Gables became Cape Cod’s first summer White House. The depot had a direct telegraph line back to Washington. Two of the five Cleveland children were born at Gray Gables. Their third child Marion Cleveland was born there in 1895, during Cleveland's second term, and their youngest child Francis Grover Cleveland was born there in 1903 after Cleveland was out of office. The Cleveland family discontinued their residency in 1904.
Cleveland enjoyed in trout fishing and the area around Bourne had many trout-filled ponds that were a huge attraction for him. Cleveland had some improvements done to the home both for aesthetics as well as to accommodate his presidential duties. He changed the name because he thought it better suited the home’s numerous gables. Besides installing the telegraph lines Cleveland also had a private railroad station built. Cleveland was often seen around the town of Bourne wearing a battered fedora and fishing boots. He was there in June of 1892 when he received word of his second presidential nomination.
On January 7, 1904, Cleveland's oldest daughter Ruth, then aged 12, died of diptheria. After Ruth's death the Clevelands lost interest in Gray Gables. Gray Gables was leased in the summer of 1904, and then sold. The building was turned into an inn by a subsequent ownder, but was destroyed by fire in 1973.