Sometime in 1937, Johnson began having an affair with a woman named Alice Glass. She was the wife of one of his major supporters. Lady Bird had met Alice when the two of them attended the University of Texas. Alice came from Marlin, Texas. She was almost six feet tall with honey-blond hair and blue eyes. The affair between Johnson and Glass was written about by Johnson's leading biographer, Robert Caro, in his book "Path to Power" and also in a memoir published in 1993 by former Texas governor John Connally, one of Johnson’s closest political allies. Johnson wasn't the only man that Glass broke her marriage vows for. She also had an affair with Charles Marsh, the wealthy publisher of the Austin American-Statesman and one of Johnson’s primary patrons. She later became Marsh's wife.
The affair was a risky one to Johnson's political career. Lady Bird became aware of the affair. When Alice had moved to Longlea, an eight-hundred-acre horse farm in the Blue Ridge Mountains in Virginia, she had more frequent contact with LBJ. Lady Bird is said to have been intimidated by Alice, who was not just physically taller, but also very gregarious, outgoing and seductive, qualities that LBJ was critical of his wife for lacking. According to lobbyist Frank Oltorf, who knew Johnson and Glass, “Alice had a great presence. When she walked into a room, everyone looked at her. She was tall, slim, good-looking, and extremely smart. She had a voice that was both sexy and soothing.” Oltorf said that Johnson had found his match in Glass because she was someone as smart as he was, and as able to control other people.Lady Bird never publicly expressed how she felt about the affair. She remained publicly supportive of her husband and pretended that he was a good and faithful husband.
The affair continued until the late 1940s. But it was not Johnson's last. He once bragged that he’d “had more women by accident than Kennedy ever had on purpose.” When Johnson became President, he continued to cheat on his wife, even in the White House. Once, after Lady Bird walked in on her husband having sex with one of his secretaries on a sofa in the Oval Office, an angry Johnson ordered the Secret Service to install a buzzer system to alert the President whenever his wife was close by. This incident occurred just months after he took office. A former Secret Service agent said, "if we saw Lady Bird heading for the elevator or stairs, we were to ring the buzzer."
In 1982, a woman named Madeleine Brown held a news conference held at the Dallas Press Club, at which she claimed that she was Johnson’s mistress for 21 years. Brown said that Johnson set her up in a two-bedroom home with a maid and gave her credit cards and cars. She also described Johnson as “kinky” and said that their meetings would generally last around 30 minutes. She said that the affair began in 1948 after a party at the Adolphus Hotel in Dallas and it continued until 1967. The Dallas Morning News published the allegations under the headline "Dallas Woman Claims She Was LBJ’s Lover" and the story quoted a spokesman for Lady Bird Johnson and the Johnson family as stating the claim to be false. Brown said she wanted to clear the record, after published reports of an affair between Johnson and Glass were made three weeks earlier.
Brown said that she attended a reception at radio station KTBC (owned by Lyndon Johnson) in 1948 while working for a Dallas advertising agency. She said that she met Johnson there, that he flirted with her and that she was "dazzled" by him. She saw him again at another reception three weeks later at the Adolphus Hotel where she claimed that Johnson invited her up to his apartment in the hotel. Brown told People magazine that her relationship with Johnson was "purely physical". She also said that she was aware that he had other women that he was having sex with. Brown said that she told Johnson in April 1950 that she was pregnant and he assured her he would provide for her needs. She claimed that she was told that she would be contacted by attorney Jerome Ragsdale. Brown alleged that after the birth of her son Steven in 1950, Ragsdale provided her with "a six-room house for $15,000, complete with a live-in maid" and paid various charge cards supplied to her. According to Brown, she continued to meet with Johnson throughout the 1950s and 1960s. She said she had a "paper marriage" to a Dallas businessman named Brown but she never actually lived with the man. Brown said that her sexual relationship with Johnson abruptly ended in 1967 after she was badly injured in a car accident in which her son, Steven, was driving. She said she had one last meeting with Johnson in 1969 at the Shamrock Hotel in Houston, Texas.
Brown stated that she was hospitalized after suffering a heart attack in February 1987 and she called her son Steven to her bedside in order to confess the true identity of his father. Brown had no love letters from Johnson and she told People Magazine that the few people who could corroborate her story were dead.
On June 18, 1987, Brown's 36 year old son Steven Mark Brown filed a $10.5 million lawsuit against Lady Bird Johnson in a Texas District Court, which claimed that Lady Bird and two of Johnson's friends, Jesse Kellam of Austin and Jerome T. Ragsdale of Dallas, had conspired to deprive him of a share of the Johnson estate. In a court document, Brown stated: "My legal birthrights have been violated and a conspiracy was formed to deprive me of my legal heirship." The suit included an affidavit from his mother, Madeleine Brown, swearing that she met Johnson in 1948 at a social function within a Dallas hotel, and that the two had an affair that lasted from 1948 until 1969. Her affidavit swore that Ragsdale, a Dallas attorney who was deceased by the time the lawsuit was brought, was assigned to assist with legal issues regarding her pregnancy and that he claimed to be the child's father in order to hide Johnson from negative publicity.
Lady Bird Johnson responded through her press secretary Betty Tilson, who said: "We're convinced the whole thing is made-up, frankly" and that the Johnson family were unfamiliar with Steven Mark Brown". She also stated that the first lady did not know Ragsdale either.
In 1989, Steven Brown's lawsuit was dismissed after he failed to attend court. He died in 1990, at age 39,from lymphatic cancer. That same year Madeleine Brown was convicted of forging the will of an elderly relative, who had died the previous year. She was sentenced to ten years in prison, but her conviction was overturned in 1994. The appeals court ruled that she had not personally signed the original will.