When American voters cast their ballots for President Theodore Roosevelt in 1904 they probably didn’t expect that his daughter Alice, already known for her rebellious larks, would turn out to lay groundwork for diplomacy that would end the Russo-Japanese War. When they voted for Franklin D. Roosevelt in 1932, they probably didn’t imagine that one of his children would influence her father’s vice presidential pick, while another would make waves trying to fly his dog across country on the U.S. military’s dime.

That’s because for all the vetting that goes into picking our next president during campaign season, Americans tend to spend little time worrying about the candidate’s children. Just think about this campaign season: The “invisible hand” of 35 year-old Chelsea Clinton, according to those in Clintonworld, “shapes almost every significant decision her parents make.” And yet little time is spent deciphering Chelsea’s foreign policy views or email habits. And Donald Trump has four adult children, all trusted members of his inner circle. Ivanka Trump, in particular, serves as a surrogate and adviser on the campaign trail. But who knows what her positions are on Mexico, Iran and income taxes?

In every presidential contest since 1789, the American people have been elevating to power not just one person but an entire family, usually including several adult children. These children aren’t forced to debate each other; they don’t release their financial or health records, or face endless scrutiny on their policy positions—and yet a tour through history shows that they sometimes turn out to have a significant impact on the presidency. Some have provided valuable assistance to their fathers and the nation, serving as trusted advisers or surrogates. Others have tainted their father’s administration with scandal and corruption, distracting the commander-in-chief and providing fodder for the press. If voters knew the effects famous scions have had on U.S. history—both good and bad and in between—they might want to pay closer attention not just to whose name is on the ballot, but to the children standing at that person’s side.

***

John Adams, who was 61 when he became president in 1797, had one child who helped his cause and three others who worried him. When Adams appointed his eldest son, John Quincy Adams, minister to Prussia, the Republican opposition immediately called for the president’s resignation, arguing that he was setting a dangerous precedent by giving a job to a family member. But the furor died down once it became clear just how impressive the younger Adams, who had been a favorite of George Washington, was. In his father’s administration, he renewed a treaty with Prussia and went on, under later presidents, to play a key role in ending the War of 1812, negotiate the annexation of Florida and draft the Monroe Doctrine—all before becoming president in his own right two decades later.

Adams’ two other sons, Charles and Thomas, were often paralyzed by crippling bouts of depression. Both also struggled with alcoholism, and neither man could earn a steady living—despite their father’s constant efforts to help their fledgling legal careers. In 1798, Adams disowned Charles when he discovered that his son was cheating on his wife and neglecting his two children.

Adams’ only daughter, Nabby, married a scoundrel, William Smith, who kept losing money on speculative real estate deals and eventually landed in debtors’ prison. When Smith insisted that his president father-in-law bail him out by putting him on the federal payroll, Adams refused. The constant family turmoil left the commander-in-chief feeling overwhelmed. “My children give me more pain than all my enemies,” the exasperated Adams wrote to his wife, Abigail, in late 1798.

Presidential children, though, often prove to be invaluable buffers. President Rutherford B. Hayes’ gregarious second son, Webb, had a flair for politics, which Hayes mined by appointing him, just 20 years old at the time, as his private secretary after Hayes took office in 1877. Father and son worked side-by-side for all four years, during which Webb enabled his father’s firm stand on race relations.

When Hayes got some serious pushback from southerners for appointing the former slave, Frederick Douglass, as the United States marshal of the District of Columbia, the president refused to back down. Instead, he asked his son Webb to act as the official greeter at White House social functions—a duty typically served by the marshal. Douglass was appreciative of Hayes’ clever strategy to keep him in the post, saying, “I was ever a welcome visitor at the Executive Mansion on state occasions and all others, while Rutherford B. Hayes was President of the United States. I have further to say that I have many times during his administration had the honor to introduce distinguished strangers to him, both of native and foreign birth, and never had reason to feel myself slighted by himself or his amiable wife.” Webb went on to serve in three wars—the Spanish-American War, Philippine-American War and First World War—and win the Congressional Medal of Honor.

The youngest president in our history, Theodore Roosevelt was just 42 when he succeeded William McKinley. While the five children from his second marriage ranged in age from three to 14, Alice, the daughter from his first marriage, made her debut at the White House just a few months after TR’s inauguration. The rambunctious young woman soon captured the nation’s heart with her beauty and fashion sense. She also delighted the fawning press corps with her unconventional behavior—she smoked in public, gambled at the racetrack and drove her car at high speeds through the capital. And that doesn’t even include the time she donned a live boa constrictor; or the afternoon she spent shooting at telephone poles from a moving train. TR was mightily irritated by his daughter’s antics, which also included hobnobbing with the sons and daughter of the plutocrats he was trying to crush; he once told a friend that he could either be president or control Alice—he could not do both.

But early in his second term, TR changed course. Realizing that her quirkiness reflected a kindred spirit, he began to admire her independence and originality. What’s more, he turned Alice, whom he had once labeled “the liability child,” into a political asset. In 1905, TR sent “Princess Alice” off on a four-month tour of Asia. “Miss Roosevelt,” a Japanese reporter noted that summer, “has charmed us all … she reminds me of her father and she has the … same way of giving you her undivided attention for the time you are speaking with her.” Serving as his goodwill ambassador, Alice helped pave the way for the summit in Portsmouth, New Hampshire, between Russia and Japan, which led to the end of the Russo-Japanese War. For these diplomatic efforts, which turned America into the premier power broker on the international stage, TR would nab the Nobel Peace Prize the following year. Of Alice’s contribution, the president observed in late 1905, “She has behaved might well under trying circumstances.”

There have been other female trailblazers in the White House. When Woodrow Wilson became president in 1913, he brought his three daughters—Margaret, Jessie and Eleanor—with him to 1600 Penn., where these bright and articulate women, all in their mid-20s, reminded him again and again how important it was that women be able to vote. As Jessie repeatedly told her father, it was a grave injustice that only one-fifth of the family possessed the franchise. While the Wilson daughters said little publicly at first, after a couple of years in Washington, they became fierce advocates for the Nineteenth Amendment. In 1915, Margaret served as hostess of the annual convention of the advocacy group that would later become known as the League of Women Voters.

In 1918, Wilson himself finally began urging House members to vote for the Constitutional amendment, citing “the marvelous heroism and splendid loyalty of our women” during the Great War. But his loyalty to the young woman whom he himself had sired also influenced his steady evolution on the issue.

And then there was Anna Roosevelt, Franklin Roosevelt’s daughter, who very probably saved a president’s life. In 1944, this 37-year-old mother of three, who had been working as a journalist in Seattle, moved to Washington to settle into the Lincoln Suite and become her father’s unpaid personal assistant. That March, an alarmed Anna noticed that Dr. Ross McIntire, the president’s personal physician, had not been monitoring FDR’s blood pressure regularly. At her request, the frail president was soon whisked off to Bethesda Naval Hospital for a full medical work-up. Diagnosed with congestive heart failure, the president began taking medication and cutting down on his smoking and drinking.

The eldest of five siblings, Anna became her father’s constant companion during the last year-and-a-half of his presidency. She was also his confidante and his gatekeeper, and as such wielded enormous influence. According to Washington insiders, that summer, Anna convinced her father to choose Harry Truman rather than Supreme Court Justice William O. Douglas as his next vice-president. “Daddy’s girl,” asserted Life in early 1945, “is running Daddy.”

In February 1945, Anna accompanied the president to Yalta where he met with Soviet leader Joseph Stalin and Prime Minister Winston Churchill to discuss post-war plans for Europe. “The biggest difficulty in handling the situation here,” Anna wrote to her husband from Russia, “is that [FDR’s two doctors and I] can, of course, tell no one of the ticker trouble. It’s truly worrisome.” Anna’s non-stop attention and care helped the terminally ill FDR function at a high-level during the event-filled weeklong conference.

While Anna provided vital support until FDR’s death in April of 1945, her four younger brothers—James, Elliott, Franklin Jr., and John—often added to the president’s burdens. On the night that FDR gave his first inaugural address in 1933, an inebriated John, then finishing high school at Groton, stunned the Washington police by driving his car right up to the front door of the White House. While a sergeant quickly shooed John away, this embarrassing incident was widely covered by the national press corps. That was just two years before the Associated Press reported that Franklin Jr.’s “accident and speeding record since his advent to Harvard in 1933 resembles the French war debt.”

And then there were the two older sons, who spun scandals of a slightly more serious nature. In the summer of 1933, the 22-year-old Elliott titillated the nation by remarrying just five days after obtaining a quickie divorce from his first wife in a Nevada courtroom. A few years later, FDR’s eldest son, James, was forced out of his job as his father’s secretary after several leading magazines reported that he had previously used the family name to rake in huge commissions as a life insurance agent. The reports made the case that big donors and corporations had been trying to influence the president by lavishing business on his eldest son.

Shortly before FDR’s death, Elliott, then an army colonel, created another public relations disaster for the commander-in-chief. In January of 1945, just as Elliott’s name was submitted to the president for promotion to brigadier general, the public learned that he had sent his 130-lb. English bull mastiff, Blaze, to his third wife, A-list actress Faye Emerson, from Europe to California on a military transport plane. After a stopover in Memphis airbase, Blaze ended up bumping three soldiers off the flight. The resulting furor dominated the front pages for weeks. According to the New York Herald Tribune, the story received more press coverage than Russia’s latest skirmishes with the Germans. After a prolonged investigation of “L’Affaire Blaze” by a Senate subcommittee, Elliott was eventually exonerated from any wrongdoing as there was no evidence that he had ever personally demanded that the dog unseat the soldiers. And despite the bad press, FDR went ahead and approved his military promotion. Meanwhile Blaze settled safely in Hollywood.

“The reality is that the presidency is like a combat situation, and family members can provide valuable help with the excessive demands.” That’s what President Dwight Eisenhower’s granddaughter Susan says about her father John’s tenure as a White House aide in Ike’s administration, where John was tasked with delivering bad news to the high-strung and often ill-tempered president. “My father agreed to serve as my grandfather’s sounding board.”

Or, family members can provide almost unending grief.

Chelsea Clinton or the young Trumps—or any other of the progeny of 2016—might not be quite so likely to drum up scandal as, say, FDR’s sons. “We will never be caught dancing on tables,” says Eric Trump, Donald Trump’s third son. Or perhaps as impressive as, say, John Quincy Adams. But history tells us this: That the presidency—and the perks, pressure and spotlight it brings with it—affects First Children in unpredictable ways. And it’s probably worth thinking about just who those people are before we elect their parents.