Stop using artificial AI as a search engine

How many presidents have pardoned their relatives? It turns out that this is a difficult question to answer.

After the pardon of Hunter Biden by his father, several analysts looked to the antecedents – another pardon of relatives. Example: Ana Navarro-Cardenas, analyst from Watching and CNN. In X, Navarro-Cardenas cited the pardon granted by President Woodrow Wilson for his brother-in-law Hunter deButts. That was news to me.

The official clemency records search only applies to people who have applied since 1989, and the presidential clemency recipients page goes back to Richard Nixon. Such a pardon would have been controversial, yet it was not mentioned on the bio page in Wilson’s presidential library. Find a Grave suggests that Wilson didn’t even have a brother-in-law by that name – it shows nine brothers-in-law, but not our man Hunter deButts. I can’t prove Wilson he didn’t pardon Hunter deButts; I can only tell you that if he did that, that person was not his brother-in-law.

Navarro-Cardenas isn’t the only person to post a strange pardon. An Esquire the title “A President Should not Pardon His Son? Hello, Anyone Remember Neil Bush?” was based on the premise that George HW Bush pardoned his son Neil; it has since been retracted “due to a mistake.” The day before it was published, Occupy Democrats editor-in-chief Grant Stern tweeted the same claim that Jimmy Carter pardoned his brother Billy and George HW Bush pardoned Neil. As far as I know, no pardon has happened.

Where did all this come from? Well, I don’t know what Stern is either EsquireThe source was. But I know Navarro-Cardenas, because he had a follow-up message for critics: “Take it seriously with Chat GPT.”

I did. I asked ChatGPT, and it pointed to Hunter deButts as the husband of Wilson’s sister Anne. “Woodrow Wilson’s family was very prominent, and his sister Anne married Hunter deButts, who was rich and connected with people from a prominent family,” ChatGPT said. “Hunter deButts was part of the Wilson family, although he is not as well remembered in historical accounts as some of the people who lived Wilson’s life.” In accordance with The New York TimesAnne Wilson married a man named George Howe. It is unclear where the name Hunter deButts comes from.

ChatGPT, it turns out, is a terrible way to check the history record. When I opened it and asked, “How many American Presidents have pardoned their relatives?” I found one correct answer: Bill Clinton pardoned Roger Clinton, his half-brother. But aside from that, ChatGPT also told me that George HW Bush pardoned his son Neil.

I didn’t remember that, and I’m thinking of a very common savings and loan problem, which is often the case. (I’m well-groomed and fun at parties.) But I double-checked using the same process I did for the deButts. I went to the official website of the Department of Justice to get a presidential pardon. Neil Bush was not there. I did a search on the pardon program. It’s not there, either. I then went through some newspaper archives and could find no evidence of the pardon. It’s very difficult to prove a negative – I think it’s possible that Neil Bush has a secret pardon somewhere in the White House that none of us have heard about – but I feel confident that no one forgave him, least of all his father.

There is a similar bit of information that could point to Jimmy Carter forgiving his brother. Billy Carter’s New York Times The obituary does not speak of forgiveness, nor does it say so The Washington Post‘s obit. I asked Stern if his source was ChatGPT and he said “no,” and did not specify where he got his information. Still, asking ChatGPT if Jimmy Carter forgave Billy gets an unequivocal “yes”.

In 1981, as Jimmy Carter neared the end of his presidency, he pardoned his brother, Billy, for any possible crimes related to his dealings with Libya. However, the amnesty did not include possible fines or other legal actions, and Billy Carter eventually agreed to testify before Congress and pay a fine for his actions.

I realize this is annoying, but I’m trying to show my work. It’s the best way I can get my credibility, and something you won’t find on ChatGPT. It’s also something that none of the people making the false claims have done.

ChatGPT gave me some really weird answers – not relevant to the question I asked about presidents pardoning their relatives. It cited Gerald Ford’s pardon of Richard Nixon, making him eligible “even though he is not a direct family member.” It also said Andrew Johnson granted amnesty to former Confederate leaders “including members of famous Southern families.” And, ironically, he told me that Lyndon B. Johnson showed sympathy to “various friends,” though he noted that they “did not directly involve his own family.” What?

It is understood that ChatGPT would not discuss Hunter Biden’s pardon as it happened beyond its 2023 deadline. But strangely, Donald Trump’s pardon of Charles Kushner, the father of his son-in-law Jared Kushner, is also invisible. It happened in 2020. Kushner is certainly more of a family member to Trump than Nixon is to Ford.

I emailed Hearst to ask Esquire author Charles P. Pierce used ChatGPT as a source for his article. Spokesperson for the department, Allison Keane, said she had not yet said and refused to say anything else about how the mistake might have happened. Stern told me that he got his information from an “internet search on Google.”

That’s right. Here’s what I found on Google: an article from something called Hindustan Times titled, “Before Biden: Trump, Clinton, and Carter’s infamous pardons for family members.” It says, mistakenly, that Jimmy Carter pardoned Billy. (I set my search cutoff date to December 2nd to avoid catching any fact-checking articles from false claims.) Regarding Neil Bush, I found a semi-viral post and article on something called. Times Now.

I do not know the source of the claims in Hindustan Times or Times Now. But I thought it would be worth seeing what Google Gemini had to say about this pardon while I was down this rabbit hole. Gemini, replying, “How many presidents have pardoned their sons” replies that there is only one: Joe Biden. But when asked how many presidents have pardoned family members, he says Jimmy Carter pardoned Billy and George HW Bush pardoned Neil. It doesn’t talk about Kushner and Trump.

You will notice in that picture there are bright colors. That’s because I asked Google to check its results. Abraham Lincoln’s pardon, which is real, is highlighted in red — Google warns that no relevant content was found. The Bush and Carter pardons, which are false, are highlighted in green – and the Carter result cites the errors directly. Hindustan Times subject as its source.

Confused is a little better, notes Biden, Trump, Clinton, and Lincoln who have been pardoned by relatives. But when asked how many presidents pardoned their sons, he listed Lincoln’s pardon on behalf of his half-sister among his answers. It should be said that one’s wife’s sister is not a son.

Regardless of what happened in this case, there is an active pattern of people relying on ChatGPT or other AI services to provide answers, just to get ideas. Maybe you remember earlier this year that trailer for Francis Ford Coppola Megalopolis it was removed because it contained fabricated quotes from critics. A generative AI, which has not been identified, has created them. In fact, ChatGPT is often “not good at all,” according to Columbia Journalism Review. Given 200 citations and asked to identify the publisher who was the source of those citations, ChatGPT was partially or completely wrong more than three-quarters of the time.

Even using ChatGPT to help with the writing process is risky. Ask Jeff Hancock, founder of the Stanford Social Media Lab and noted researcher of disinformation. The legal document he filed cited non-existent sources. Hancock insists that he wrote the document himself, but he used GPT-4o to compile its citation list, resulting in two citations made and one citation attached to the wrong authors.

Now, an AI defender might – rightly – say that a real journalist should check the answers provided by ChatGPT; that fact-checking is an important part of our work. I agree, that’s why I went with you on my own assessment of this article. But these are public and embarrassing examples of something that I think happens a lot in private: the average person uses ChatGPT and trusts the information it provides.

One advantage that old-school Google Search has over so-called search engines is that it links directly to primary sources. Answer engines just give you the answer, and it’s often not clear what the source is. For me, using ChatGPT or Google’s AI function creates more work — I have to check the response against the primary source; Good old Google Search just gave me that source directly.

But people who are less careful and more naive than me, which I suspect is the majority of people, simply stop at the answer and never check to see if it is correct. This, of course, is the purpose of a response engine – it’s what it’s designed to do. (Someone, perhaps someone involved with PerplexityAI, will now say, “ah, but we can explain identification with footnotes.” Same problem: who clicks on footnotes?)

This is all bad design, of course. Technology that works for people actually takes human behavior into account. Maybe there’s a way to make generative AI useful, but in its current state, I really feel sorry for anyone stupid enough to use it as a research tool.

I know people are sick of talking about glue on pizza, but I find the massive destruction of our information space that has taken place appalling. (Just search Amazon if you want to see what I mean.) This happens in small ways, like Google’s AI incorrectly saying that male foxes mate for life, and big ones, like spreading false information about a major news event. What good is an answering machine that nobody trusts?