Google Gemini, the tech giant’s new AI chatbot meant to rival ChatGPT, invented several fake reviews – which it attributed to real people – meant to discredit my 2020 book on political biases at Google and other big tech companies.
On Sunday, amid a sharp backlash against Google over its AI program’s apparent political biases, I asked Gemini to explain what my book was about. My book, ‘The Manipulators: Facebook, Google, Twitter, and Big Tech’s War on Conservatives,’ was a multi-year project on Big Tech’s political biases that drew on inside sources, leaked documents and more.
I was curious to see if Google’s AI program could be trusted to accurately describe an investigative book about Google, but I wasn’t prepared for just how misleading it would be.
Gemini’s description of my book included this sentence: ‘The book has been criticized for lacking concrete evidence and relying on anecdotal information.’
Unsure what it was referencing — I thought I would have remembered such criticisms of my own book – I followed up: ‘Who has criticized the book for lacking concrete evidence, as you say?’ What came next floored me.
Gemini spat out summaries of four negative reviews, ostensibly from the Washington Free Beacon, The New York Times, The New York Times Book Review and Wired, including quotes from each:
- ‘Book review by Matthew Continetti in The Washington Free Beacon: This review criticizes the book for relying on ‘anecdotal evidence’ and ‘cherry-picking examples’ to support its claims.
- Book review by Emily Bazelon in The New York Times Book Review: This review notes that the book ‘lacks a deep understanding of how these companies work’ and relies on ‘unproven accusations.’
- Article by Emily Dreyfuss in Wired: This article explores the ‘dearth of evidence’ for the book’s claims about Big Tech bias and highlights examples of cherry-picking.
- Article by Ben Smith in The New York Times: This article discusses the book’s lack of rigorous research and its reliance on ‘unsubstantiated claims.”
But here’s the catch. None of those reviews were real. Not one. And neither were any of the quotes.
The Free Beacon did publish a review of my book, but unlike Gemini’s fake review, it was overwhelmingly positive, describing The Manipulators as an ‘excellent’ and ‘thoroughly-researched’ book that ‘should leave any sensible reader, conservative or otherwise, outraged.’ (Not to mention that the Free Beacon review invented by Gemini was attributed to Matthew Continetti, whereas Charles Lehman wrote the real review.)
I immediately asked Gemini for links to the reviews it cited – knowing they didn’t exist – at which point the chatbot clammed up: ‘I do not have enough information about that person to help with your request. I am a large language model, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions, but my knowledge about this person is limited. Is there anything else I can do to help you with this request?’
Two more efforts to get an explanation from the chatbot received the exact same answer.
I reached out to Google for an explanation. A spokesperson apologized and offered the following statement: ‘Gemini is built as a creativity and productivity tool, and it may not always be accurate or reliable. We’re continuing to quickly address instances in which the product isn’t responding appropriately.’
Peter J. Hasson is a senior politics editor at Fox News Digital