How AI Protects (and Attacks) Your Inbox
When Aparna Pappu, vice president and general manager of Google Workspace, spoke at Google I/O on May 10, she laid out a vision for artificial intelligence that helps users wade through their inbox. Pappu showed how generative AI can whisper summaries of long email threads in your ear, pull in relevant data from local files as you salsa together through unread messages, and dip you low to the ground as it suggests insertable text. Welcome to the inbox of the future.
While the specifics of how it’ll arrive remain unclear, generative AI is poised to fundamentally alter how people communicate over email. A broader subset of AI, called machine learning, already performs a kind of safety dance long after you've logged off. “Machine learning has been a critical part of what we’ve used to secure Gmail,” Pappu tells WIRED.
A few, errant clicks on a suspicious email can wreak havoc on your security, so how does machine learning help deflect phishing attacks? Neil Kumaran, a product lead at Google who focuses on security, explains that machine learning can look at the phrasing of incoming emails and compare it to past attacks. It can also flag unusual message patterns and sniff out any weirdness emanating from the metadata.
Machine learning can do more than just flag dangerous messages as they pop up. Kumaran points out that it also can be used to track the people responsible for phishing attacks. He says, “At the time of account creation, we do evaluations. We try to figure out, ‘Does it look like this account is going to be used for malicious purposes?’” In the event of a successful phishing attack on your Google account, AI is involved with the recovery process as well. The company uses machine learning to help decide which login attempts are legit.
“How do we extrapolate intelligence from user reports to identify attacks that we may not know about, or at least start to model the impact on our users?” asks Kumaran. The answer from Google, like the answer to many questions in 2023, is more AI. This instance of AI is not a flirty chatbot teasing you with long exchanges late into the night; it’s a burly bouncer kicking out the rabble-rousers with its algorithmic arms crossed.
On the reverse side, what’s instigating even more phishing attacks on your email inbox? I’ll give you one guess. First letter “A,” last letter “I.” For years, security experts have warned about the potential for AI-generated phishing attacks to overwhelm your inbox. “It’s very, very hard to detect AI with the naked eye, either through the dialect or through the URL,” says Patrick Harr, CEO of SlashNext, a messaging security company. Just like when people use AI-generated images and videos to create fairly convincing deepfakes, attackers may use AI-generated text to personalize phishing attempts in a way that’s difficult for users to detect.
Gloss