Security News > 2023 > June > AI vs AI: Next front in phishing wars

The firm used its own AI models to determine that certain emails sent to its customers later identified as phishing attacks were probably AI-generated, according to Dan Shiebler, head of machine learning at Abnormal.

"The danger of generative AI in email attacks is that it allows threat actors to write increasingly sophisticated content, making it more likely that their target will be deceived into clicking a link or following their instructions," he said, adding that AI can also be used to create greater personalization.

According to Abnormal, another complication in detecting phishing exploits that used AI to craft emails involves false positive findings.

Because many legitimate emails are built from templates using common phrases, they can be flagged by AI because of their similarity to what an AI model would also generate, noted Shiebler who said analyses do give some indication that an email may have been created by AI, "And we use that signal to determine malicious intent."

Takes one to know one: Using AI to catch AI. Shiebler said that detecting AI authorship involves a mirror operation: running LLM-generated email texts through an AI prediction engine to analyze how likely it is that an AI system will select each word in an email.

"Legitimate emails can look AI-generated, such as templatized messages and machine translations, making catching legitimate AI-generated emails difficult. When our system decides whether to block an email, it incorporates much information beyond whether AI may have generated the email using identity, behavior, and related indicators."

News URL

https://www.techrepublic.com/article/ai-vs-ai-phishing-wars/