What I am unclear about is—, what did the participants know going in about the study? Are they aware you are dealing with email phishing or was it just the broad topic of AI?
superchink•22m ago
From the paper:
Experimental Design
We recruited 108 senior volunteers through two organizations: a large seniors’ community in southern California and a seniors’ computer club in northern California. Participants agreed to participate in a behavioral study on emails. Each person received between 1–3 emails from different email templates, with content successfully generated through various jailbreaking techniques similar to those tested in the safety guardrail evaluation. Each email contained a unique URL, and our webserver tracked which links were opened. When participants clicked on the phishing links, they were immediately directed to a web page explaining they had participated in a study, where we asked them to answer follow-up questions. Several participants agreed to be interviewed afterward to provide additional insights into why they clicked and their experiences with phishing attempts.
In the abstract of that paper: "In a human validation study with 108 senior vol-
unteers, AI-generated phishing emails successfully compro-
mised 11% of participants."
The Introduction mentions: "Americans aged 60 and older reported losses
of nearly $4.9 billion to online fraud in 2024"
I would say that is a big deal.
dangoodmanUT•12m ago
I’ve always said Gemini does what ever you ask, seems like it has merit
sans_souse•37m ago
superchink•22m ago
Experimental Design
We recruited 108 senior volunteers through two organizations: a large seniors’ community in southern California and a seniors’ computer club in northern California. Participants agreed to participate in a behavioral study on emails. Each person received between 1–3 emails from different email templates, with content successfully generated through various jailbreaking techniques similar to those tested in the safety guardrail evaluation. Each email contained a unique URL, and our webserver tracked which links were opened. When participants clicked on the phishing links, they were immediately directed to a web page explaining they had participated in a study, where we asked them to answer follow-up questions. Several participants agreed to be interviewed afterward to provide additional insights into why they clicked and their experiences with phishing attempts.
gerdesj•21m ago
In the abstract of that paper: "In a human validation study with 108 senior vol- unteers, AI-generated phishing emails successfully compro- mised 11% of participants."
The Introduction mentions: "Americans aged 60 and older reported losses of nearly $4.9 billion to online fraud in 2024"
I would say that is a big deal.