Security News > 2021 > February > Experts Find a Way to Learn What You're Typing During Video Calls
A new attack framework aims to infer keystrokes typed by a target user at the opposite end of a video conference call by simply leveraging the video feed to correlate observable body movements to the text being typed.
To achieve this, the recorded video is fed into a video-based keystroke inference framework that goes through three stages -.
Keystroke detection, which retrieves the segmented arm frames to compute the structural similarity index measure with the goal of quantifying body movements between consecutive frames in each of the left and right side video segments and identify potential frames where keystrokes happened.
Word prediction, where the keystroke frame segments are used to detect motion features before and after each detected keystroke, using them to infer specific words by utilizing a dictionary-based prediction algorithm.
"Due to recent world events, video calls have become the new norm for both personal and professional remote communication," the researchers highlight.
"However, if a participant in a video call is not careful, he/she can reveal his/her private information to others in the call. Our relatively high keystroke inference accuracies under commonly occurring and realistic settings highlight the need for awareness and countermeasures against such attacks."