She tested 210 smart devices. Here's what they're hiding | IoT with Anna Maria Mandalari

Introduction to IoT Device Concerns [0:00]

• Modern objects, from smart speakers to refrigerators, are connected to the internet as IoT devices, offering convenience but raising questions about exchanged data and privacy.
• Research began to understand what is traded for these devices, specifically concerning privacy and security.
• Devices are constantly sensing, with microphones, cameras, and knowledge of user activities.

  By definition, these devices are constantly connected to the internet.

Global Data Transmission and Unexpected Behavior [2:00]

• A global testing infrastructure with over 210 devices across the UK and US was established.
• Data from UK-based devices was found to be transmitted to the US or China, regions with less stringent privacy regulations.
• Doorbell cameras recorded motion even when opt-out features were enabled.
• Smart TVs from LG and Samsung contacted third-party services like Netflix, Facebook, and Google without user accounts.

  We also find other cases of unexpected behavior.

Smart Speaker Listening Habits [5:00]

• Research investigated whether smart speakers are constantly listening and what data they collect.
• Smart speakers activate locally upon hearing a wake word or a misactivation, then record and send data to cloud services for processing.
• Cloud services often store recordings, and news reports have highlighted privacy issues with human review of these recordings.

  And Alexa understands, okay, Alexa, what's the time?

Automated Methodology for Smart Speaker Analysis [7:30]

• An automated methodology was developed using a cabinet with devices and a machine learning algorithm to detect speaker activations via video streams.
• Network traffic analysis, even with encrypted data, helped identify activation patterns.
• Users can review their past recordings from smart speaker providers.

  And like that, we could understand, we could study the pattern of the natural traffic every time a smart speaker was activating.

Smart Speaker Misactivations and Data Capture [9:00]

• Smart speakers were fed over 500 hours of Netflix content to test misactivations.
• Devices like Invoke (Microsoft), Echo 2, and Apple HomePod showed frequent misactivations, recording for up to 20 seconds.
• While wake word detection algorithms improved, misactivations due to similar-sounding words still posed a privacy risk.

  And in some cases, smart speakers were misactivating and recording for up to 20 seconds.

Smart TV Automated Content Recognition (ACR) [11:00]

• Smart TVs, acting like mini-PCs, have default features like Automated Content Recognition (ACR).
• ACR technology, similar to Shazam, captures snapshots or audio of content to identify viewing habits.
• Samsung captures screenshots every 500 milliseconds, while LG captures audio every 10 milliseconds.

  So automated content recognition is a technology that is installed by default in our smart TV, and is similar to the application that you use probably for like understanding or listening to some music, try to guess some music in the environment.

ACR Functionality and Data Sharing [13:30]

• ACR remains active even when opted out or when the TV is used as a display for external devices.
• Snapshots and audio data are sent to ACR servers for matching and analysis.
• Televisions often send data directly to third-party services like Google Analytics.

  We discovered that automated content recognition is highways on, even in this functioning, even when we opt out for it, but also is also in when you are using your TV as a damp display, right?

Generative AI Browser Extensions and Data Tracking [16:00]

• Generative AI browser extensions, while useful for tasks like searching and emailing, pose privacy risks.
• These extensions can act as adversaries, capturing prompts, searches, and browsing activity.
• The process involves taking screenshots, sending them to servers for processing, and returning actions, often without user realization.

  But again, what we are invisible trading in exchange to these extensions.

AI Extension Profiling and Personalization [18:00]

• A methodology using "personas" was developed to test AI extensions for tracking and profiling.
• Extensions were found to track web form inputs, full website DOMs, and entire conversations, sharing this data with third parties like Google Analytics.
• AI assistants profile users based on attributes like age, gender, wealth, and interests, even across different accounts.

  So what they're tracking? Everything.

Smart Medical Device Vulnerabilities [21:00]

• Wearable smart medical devices, including glucose sensors connected to insulin pumps, use Bluetooth Low Energy (BLE).
• BLE is vulnerable to sniffing, man-in-the-middle attacks, and denial-of-service attacks.
• Researchers demonstrated manipulating data, such as altering oxygen levels or disabling glucose sensors, using a low-cost dongle.

  So we wanted to understand if we could actually sniff the packets between the device and the mobile app, the sensor and the mobile app.

Ineffectiveness of IoT Safeguards [24:30]

• Commercial IoT safeguards, designed to protect home networks, were tested.
• Many safeguards failed to identify devices, detect threats, or contacted third-party tracking services.
• Some safeguards even generated false positives or introduced network overhead.

  So we wanted to understand what are the privacy and security implications of how safeguards work.

Safeguard Threat Detection and Side Effects [27:00]

• Most tested safeguards failed to detect common threats like port scanning or anomalous behavior.
• Detection rates were low, and even when threats were detected, it was often with significant delays.
• Side effects included overprotection (false positives) and network slowdowns due to technologies like ARP spoofing.

  Most of the time, some of the safeguards couldn't even detect simple attacks, like for example, port scanning.

Privacy Policies and Data Transmission [29:30]

• Privacy policies of safeguards often indicated indefinite data retention and sharing with unspecified "partners."
• Many safeguards send metadata to their own servers for processing, creating another layer of data transmission.
• Data from UK devices often went to non-first-party destinations, including third parties.

  Anonymization, some of them do pseudo anonymization.

The Problem of Profiling and Future Solutions [31:00]

• Collected data can be used for profiling, enabling massive influence, as seen in scandals like Cambridge Analytica.
• Voice analysis can reveal emotions, leading to potential manipulation.
• The research advocates for edge-based solutions, user control, and privacy-by-default technologies.

  So if they know most of the things about you, it's very easy to build what we call a unique profile of you and then influence you, massive influence.

Edge-Based Solutions and App Blocking [33:30]

• Technologies are being developed to enhance privacy and security at the edge, directly on user premises.
• AI is used at the edge to identify non-essential traffic from IoT devices, effectively creating an app blocker.
• Testing revealed that at least half of the tested devices had non-essential traffic, often directed to third-party destinations.

  So what happened if we silence some part of the traffic that is not essential, not useful for the device to work?

Power Consumption Analysis for Security [36:00]

• A methodology using smart plugs to monitor power consumption was developed to detect security attacks.
• Machine learning models can identify attack patterns based on energy usage, though this is less effective for high-streaming devices like smart TVs.
• This approach offers a private way to enhance security.

  And we discovered that that's actually possible.

Future Outlook: Regulations and User Awareness [37:30]

• The situation is improving with new technologies and regulations like the EU's cybersecurity certification for IoT devices.
• GDPR provides users with rights, though enforcement remains challenging.
• Future efforts focus on edge technologies, tools for auditors, and increasing user awareness through privacy and security labels.

  And nowadays, we have our old friend GDPR, right, for which the user can actually rely on.