The potential for constant stream of data to be intercepted or misused is a serious concern
In the early 90s, the song “Things That Make You Go Hmmm” captured a feeling of unease when something felt out of place. This sentiment is highly relevant today, as technology is creating new ways for things to feel “off.” The difference now is that these unsettling feelings are often tied to invisible systems we don’t fully understand. We bring powerful AI agents into our lives for convenience, but the questions of who is listening, what is being collected, and how this data is being used are becoming impossible to ignore.
Digital assistants like Siri, Alexa, and Google promise to make our lives easier, but they must be constantly listening for their wake words. This means every conversation, every argument, and every confidential meeting passes through their microphones. While some audio might stay on the device, a significant portion is sent to the cloud. The critical question isn’t just what is heard, but who else has access to that audio. The potential for this constant stream of data to be intercepted or misused is a serious concern. The use of Large Language Models (LLMs) like ChatGPT introduces an even deeper level of data collection. People are now entrusting these systems with their most sensitive information, including business contracts, medical records, financial data, and personal conversations. As users interact with these systems, they build detailed profiles of their behavior, thinking patterns, and interests. In the hands of a surveillance program or a hostile entity, this data is not just useful—it’s a blueprint for predicting, influencing, and even controlling individuals.
Another major issue is the power of AI to shape our understanding of reality. When an AI agent provides information, it is often accepted as fact, and when it claims to have no knowledge of something, that event effectively disappears from our collective record. If this is based on selective training data or moderation rules, the AI becomes a quiet tool for censorship and narrative control. For example, if an assistant gives a detailed account of some military operations but insists another, equally real one, does not exist, it’s not just a matter of a knowledge gap; it’s a form of information control.
The risks extend beyond data and information. With smart devices and vehicles now part of a shared network, the potential for catastrophic compromise is real. If a malicious actor could gain control of a network of connected vehicles, they could alter critical systems, change routes, or even control braking. A coordinated attack could turn transportation systems into a weapon, posing a massive threat to public safety and infrastructure. Our personal identities are also at risk. AI no longer needs hours of audio to clone a voice; a few seconds of clean speech is enough. Combined with access to emails, chat logs, and other communications, an AI can mimic a person’s writing style with uncanny accuracy. This creates a powerful tool for impersonation and fraud, allowing a fake version of you to approve payments, join conference calls, or cancel contracts.
Perhaps most concerning is our growing dependence on these systems. Many people blindly copy and paste AI-generated content without reading it, checking it, or verifying its accuracy. If the AI fabricates facts or distorts data, these errors go out untouched. We are not just outsourcing work; we are outsourcing critical thinking and when that skill fades, we become increasingly vulnerable to manipulation. The more we rely on these systems, the less we challenge them. This makes it easier for a hostile actor to turn our own tools against us, leaving us to wonder who is truly in control.
By – Kaushal Beda, Pelorus Technologies Director -Govtech
