CAN AI BE TRUSTED?

Your voice assistant hears you asking to unlock your phone. The phone unlocks, but it’s not you holding it.

Down the road, a self-driving car drives past a stop sign. At a hospital, an AI model misdiagnoses a patient. In a lab, an augmented reality map highlights neighborhoods at risk of floods, but only if it’s interpreted correctly.

These scenarios sound like science fiction, but they’re real situations UMKC researchers are studying to answer a question many are asking: Can AI be trusted?

For Rui Duan, Ph.D., assistant professor in the Division of Computing, Analytics and Mathematics, trust in AI begins with understanding its weaknesses. His research focuses on the security and privacy risks of AI systems, especially those that rely on voice recognition.

“Most AI applications are developed without serious evaluation of their robustness,” Duan said. “Our job is to find the vulnerabilities.”

Duan’s research shows just how easily AI systems can be compromised. In one study, his team demonstrated how voice assistants like Apple’s Siri could be tricked into misidentifying users. By making tiny, almost imperceptible changes, like adding a faint background layer and adjusting the voice pitch, the AI could be fooled into unlocking a phone for someone who wasn’t the owner.

“To a human, the audio still sounds like a different person’s voice,” Duan said. “But AI wasn’t able to detect the difference. This is a serious security flaw.”

Duan and his team identified a similar issue in another project, where they explored how YouTube’s copyright detection system could be bypassed by modifying songs in ways that sound like remixes to humans but would fool the AI into thinking it was a completely new and original song.

The same vulnerabilities that fool voice recognition and copyright systems also expose another AI risk: deepfake audio. Attackers can manipulate recorded speech to make someone say something they never actually did.

Duan is working on audio watermarking, a technique that embeds undetectable patterns into recordings, to mitigate the use of deepfake audio. Think of it like an invisible signature embedded in legitimate recordings. If a deepfake audio is created using watermarked material, the system can detect and flag it.

“It’s a proactive defense,” he said. “Audio watermarking will give us evidence the audio is being misused. This will allow us to quickly identify deepfake audios instead of having to guess if one is real or fake.”

How AI Makes Decisions

If uncovering AI’s weaknesses is the first step toward trust, understanding how it makes decisions is the next. That’s what Cheng Han, Ph.D., assistant professor in the Division of Computing, Analytics and Mathematics, is working on by looking inside the machine to make it more trustworthy.

Many AI systems operate as “black boxes,” generating predictions without revealing their reasoning.

“AI models are essentially numbers stacked together,” Han said. “They make predictions based on probabilities, but we can’t always explain how they got there. That’s a big issue, especially in high-stakes areas like autonomous driving and robotics.”

Han is approaching this issue by looking at the sustainability of AI models. Large models like ChatGPT are powerful but resource-hungry, requiring massive computing power and energy. His team is developing compressed, efficient versions that could run on personal devices and be trained more sustainably.

At the same time, Han is also looking at the explainability of AI models by “unboxing” them, allowing AI decision-making processes to be more understandable and interpretable. Using techniques like prompt tuning and multimodal representation tuning, his lab fine-tunes large models with minimal parameters, boosting performance while making their logic more accessible. The goal is to create a system that can solve new problems safely, reject unreasonable guesses and function predictably in the real world.

In one study, his team discovered how a tiny sticker placed on a stop sign in a car’s camera view could derail its actions entirely, exposing vulnerabilities in the AI models that integrate visual and linguistic inputs to guide complex tasks, also known as vision-language-action models. Small tweaks in the environment or subtle manipulations in multimodal data can lead to severe task failure.

“These models are powerful, but they’re also vulnerable,” Han said. “We need to understand how they make decisions so we can build in safeguards. AI still lacks human-like judgment.”

Even as AI becomes more integrated into autonomous systems, Han emphasizes that human supervision remains essential. “AI can help with productivity, but it needs human oversight,” he said.

Human Oversight of AI

That need for human oversight becomes even more important in fields like medicine, where mistakes can be life-threatening. Yugyung Lee, Ph.D., professor in the Division of Computing, Analytics and Mathematics, often collaborates with the UMKC School of Medicine and UMKC School of Dentistry to apply AI in clinical settings. Her lab uses AI to work with massive datasets from MRIs, CT scans and rehabilitation logs, helping doctors effectively diagnose patients and personalize treatment plans.

But even the most advanced AI models can hallucinate, leading to the generation of false information. In one instance, a large language model invented medications and test results that didn’t exist. Fortunately, human reviewers caught the errors before they reached clinicians.

“Complex large language models, despite being advanced, can still produce errors,” Lee said. “These hallucinations can be especially dangerous in medicine.”

To address this, Lee’s team is working on building a multi-agent system that would include a reasoning agent that supports responses with real data and an auditing agent that acts as a fact-checker, flagging hallucinations and inconsistencies in the data.

“The goal is not just to get an answer from AI, it’s to get an answer with evidence to provide doctors with better tools to treat patients,” Lee said.

Lee sees AI not as a replacement for humans, but as a collaborator.

“AI should be our collaborator,” Lee said. “It can dive deep into data, but humans provide context and judgment that AI simply cannot. The future belongs to people who can guide and prompt AI effectively while adding that human element of perspective that simply can’t be replicated.”

Partnering With AI

ZhiQiang Chen, Ph.D., professor in the Division of Natural and Built Environment, also incorporates the AI-human collaborative philosophy in his lab. Chen uses AI to monitor infrastructure and prepare for natural hazards like floods and earthquakes. Sensors embedded in infrastructure like bridges, roads and buildings collect massive amounts of data. AI analyzes that information, detecting everything from tiny cracks to widespread flood damage.

“AI helps us process the data from these sensors quickly, allowing us to make informed decisions,” Chen said. “It’s especially useful during extreme events.”

Chen’s team partnered with NASA to build a global flood monitoring system that can generate maps within days of a disaster, giving officials a faster, clearer picture of risk. Locally, they developed an augmented reality app in collaboration with Kansas City Water Services, showing residents how high water could rise on their streets.

“Terms like ‘100-year flood’ don’t mean much to most people,” he said. “But if you can see it on your phone, see how high the water might rise on your street, that changes how you think about risk and makes the severity of the situation more tangible.”

Chen is also building knowledge graphs that encode decades of engineering expertise into machine-readable formats. These tools can help engineers design safer structures and respond more quickly in emergencies.

But like his colleagues, he’s cautious about AI’s limitations.

“AI is a learning-based tool,” Chen said. “It doesn’t think like a human. It can’t replace engineering judgment, but by processing data quickly, it enhances decision-making and situational awareness. That’s how trust in AI begins.”

Chen is also an advocate for AI ethics and education, chairing an international committee focused on integrating AI into civil engineering curriculum.

“Students need to understand how to use these tools responsibly because these tools will shape the future of our communities,” Chen said.

Chen’s focus on AI education starts in his lab, where Ph.D. student Hongyi Ren has spent the past year applying artificial intelligence and large language models to flood disaster responses and risk assessments.

“Our goal is to help responders see what’s happening faster and more accurately,” Ren said. “We’re developing large multimodal models that can interpret satellite and drone imagery to identify affected areas, and we’ve built virtual and augmented reality tools that help people visualize the impact of flooding in their own communities.”

Throughout this research experience, Ren has discovered that AI can’t replace human judgment in high-stake situations like disaster response.

“AI cannot replace a human’s intuitive judgment skills. But AI should serve as a partner that enhances the human decision-making process by providing data-driven insights,” Ren said. “Even a few seconds saved through automation could lead to faster aid and more lives saved.”

So, can AI be trusted? Whether it’s detecting deepfakes, explaining robotic decisions, preventing hallucinations in healthcare or helping communities respond faster to natural disasters, UMKC researchers are showing that trust in AI must be earned, not assumed.