DRCF research finds that while AI brings benefits to consumers, including saving them time and money, consumers struggle to understand or manage risk themselves
Even consumers with extensive experience using AI and high levels of technical knowledge may be vulnerable to harm, according to the DRCF's Consumer Interest and AI report.
The report offers insights into how consumers understand and experience the benefits and risks of AI, through analysis of responses from industry and consumer groups to the DRCF’s call for input, as well as relevant literature.
DRCF research finds that as AI is increasingly prevalent in everyday life, consumers are recognising its benefits including cost and time savings, more access to information and personalised advice and recommendations.
However, AI can both exacerbate the existing risks of digital technologies, such as bias and privacy concerns, and introduce novel ones such as providing inaccurate information (hallucination) or manipulating how people make decisions. The advent of agentic AI, where AI moves from advising to taking action, is raising new questions about consumer vulnerability, trust and risk tolerance.
The report raises particular concerns about consumer trust formed relationally through anthropomorphic or artificially empathic design: care is needed to ensure that consumers are not vulnerable to harm as a result of misplaced trust, for example because a system appears authoritative or emotionally responsive. Young people may be particularly vulnerable.
The report outlines key components that can establish consumer trust, including having human oversight and support, clear explanations of how AI is being used, and user controls, including the ability to opt out. Accuracy, responsible data use, and the appropriateness of the use case are also important.
Different groups are also found to perceive risk differently, and risk tolerance varies in different situations. Women and older consumers are less likely to trust AI, but those who are financially vulnerable are more willing to use AI. People's lived experience and outcomes inform their assessments of the quality of AI.
However, just because someone uses AI extensively, that doesn’t mean they have an informed understanding of how much to trust it. The research suggests that even the most technically literate users can be vulnerable, for example when subject to financial pressure or emotional stress.
Additionally, consumers sometimes assume that they are protected by regulation when in fact they are not. For example, although financial and debt advice is generally regulated, similar services offered by AI may not be.
Accountability and routes to redress for AI harms are not yet fully clear: the research highlights the importance of audit trails or decision logs, clear allocation of responsibilities, and clear routes for consumer complaint and redress.
Overall, it’s hard for consumers to understand when they need to protect themselves against risks and how to do so. This may lead to them either being over-cautious and so failing to reap the benefits of AI, or over-confident and so putting themselves at risk of harm. Consumers shouldn’t be expected to understand and mitigate the full risks of AI themselves.
Key findings include:
- Experience using AI does not eliminate vulnerability. Even heavy users of AI can be exposed to harm, depending on how and where they use it.
- Greater use of AI does not equate to informed trust or acceptance of risk. Consumers may use AI extensively without understanding its technicalities.
The report forms part of the DRCF's ongoing work programme on consumer interest and AI. Its findings will inform the next phase of engagement with regulators, policymakers, and industry to focus on practical ways to help consumers benefit from AI while protecting them from harm.
This will include exploring effective consumer controls and human oversight, as well as mechanisms to ensure clearer transparency and accountability. The DRCF will continue to work with regulators, consumer groups and civil society to ensure consumers’ experiences and needs remain at the heart of this work.
For further information, contact the DRCF press office at DRCF@ofcom.org.uk.
AI has enormous potential to make our everyday lives easier – many people are now using it regularly, and we are already seeing the benefits it can bring through time and cost savings. "Our research shows that even the most technically and AI savvy consumers can be at risk when using AI. To maintain trust in AI, consumers need to be confident that safeguards are in place, and that they can find help if something goes wrong. “It’s essential that we continue to monitor and understand where consumers feel protected, and where they don’t, so that we can build and maintain trust in these systems.
Kate Jones, CEO, DRCF