User testing digital journeys against the FCA Consumer Duty Act
Supporting major UK insurance provider to meet to the FCA Consumer Duty Act deadline through rapid user testing
-
Tools
UserTesting.com
MS Office
MS Teams
Sharepoint
Float
Miro
-
Deliverables
UX insights/recommendations spreadsheet
Insights & recommendations report (per journey)
Prioritised backlog of actions
-
My Role
Discussion guide / screener question writing
Usertesting.com input
Leading user testing sessions
Expert reviews
Thematic analysis of findings
Report writing
Presentations to stakeholders
The Challenge
The Consumer Duty Act 2023 was introduced to prevent the exploitation of vulnerable customers and to make purchasing financial products more transparent for consumers. The client had just 10 weeks to demonstrate compliance or risk heavy fines from the Financial Conduct Authority.
In preparation for the implementation of the Financial Conduct Authority’s (FCA) Consumer Duty (CD) Act, the client’s team scoped 43 digital user journeys across multiple insurance products that needed to be evaluated. The challenge was to ensure all user journeys were compliant with the CD act before 31st July 2023.
-
The insurer must act to deliver good outcomes for their customers
-
An insurer must:
Act in good faith
Avoid causing foreseeable harm
Enable and support customers to pursue their financial objectives
-
The four key elements of the firm-consumer relationship:
The governance of products and services
Price and value
Consumer understanding
Consumer support
The focus for this project:
Due to the narrow timeframe of the project and limitations as to what could feasibly be assessed before the deadline, the focus for this project was constrained to only outcomes 3 (Consumer understanding) and 4 (Consumer support).
-
Firms should offer products and services that meet customer needs and are appropriate for their circumstances.
-
The cost of products and services should be reasonable and reflect the value received by customers.
-
Customers should have a clear understanding of products and services, including their risks and potential benefits.
-
Firms should provide helpful and accessible customer support.
Approach
Myself and a team of 3 other team members embedded into the clients UX research and design team to lead rapid qualitative user testing sprints. Each sprint focused on a specific user journey and included 12 participants, selected to reflect the diversity of the client’s customer base. These rapid user testing sprints involved creating tailored screener questions and discussion guides based on expert reviews of each journey, setting up and piloting unmoderated usability study sessions in UserTesting.com, thematic analysis of the data collected from the sessions, and creating a categorised and prioritised insights/recommendations report for each of the 43 journeys. We then conducted a initial workshop for each journey with the client’s compliance representative to rate each recommendation using the MoSCoW prioritisation method, delivered a structured insights and recommendations report with a light touch from the project QA, and individually presented a full report readout to all relevant stakeholders.
Research Questions
To ensure alignment with the FCA’s Consumer Duty Act, specifically outcomes 3 and 4. The user testing was designed around the following core questions:
Comprehension
Do participants understand the content on each page?
Do participants understand the available options, including the associated benefits, risks, and costs?
Do participants understand any actions required by them and any consequences of inaction?
Discoverability & Navigation
Is key information discoverable (e.g., not hidden within a blocks of text)?
Can participants easily navigate the journey to locate what they need?
Do they know how to access additional information or support, if required?
Effective Decision-Making
Can participants identify and do they understand the information needed to make effective decisions?
Do participants understand the key features, benefits, costs and risks of a product or service where they need to evaluate or make a choice about the product or service?
Disclosure & Regulatory Understanding
Do participants understand legislative, regulatory, and mandatory disclosures?
Support & Service Accessibility
Do participants understand how they can access any additional information or support they might need during and after purchasing a product? (For example, to make an enquiry or a claim)
For digital-only products:
Do participants understand that the product is digital-only?
Do participants understand how to get support for this product type?
Testing Methodology
Unmoderated testing:
(automated tests without the presence of a researcher) of all journeys using the live website and/or prototypes.
Test A: Unmoderated testing on Usertesting.com with 6 participants focused on outcome 3.
Test B: Unmoderated testing on Usertesting.com with 6 participants focused on outcome 4.
Moderated testing
(live sessions led by a researcher) of select journeys that have remaining research questions unanswered by unmoderated testing and/or lack deep insights.
Test C: Moderated 1-1 sessions via live conversation on Usertesting.com with 6 participants focused on both outcomes 3 and 4.
Expert review:
The research team conducted an expert review to address any journeys that had remaining gaps after the unmoderated and moderated testing, to ensure all Consumer Duty assessment areas were covered. (Pictured below)
Recruitment:
Participants were recruited through the UserTesting.com platform panel using a screener questions. The recruitment criteria included:
Aged between 18–65.
Must currently hold the relevant insurance product linked to the journey being tested.
Must have purchased the product within the past 6 months.
Must be the main policyholder.
There must be at least one participant per journey with an accessibility need.
Outcome & Impact
We produced a UX recommendations spreadsheet (shown left) documenting 300+ insights from the journey assessments, as well as a prioritised backlog of actions for product owners to progress. The impact of this work included:
Giving the client a clear audit trial to track and progress actions with product owners to ensure compliance with CD across all digital journeys.
Enabling the client to demonstrate to the FCA that they have carried out work to ensure their products comply with CD act 2023.
Provided the client with an understanding of the most critical user journeys and where the biggest UX issues were.
We also implemented an agile way of running simultaneous research sprints to gather user insights under a very tight deadline.
Pictured below are some example slides from one of the UX insight reports (left shows an example executive summary of recommendations, and right shows an example of an insight slide).
