Designing an AI tool to identify illegal digital imagery at scale while reducing the psychological impact on human analysts.

The brief

Sharing of illegal digital imagery has grown at an alarming rate in recent years with the rise of cheap and easily anonymised file sharing tools.

We worked with Qumodo to improve the user experience of their software – a tool that allows governmental organisations to process high volumes of illegal imagery by combining human expertise with AI and machine learning. We were also tasked with creating a new design identity and redesigning key areas to onboard a new non-profit partner looking to use the software. We also helped embed some useful new design practices and ways of working.

The brief was to provide a redesign that improved upon key functionalities and solve existing usability issues, as well as adapting the software for the new non-profit partner.

The challenge was to turn a highly complex process into an intuitive, easy to use experience. The goal was both to enable the analysts to do their work and, importantly, to reduce the psychological impact that comes with screening large volumes of illegal imagery. A key commercial consideration in the design was the extensibility of each feature so that they could be marketed as standalone products as part of a suite of tools.

Understanding the challenge

The first task was to understand the non-profit partner's workflow, current technology setup and overall organisational aims and pain points. How could our software match and improve their way of working and help them better deal with high volumes of illegal imagery? We held a workshop with the non-profit's key stakeholders to map out their current process, uncover key pain points and future business aims to inform our approach to the redesign.

Working collaboratively with our client's in house product team, we created user personas and mapped out the nonprofit's end-to-end journey in order to understand how and where the software could best fit into their process and what efficiency improvements could be made.

Alongside this, we set about gaining a clear understanding of the current software, any already identified pain points, usability issues and previous research and design work done. We combed through user feedback, academic research and process maps. We also conducted a heuristic audit to get an objective sense of the current usability of the platform.

From here we set about creating an interactive document (using design collaboration tool Miro) to function as the source of truth for understanding the real world problems the software looked to solve, the target audience, customer journeys and design practices. This included creating a mission statement, personas, storyboards and an extensive user experience map that aligned both customer journeys before ideating design solutions.

Mapping out and accounting for users' emotions became a core part of the work towards reducing psychological impact.

Our solution

A key design constraint was that we were restricted to designing for use on one screen (i.e. not multiple split screens). The core UX challenge revolved around designing for the sorting through of large volumes of imagery more efficiently with the ability to quickly tag, categorise and search for patterns without losing sight of the main task. Not only this, but also building trust, purpose and value between the AI and the users - creating a symbiotic relationship between the two.

The resulting design was a high-fidelity prototype alongside a new modular design language that addressed key usability issues, improved navigation, a more intuitive and user friendly UI and allowed for more flexible workflows that fit with the nonprofit partner's workflow.

The outcome

The work we delivered will be further iterated and developed by Qumodo as they look to expand their capabilities and grow their customer base. This includes adding new features and capabilities, applying the design principles we established across the product and doing further research to better understand their users’ changing needs.

Ready to talk about your project?

Contact us