Designing user experience for AI applications

Who this is for and what you will get

This guide is for developers building AI applications and for people who design their products. By the end you will be able to check your own app against a short list: is it useful, is it reliable, does it work for users with different abilities, do people want to use it, and does it help users trust it at the right level?^[1]

What user experience covers

User experience (UX) describes how a user interacts with and uses a specific product or service, whether that is a system, a tool or a design.^[1] When you build AI applications, you need to make the experience effective and also ethical, so that the system addresses real user needs.^[1]

A good application is useful, reliable, accessible and pleasant.^[1]

• Useful: the functionality must match the purpose of the app. An app that automates grading should assign scores accurately and efficiently against predefined criteria, and an app that generates revision flashcards should produce relevant and diverse questions.^[1]
• Reliable: the app should perform its task consistently and without errors. AI, like people, is not perfect, so you need to plan for errors and for moments when a person must step in and correct the result.^[1]
• Accessible: extend the experience to users with different abilities, including people with disabilities, by following accessibility guidelines and principles.^[1]
• Pleasant: the app should be enjoyable to use. An appealing experience encourages users to return, which can also increase business revenue.^[1]

Not every problem needs AI. AI is best used to augment the experience, for example by automating manual tasks or personalizing it for each user.^[1]

Trust: avoid both mistrust and overtrust

Trust is central to AI design. Users need to believe the application will get the work done, deliver consistent results, and produce results they actually need. There are two risks. Mistrust means a user has little or no trust in the AI system and rejects the application. Overtrust means a user overestimates what the AI system can do.^[1]

An example of overtrust: if an automated grading system seems accurate, a teacher may stop checking some of the papers. The grades could then be unfair or inaccurate, and the teacher could miss chances to give students feedback and help them improve.^[1]

Two ways to put trust at the center of the design are explainability and control.^[1]

Explainability: help users understand the output

Explainability means understanding how an AI application makes its decisions. When AI informs decisions in education, teachers and parents especially need to understand that process. Design for it by adding details that show how the AI reached its output, and make clear to the audience that the output was generated by AI and not by a human.^[1]

Wording matters. Instead of a line such as “Start chatting with your tutor now”, a clearer line says that the AI tutor adapts to the learner's needs and helps them learn at their own pace. The second version tells users what they are talking to and how it will work with them.^[1]

An AI for students can be designed not to reveal answers directly. It can instead guide the user to think through how to solve the problem.^[1] Describe what the app can and cannot do in simple language, because students and teachers may not be AI experts.^[1]

Control: let users shape the result and the data

Generative AI creates a collaboration between the AI and the user. Users can change prompts to get different results, and they can edit the output after it is generated. That sense of control is part of trust.^[1]

Microsoft Copilot, formerly Bing Chat, is one example. Users can tailor a prompt by format, tone and length, and they can add changes to the output and modify it. Copilot also lets users opt in and opt out of the data the AI uses. A student might want to combine their own notes with a teacher's resources for revision, and choose which materials are included.^[1]

One aim of these features is to prevent overtrust and unrealistic expectations. One suggestion is to create friction between prompts and results, for example by reminding the user that this is AI and not a fellow human.^[1]

Feedback and error handling

AI applications are bound to make mistakes. They should be built to receive feedback from users and to give feedback to them. That helps the system improve and builds trust. A feedback loop can be as simple as a thumbs up or thumbs down on each output.^[1]

The app also needs to communicate its capabilities and limits clearly. An app trained on History and Math may not be able to answer questions about Geography. In that case it can say which subjects it was trained on and that it cannot answer the question.^[1] Keep error handling simple and easy to explain.^[1]

A checklist for your own app

Take an AI app you have built and check it against these four points:

1. Pleasant: add explanations where users need them, encourage exploration, and word error messages carefully.^[1]
2. Usability: if the app is a web app, make sure the main tasks can be done with a mouse and with a keyboard.^[1]
3. Trust and transparency: do not treat the AI output as complete. Consider how a person can verify it, and what other measures could make the app more transparent.^[1]
4. Control: give users control of the data they provide, and offer a way to opt in and opt out of data collection.^[1]

Limits and common questions

These principles do not replace testing. They describe what an app should do, but the exact wording, button placement and feedback method still need to be checked with real users. Where this matters depends on the app, so judge each point in its own setting.^[1]

Next steps

If you are new to this area, start with the basics of user experience and design thinking. To go further on the security side of AI applications, continue with the material on securing AI applications.^[1]

Adapted from lesson 12 of Microsoft's open-source course *Generative AI for Beginners*^[1], released under the MIT License.