User Test Plan
Table of Contents
The system : GrubHub Mobile App
1. Goals
Improve the process of users making orders via the application (e.g. by providing more information about a dish).
Increase the chances of users ordering unknown dishes via the application.
Make the process more transparent between customers and restaurant proprietor.
2. Research Questions
- Do users know what they are ordering? How do users figure out what a particular dish looks like?
- How would a user find information concerning a dish?
- How easily and clearly is it for users to get information about the dish they want in GrubHub?
- What triggers the ordering/buying decision? What obstacles or usability flaws are encountered during the ordering process?
- Case A: the user already has something in mind
- Case B: the user doesn't know what to order
- How easy is it for users to find the total ordering process? What is the length of time that user spend during the ordering process?
- Do final orders match user original expectations?
3. Participant Characteristics
- Age
- Gender
- Frequency of use
- Experience of use
Potential users
- Know how to use a smartphone.
- Typically, they are busy, in a rush, they don't have the time to pick food by themselves.
4. Test Script
Participation introduction & consent:
Hi, everyone! My name is [name], and this is [other person present], they will be responsible for the recordings during this trial.
We would like to remind you this is not a test based on your individual skills, rather than a trail on the application at hand. With that in mind, when you are answering questions please try to think aloud and talk us through what you may normally do given the situation. Say what you're looking at, what you're trying to do, and what you're thinking. There are no criteria for your answer, and you don't need to worry about whether your answers or behaviors are right or wrong, just interpret the question, and take your time to respond and react. If you struggle with using or understanding the application, your feedback is valuable for us to make helpful changes to improve this application.
The session should take about an hour. If you need a break, just let me know at any point. If you have any questions during the session, feel free to ask them. I may not answer you immediately since we are acting mainly as observers, where we are interested in how people will react and solve the issue when nobody is sitting next to them to help, but if your issue puts a hindrance on the trail we will answer your question promptly.
We would like at this time to make you aware that what you say and do will be recorded strictly for our observations only. These recordings will not be used for any other purposes beyond this particular trial. [Hand over consent form]
Thank you for agreeing to participate in our research study today. We're working on the design for the mobile application GrubHub, and during the session, we'll ask you several specific questions to test which elements or functions could further be improved upon.
Explain any equipment:
Our test device is an iPhone 6, and it will be put in front of you before we begin the trail. GrubHub is an online and mobile food-ordering application, and it has been already been pre-installed onto the testing device. We will ask you to use the GrubHub application yourself in the session.
Pre-questions:
- Do you have any questions so far?
OK. Before we look at the application, I'd like to ask you just a few quick questions.
- First, what is your occupation? Could you summarize an average day for you?
- How much time do you typically spend using your smartphone, in specifics to web browsing, and other applications, besides texting and social media?
- What would be the split between web browsing and using applications? A rough estimate percentage wise?
- What type of applications do you use most frequently?
- Do you have any favorite mobile applications?
OK, great. We're done with the questions and we can start introducing our tasks.
Introduce tasks:
We have all together 7 tasks, and I'll give them to you one at a time.
Please think aloud as you work and I may prompt you to ask what you are thinking during the session. Once you are stuck and have attempted to fix the problem yourself, please tell us if you cannot continue without aid.
Do you have any questions before we begin?
[Task #1] We will begin our first task now.
[Present participant with the first task. Read it aloud also.]
[When complete.] Thank you. We'll now move to the next task.
[Task #7] This is our last task.
Task completion and questionnaire:
Thank you. We have completed all tasks.
I have a brief questionnaire here that I'd like you to complete. The information you provide is for our use only. Your name is not stored with the questionnaire data.
Debrief:
Once again, I'd like to say thanks for coming today. Your efforts help us to improve the GrubHub application.
Do you have any questions or comments for our test now that we're done?
5. Tasks List
Scenario : It is dinner time. You are at the point where you are hungry, but at the same time you are really busy, and simply do not have the time to pick up some food. You decide the best route would be to order food for delivery. To do so, you pickup your smartphone and load up the GrubHub app.
| # | Task | Data |
|---|---|---|
| 1 | Spend about 5 minutes playing with the app and get used to it and explain to us what you think of it and the conveniences that this app provides. Who do you think GrubHub is made for? | First usability, opinions, first impressions |
| 2 | Pick a dish you are familiar with, find a restaurant that provides the dish you are looking for, and find out how much the entire order will cost you. | Ease of use for a basic task: – What were the details of the criteria you used to make your decision? (e.g. flavor, location, price, rating, reviews) |
| 3 | Find a dish that you are not familiar with, or was recommended from a food genre that you never had. | Follow up: – How would you figure out what this dish is? – Describe what this product looks like? – What will you use to figure out the details of this dish? (e.g. Social media, search engine) – List the details (e.g. ratings, pictures, etc.) |
| 4 | Figure out what the dish looks like within and outside the app. | During the process: – What do you plan to do? – What do you think about the app? – Walk us through – What additional functions would be an added convenience to your overall experience? |
| 5 | Pick three different dishes you are familiar with (each in different cuisines) | – Why did you pick these particular dishes |
| 6 | Choose one you would buy | – Why this decision? |
| 7 | Repeat the same with dishes you are not familiar with (each in different cuisines) | – Walk us through you decision making – What information triggered the decision? |
6. Data & Evaluations measures
In order to answer the research questions, we will collect both performance and preference data during the test sessions:
Performance Measures:
- Number and percentage of tasks completed with and without prompts or assistance. [Assistance example: Participant complete the task when asked, "Could you please have a look at other areas in this page?"]
- Number of errors of omissions.
- Number of negative comments.
- Time to finish each task.
- Time of staying each page.
Preference Measures:
Ratings and rational concerning
- Usefulness of the product.
- How well does the product meet with participants' expectations.
- Ease of use overall.
- Appropriateness of site's functions.