User Test Report
Table of Contents
- Introduction / About this report
- Executive Summary
- Methods
- Participants
- Summary of Results
- Recommendations
- Conclusion
1. Introduction / About this report
This report focuses on user testing of the GrubHub mobile application. GrubHub is a Chicago based company, founded in 2004.
According to GrubHub’s website: “Grubhub is the nation’s leading online and mobile food ordering company dedicated to connecting hungry diners with local takeout restaurants. The company’s online and mobile ordering platforms allow diners to order directly from more than 50,000 takeout restaurants in over 1,100 U.S. cities and London. Every order is supported by the company’s 24/7 customer service teams. Grubhub has offices in Chicago, New York and London.”
This mobile application allow users to order food for delivery. The selection of restaurants and food is based on their location. The app offers different filters that help users by narrowing down the selection of available restaurants by their prefered food or cuisine. The users can then place the order directly from his or her smartphone or computer to be delivered.
Regarding the user report, the first part describes the methods applied to the conducting the user test; then we will give a description of our participants and a summary of the relevant characteristics. In the third part, we will provide a summary of the results, overall and by tasks, and give the participants feedback. Finally, we will provide recommendations based on our observations and the results of our individual user tests.
2. Executive Summary
Our group conducted four face-to-face user tests toward GrubHub separately during Feb. 8th to Feb 19th. The purpose of our user tests was to assess the usability of GrubHub, the process of users making orders, and then to retrieve and analyze user experience data. We combined our individual records and data, and then contributed to this final test report towards GrubHub.
Four participants took part in our user tests, and they were asked to finish seven tasks we assigned. Each user test lasted around 45 minutes.
In General, most participants quickly learned the basics of how to operate GrubHub in terms of searching for local restaurants, finding certain cuisines, and being able to checkout.
However, the test identified several problems which influenced user experience and decisions they made:
- Users struggle when they don’t know the descriptions and pictures of unfamiliar dishes.
- The review function is not obvious.
- User cannot copy the name of food directly from within the application.
- User may rely on a third-party application to figure out what the unfamiliar food contains and looks like.
This report contains the analysis of four participants, summary of results, satisfactions ratings(from the comments), task completion rates, errors, participant feedback, and recommendations for improvements.
3. Methods
In this section, we will summarize the methods we used to run our user tests.
Participants were friends or relatives that answered the characteristics defined in the “User Test Plan”.
The participants answered the characteristics defined in the plan for the potential users:
- Know how to use a smartphone and basic applications
- Typically, they are busy, in a rush, they don’t have the time to pick up food for themselves.
The decisions of the tasks we are proposing in this plan have been made to answer the research questions. Some of them are:
- Do users know what they are ordering? How do users figure out what a particular dish looks like?
- How would a user find information concerning a dish?
We expect the following results:
- Users will struggle to find out what a dish looks like, especially if they have never experienced the certain dish
- Users will tend to use an outside source, such as a search engine, to find out what a dish looks like
- Order decisions are made regarding the ratings, the prices, and the popularity of dishes
The user test we ran, can be decomposed in three main parts:
- A pre-questionnaire: allows to collect information about the participant background, his or her savviness with smartphones
- A set of seven tasks
- A debrief: allows to collect the participants’ feedback on the overall test and the GrubHub application
The complete plan can be found on our portfolio: User Test Plan
4. Participants
Number of participants: 4
The participants have been selected based on the criteria defined in the user test plan. We will provide a description of our participants in the following section. Only three characteristics are relevant:
- Frequency of use: how often does the participant uses his or her smartphone, expressed in a percentage of the free time;
- Experience of use: what experience does the participant have with a smartphone, ranked between “low” (almost none) and “high” (savvy);
- Profile: what are the participant main occupation in life (ie. is the participant often in a rush)
These three are summarized at the end of this section. The other characteristics allow us to have more background but are not relevant to the completion of the test.
Description of the participants
| # | Guillaume Boulanger | Denise Nunes | Jie Chen | Kayla Neetz |
|---|---|---|---|---|
| Gender | Male | Female | Female | Female |
| Age | 22 | 33 | 25 | 24 |
| Citizenship | French | American | Chinese | American |
| Frequency of use (smartphone) | 40% of his free time. (70% web browsing / 30% apps) | 80% of her free time. (50% web browsing / 50% apps) | 70% of her free time. (30% web browsing / 70% apps) | 75% of her free time. (80% web browsing/ 20% apps) |
| Experience of use | High | High | High | High |
| Profile | IIT graduate student. Co-founder of a startup. Two activities take his majority of time (makes him a good potential users). | Mother of one, versatile nurse that works for Rush hospital. | Graduated in 2015, now a junior product manager, always working overtime | Graduated in 2015 from Columbia College of Missouri in Business Administration and Management, currently a Technical Recruiter at IT Associates, usually works through lunch |
Summary of the characteristics
| Description | Value | Comments |
|---|---|---|
| Frequency of use | Average 73.3% of the free time | Our participants use relatively often their smartphones (more than the half of their free time) |
| Experience of user | High | Our participants are savvy with the usage of a smartphone. This has to be kept in mind regarding the completion of the following tasks (eg. participants may not encounter problems that a smartphone novice may have) |
| Profile | “Busy schedule” | Our participants have a busy schedule which makes them appropriate users for this test |
5. Summary of Results
5.1. Overall Task Completion
- No problems searching for familiar dishes
- Completion of the tasks without any major issues
- Struggle with searching for a picture of an unfamiliar dish
5.2. Results by tasks
Seven (7) tasks were assigned to the participants. Here is a summary of the results by tasks:
Task #1: Spend about 5 minutes playing with the app and get used to it and explain to us what you think of it and the conveniences that this app provides. Who do you think GrubHub is made for?
| Name | Results |
|---|---|
| Boulanger | - No problems to understand the way to use the application - Navigation through the menus, tabs, and filters |
| Nunes | - First impression of the usability of the app wasn’t intimidating |
| Jie | - Quickly learned the basic operations on GrubHub - Found two unfriendly designs and one surprise. |
| Kayla | - Seemed simple to use at first, but the interface was a bit overwhelming and misguided - Could not search for restaurants locally, so had to switch to Chicago, specifically IIT |
Task #2: Pick a dish you are familiar with, find a restaurant that provides the dish you are looking for, and find out how much the entire order will cost you.
| Name | Results |
|---|---|
| Boulanger | - No problems to complete this task - Users used the cuisine button to select a known cuisine, then narrowed down the results using filters, and finally selected one of the first rated restaurant and picked a dish - Only check the ratings, not the reviews |
| Nunes | - Phase One: User inputs home address first, as she always does, searches for a cuisine she’s familiar with, searches for a restaurant that delivers - Phase two: Final, from the menu, she searches for the dishes that contains lamb by selecting the lamb item |
| Jie | - She did not use filters function, directly finding dish in the searching bar. - It took her a while to hesitate choosing a result after she searched, considering coupon, price and ratings. At last, she chose one with medium price but high rating. |
| Kayla | - Searched for burger in search bar, and was not aware or acknowledged cuisine button or filters - Did not consider ratings or reviews, more or less just wanted to pick something that appealed based on restaurant name - Wanted to change order and restaurant since order was too expensive |
Task #3: Find a dish that you are not familiar with, or was recommended from a food genre that you never had.
| Name | Results |
|---|---|
| Boulanger | - Referred to an unknown cuisine (using the menu of the same name) - Only look at the most popular dishes when the cuisine is unfamiliar, having all the menus is useless |
| Nunes | - Phase one: Goes thru phase one of task two expect selects a cuisine shent familiar with - Phase two: searches thru most popular dishes and reads description for ingredients she likes and that she can visualize – She prefers the convenience of a picture of the dish, instead of googling for it |
| Jie | - She swiped in the home page and found an unfamiliar type of cuisine. - She was confused with some unfamiliar name of dishes, she could not know what they were when there was little picture or ingredient information. - She chose one unfamiliar food because of the descriptions and picture. She thought these information were more reliable than pure name. She also paid attention to the reviews. - She didn’t use google to find each unfamiliar dishes, because it was time-consuming. |
| Kayla | - Unaware of cuisine options in app, thought of Taiwanese, and searched through the search bar - Did not know or could find description of dish, and had to use Google or another outside source to get visual representation - Chose food purely off of novelty, reviews or description had no influence |
Task #4: Figure out what the dish looks like within and outside the app.
| Name | Results |
|---|---|
| Boulanger | - Users don’t know how to do - No easy way to find out, no way to find out into the app - Would use search engines like Google |
| Nunes | - Has no problem with using Safari to search for dish outside of app, but prefer if she could open a web browser within the app or be able to copy text within app - Wishes there were pictures of the dish added to the app on the page of the dish |
| Jie | - She had negative attitudes toward this task. She could not copy the name of unfamiliar food. She thought it was time-consuming and complex to use a third-party application to figure out what the unfamiliar food looked like. |
| Kayla | - Confused since there was no way to get description inside of the application - Did not feel confident enough to place order since there was not enough detail when trying to pick something she had never ordered before - Suggested that there should be a link to their menu for descriptions and pictures of food inside the application |
Task #5: Pick three different dishes you are familiar with (each in different cuisines)
| Name | Results |
|---|---|
| Boulanger | N/A |
| Nunes | - She picks the dishes that taste good, are crowd pleasers, and taste the same across different restaurants |
| Jie | - Behaved proficiently in this task and added three familiar food in the shopping cart successfully. - This task had many processes, and she presented a little bit impatience at last few steps. |
| Kayla | - Easy task, simply searched for food items that she would not have to go out of her comfort zone to order - Found restaurants she was familiar with and trusted, rather than depend on reviews and descriptions |
Task #6: Choose one you would buy
| Name | Results |
|---|---|
| Boulanger | N/A |
| Nunes | - Chooses Pizza bc crowd pleaser, easy to warm up, quick delivery |
| Jie | － She compared three dishes in the shopping cart and ultimately chose the nearest one, because she mentioned that this task happened at night and she was hungry. She wanted to receive the food as soon as possible. |
| Kayla | － Explained it was the safest, and cheapest option |
5.3. Errors
Finding out a more precise description of a particular meal:
- Long-press the name of the meal (nothing happens)
- Click on the name of the meal (redirects to the checkout)
5.4. Participants’ Feedback, Likes, Dislikes, & Recommendations
Here are summarized the participants’ likes, dislikes, and recommendations.
Likes:
- User can order a future delivery at specific time.
- Some dishes provide ingredient information and pictures.
Dislikes:
- The UI of the menus
- The review function is not obvious
- Ingredient information and pictures are not unified, some dishes have but some don’t provide.
- User would feel confusing about unfamiliar food when there is no description or picture
- User can’t copy the name of food directly from the application.
- User may rely on a third-party application to figure out what the unfamiliar food contains and looks like.
- Not many options for restaurants locally
- Delivery fee and minimum order charge narrows down food choices
Recommendations:
- Add more precise descriptions, with pictures of the meals
- Use tabs instead of a listing for the menus
- Reviews become more obvious and provide updating real pictures.
- Add way to search type of food within or outside the application (Database, Search Engine)
5.5. Summary
| Tasks | Results | Errors/Problems | Comments |
|---|---|---|---|
| #1 | All four participants finished this task. They quickly learn basic operations on GrubHub. | One issue with the no presence of local restaurants | |
| #2 | Two main ways to do a search: one is to use the cuisine tabs (and filters), the other is to directly search for a dish using the search bar. The final selection is made based on the restaurant’s name, rating, and overall aspect of the menu. | The process can take a while, depending on the selection criteria the user has: find the good balance between price and ratings. | |
| #3 | Usually, participants were confused by the unfamiliar names of the dishes. Again there were two situations: either they had to do a Google search or they selected a dish with an actual picture/description available. In some cases, having the reviews help also to take a decision. | Having an illustrated description aside the name of a dish is gamebreaker: usually participants tend to choose this one. | |
| #4 | Participants would use a third party application (eg. Google) to search so pictures of the dish. | Could not copy/paste the name of a dish, which implies more difficulties to do a search. | Here again, the fact of not having detailed information about a dish push the users not to buy unfamiliar food. |
| #5 | Users selected without any issues three dishes they were familiar with. | This task was longer than the other ones. | |
| #6 | Criteria varies from a participant to another (delivery time, price) |
6. Recommendations
Based on what we saw, the results of the user test, and the feedback of the participants, our team recommends the following:
| Location/Feature | Object/Screenshot | Observation/Data | Severity | Recommendation |
|---|---|---|---|---|
| (Image Below) | Items | Users struggle to find description on the different dishes | High | Add more detailed descriptions like pictures of the dishes |
| (Image Below) | Reviews | No pictures are available to “introduce” unfamiliar dishes | High | Emphasize the reviews with pictures. Allow users to link a review to a specific dish. |
| (Image Below) | List | Too much items | Medium | Use table instead |
7. Conclusion
GrubHub provides most basic functions as a food-delivery application and it is not too difficult for users to find the information they need either within in the application or outside via sources such as Google, Yelp, or the restaurant’s webpage. However, GrubHub is not outstanding, because in most of our cases it did not leave a strong positive impression on our users being tested which means other users can and will choose other applications to substitute it.
Some functions still need improvements, like user confusion over names of unfamiliar dishes, too many options before adding one dish into the shopping cart or overall user information overload, reviews are not easy in all cases to find and read, visual and textual information in the restaurants menus are not always provided where it would be beneficial to the user.