Rollin Poe takes takes to the pitch with FIFA Mobile
The Scope Test A System and Produce Suggestions for a Corporate Partner
Myself and three team members worked with EA Canada as an industry partner to conduct an evaluation of their FIFA Mobile product. The needs and scope were set by EA and centered around the onboarding process and new-user conversion, specifically focussing on the affective user journey and transfer of knowledge. A three stage evaluation method was put in place with a target demographic. A comprehensive affective analysis was generated and design changes were proposed.
Time: June, 2020
Status: Complete
This evaluation was prepared and accomplished solely by the authors in their personal capacity as fulfillment of the requirements of Simon Fraser University’s IAT 432 – Design Evaluation course. The opinions expressed, conclusions reached, and suggestions made are the authors own and do not reflect the views of Electronic Arts (EA), Fédération Internationale de Football Association (FIFA), or any partners associated with the game, FIFA Mobile.
A Couple Key Aspects
- Working with a corporate sponsor on a deployed product
- Understanding client needs and implementing an appropriate response
- Design and execute a research plan and provide insightful analysis
- Targeted demographic testing for affect, usability, and learnability
The Starting Playspace
Client Needs
The inital meeting with EA was an opprotunity to gain insight into the current areas that would benefit from evaluation. The team also worked with our client to identify key target demographics and potential areas of focus for user testing and analysis.
In conjuction with EA we settled on evaluating the onboarding for FIFA Mobile, specifically for affect in the user journey and transfer of knowledge from the tutorial to the gamespace and store focussing on players who have engaged with the FIFA console product but have not transferred to FIFA Mobile.
Real World Researching and Testing
Methodology
We took a three-staged, mixed methods approach to gain a wholistic understanding of the onboarding process for new users. All three stages were completed in a single session and were done in sequential order.
Pre-Play Questionnaire
An online pre-play questionnaire was used to gauge users’ gaming backgrounds and their initial impressions of gameplay elements, such as menu layout and in-game currencies. Qualitative and quantitative data was collected using Likert scales, ranked choice, and open-ended questions.
Think Aloud Play Session
Facilitators prompted participants to talk about their actions, thoughts and feelings during onboarding using a think-aloud technique. Each participant’s screen and audio was recorded. The goal was to extract an understanding of usability and affect by recording qualitative comments and coding them.
Post-Play Questionnaire
Upon completion of the play session, an online post-play questionnaire was used to evaluate the transfer of knowledge from onboarding tasks, and extract a formative conceptual model of in-game elements. Qualitative and quantitative data was collected using Likert scales, ranked choice, and open-ended questions that was then compared to responses in the pre-play questionnaire.
Data Analysis and Synthesis
Transcribing and Coding
Each participant had audio and video recorded during their think-aloud gameplay session. Transcripts were produced for every participant. Led by myself, the team coded each transcript for affective comments. For example, the comment below could be coded for affect (frustration) and usability (lack of control).
"I didn't think the tutorial was fun, but I guess that's not the main objective of tutorials. If I saw a skip or exit button I probably would've clicked it and looked for the online modes."
User Journey
EA specifically mentioned wanting to have a better understanding of the user's affective journey through onboarding. Based off the data gathered from the think aloud sessions and in conjunction with responses from the questionnaires, I proposed constructing an average journey map for new player onboarding, noting key high points and pain points. (Link to full journey map below.)
Quantitative Analysis and Visualization
From the questionnaire responses we received a wealth of numeric data from likert scales, ranked-choice, and open questions. I then led the charge into data analysis and visualization both for our sake as researchers to understand the data, as well as for creating distilled artifacts for our client. A small handful of the data visualizations I created can be seen above.
Some of the many Proposed Design Changes
Onboarding Length Condense Tutorial Stages
A key player frustration was that they felt the tutorial took too long. By combining the shooting and passing stages and the dribble and defending stages the tutorial would feel less start and stop and would allow for more complex training scenarios such as a brief 3x3 match. This would help align with the players initial ideas of what playing a FIFA game would be like and their goal of getting into a "proper game".
Lack of Control Highlight and Provide Areas of Exploration
One of the single biggest frustrations was that players felt they lacked autonomy and were locked into the tutorial with no options. To alleviate this, at an early opportunity, Eden Hazard could highlight areas available for free exploration allowing players to pursue the gameplay that most interests them.
Error Prevention Key Learning Moments
The lack of autonomy and long length led to players mashing through text boxes, not bothering to read them. Even in early stages participants preferred not to read but to learn via gameplay. Inclusion of finger gestures allows players to get right into the action. Additionally, allowing players to reread the tutorial textboxes as they need avoids the pain point of needing to fail the mission and respawn in order to learn what they need to do.
Confusion about the Currency Systems
There are numerous types of in-game currency and players were often confused about what each one did, how valuable they were, and where they could learn more about they currencies on offer. Seeing that micro-transactions are a core part of FIFA Mobile's business strategy, this is problematic. We propose information headers on the store pages explaining the use of each currency, as well as increasing discoverability of information affordances on rewards earned through gameplay.