How can you improve your game's tutorial experience with user behavior data?

Games are an interesting medium compared to other software because of the agency they provide the players with. So it becomes important to quantify everything the player impacts. Heatmaps are particularly useful for understanding where the players are struggling. Positions, orientation, and input frequency are some that I find particularly interesting.

Most players quit very early on in games. For the best insights into why, metricize everything. You want to know how long they played before quitting, but also what their settings were, what the last step in your tutorial they completed was, how many times they opened a menu, etc. you can even use position tracking data to generate a heat map of where players went in your game world. What you’re looking for are patterns, sequences of events that lead to retention, and sequences that lead players to quit. Think about things where players might be going off the rails of your experience, then add metrics there to see if you are right.

User behavior data is a valuable tool in comprehending how players establish and participate in virtual communities. By examining collaboration patterns, communication methods, and social structures within the game, developers can create a more community-driven and socially engaging gaming experience. This not only enhances our understanding of user behavior but also contributes to the development of a vibrant and interconnected gaming ecosystem that mirrors real-world social dynamics.

Utilizing user behavior data to enhance your game's tutorial experience is key to optimizing engagement and retention. By analyzing how players interact with your tutorial, you can identify pain points, adjust difficulty levels, and tailor instructions to match different learning styles. This data-driven approach fosters a more intuitive and enjoyable onboarding process, ultimately leading to higher player satisfaction and longer-term success for your game

By tracking player actions, such as completion rates, time spent on each tutorial section, and areas of confusion or frustration, you can identify where players struggle or lose interest. This data allows you to pinpoint specific areas of the tutorial that need improvement or clarification. For example, if many players repeatedly fail to understand a particular game mechanic, you can adjust the tutorial to provide clearer instructions or additional examples. Additionally, you can personalize the tutorial experience based on individual player preferences and learning styles. For instance, if a player demonstrates proficiency in certain areas, you can dynamically skip or streamline those sections to maintain engagement.

Categorically there are four main types of data I think about. Quantitative metrics are where you fire off events from actions in your game and save them to a server someplace. Survey data is where you create a form asking a bunch of questions and send them to either your players, or a list of potential players provided by a third party. Qualitative user study data is mainly from running a controlled test in a user research lab or online with a camera, running a set of users through a specific part of your game and asking specific questions with follow-ups. Anecdotal information is user feedback from social media, bug reports, Q&A sessions etc. Don’t act on this alone, but use it to help inform where to point other data gathering efforts.

By merging traditional analytics with these advanced tools and psychological concepts, a more comprehensive profile of player behavior can be created, resulting in a holistic approach to game design that takes into account both conscious and subconscious player reactions.

Once you have data to work with, try and build a user segmentation model. Do this by looking at the pattens in your data either manually in simple cases, or going so far as using a deep learning clustering algorithm with complex situations with many data points. You can create personas for different segments if it helps, but focus on the outcomes that perform the best against your KPIs and then look at where the data diverges for users who aren’t getting to those outcomes. If you can see that more of the players who do A, B, and C quit than players who do A, B, and D, you can figure out how to encourage players to do D instead of C and be confident that this will improve their experience.

To gain a deeper understanding of user behavior, you can consider conducting ethnographic studies or player interviews. This will provide qualitative data that reveals cultural nuances, player motivations, and community dynamics. By adopting a sociocultural perspective, you can customize your tutorial to cater to a diverse range of players, ensuring inclusivity and accessibility. Additionally, by incorporating elements of choice architecture inspired by behavioral economics, you can optimize the tutorial experience. Understanding how players make decisions and respond to different influences will allow you to create a tutorial that not only imparts necessary information but also aligns with their cognitive biases.

I suggest that you don't think of your "tutorial" as one monolith or a single flow that is just at the beginning of the game or for new players. A modular tutorial system that is flexible and robust can track data about feature usage over time and account for the facts that players: 1.Don't need to know everything up front 2.Will usually require multiple teaching moments spaced over time 3.Will forget things Think about how players can demonstrate understanding of a feature before you trigger a tutorial to avoid unnecessary prompts, how frequently they should use a feature to know that they didn't forget about it after being shown once, and retriggering tutorials if a player comes back to a game after a long period of time.

Since fluctuations happen over time, and as you iterate you’re going to get down to very small changes with very small results at some point, it is ideal to have an A/B testing system. This lets you run two versions of your game at once for separate cohorts of players, so that you can clearly see the differences between them. This is better than before and after changes because numbers might go up and down for other reasons that you can’t account for, such as the release of another new game. Don’t get into small minor adjustments too quickly though. To make progress the fastest you need to explore vastly different possibilities. Remember: small change small result, big change big result. Slowly narrow down to smaller changes over time.

To enhance your tutorial testing and iteration strategy, you can integrate principles from finance and business. By using a cost-benefit analysis, you can prioritize iterations based on their potential impact on player engagement and retention. Assigning a monetary value to each improvement opportunity helps you allocate resources efficiently, focusing on changes that offer the highest return on investment. Additionally, you can optimize the experimentation process by leveraging A/B testing methodologies inspired by financial risk management. By randomly assigning players to different tutorial versions and comparing their performance metrics, you can objectively evaluate the effectiveness of each iteration while minimizing bias.

Dive into resources on interactive narrative design and transmedia storytelling to see how user behavior data can improve player engagement in evolving narratives. Books like "Hamlet on the Holodeck" by Janet H. Murray provide insights into the connection between storytelling, interactivity, and technology. Additionally, looking into gamification and behavioral economics can offer a fresh perspective on influencing user behavior. Titles such as "Hooked: How to Build Habit-Forming Products" by Nir Eyal and Ryan Hoover and "Predictably Irrational" by Dan Ariely explain the psychology behind user engagement and decision-making, providing valuable insights for game design.