Double Loop Learning in Game Development
I recall a conversation at Pocket Gamer 2019 with a fellow game developer. He had worked in a large studio of 150 people, and his team had the mandate to come up with a shooter game for mobile phones. He had a team of twelve developers, and they had three months to build the first version of the game, after which it would be subjugated to a company-wide playtest.
This games company had gone through many iterations of figuring out how games should be developed so that the management team would give full control to the teams and the staff. The team would be given three months of autonomous, peaceful development time, after which they’d need to show their progress to the whole staff. Based on the playtest results, the team would propose to the management on continuing or killing the project.
The fellow game developer explained that on paper, the process seemed great. But after eighteen months and five projects were killed, the team decided on its own to disassembled and merge with other game teams. There was fatigue on innovation, fatigue on starting new things that didn’t strike lighting in the bottle. The team couldn’t figure out a great shooter game.
I’m all about learning from your projects and iterating the process, the mindset, and what holds true to the business you are in when you are making games. So I asked the fellow game developer: “How did you secure the fact that you’d learn as much as possible from each prototype your developed and playtested?”
- We built two prototypes early on and killed the one we inside the team didn’t like. This decision was made on playtesting and hearing everyone on the team.
- No one in management ever ruled over what we had decided, which gave the team lots of autonomy. There was a strong sense that we could figure the shooter game out as long as we played lots of shooters, discussed what was great about them, and applied those insights into prototypes.
- We built five different shooter prototypes that reached a company-wide playtest. After each test, a survey was sent out to the company. The vital question was: “How likely you’d be to continue playing the game after the playtest?” None of the games got an average grade of over six, which was considered quite bad.
- We tried portrait and landscape mode; we tried “virtual stick” control and simple command decisions for your character’s movement. We did an FPS shooter and an “Archero” style shooter. Many more interesting, proven concepts were tried.
- Eventually, we never found the core gameplay that people in the company would have loved. We documented each prototype and all the feedback we got from the staff into our company wiki.
Then I asked, what did you learn about shooters.
- That it’s hard to make a fun shooter game for mobile phones.
- He said that the whole team was fed up with shooters after a year and a half to make it work. They just wanted to move on.
This encounter with the fellow game developer was a year ago, before the COVID epidemic. I’ve had time to contemplate the discussion. I’ve reflected on my understanding of what had happened and what could have gone differently. As I’ve spent more time on the concepts of learning from experience, I believe that this example case, like many other game development experiences, stems from an ineffective way of learning.
I believe that many game developers are approaching their work with a single-loop learning process. I think that success comes from understanding how to reflect better and learn from your decisions and actions, which is called double-loop learning.
The easiest way to describe what single-loop learning is to give an everyday example.
You decide to go outside, and you tie your shoelaces. The tying of shoelaces is something we’ve done since we were like five years old. At first, we learned to tie a knot. Then eventually, we learned to tie the bunny-ears. Finally, we start getting better at it, were we could tie our shoelaces blindfolded and as fast as possible.
But that’s about it. In a few years, the way we tie our shoelaces becomes a habit that doesn’t change. We continuously do it for decades, but the experience never evolves. Tying shoelaces is the simplest example of Single Loop Learning.
Another example is reading books.
Goodreads is terrible because it rewards the completion of books. The Annual Read Challenge is about how many books you’ve completed, not about reflection, re-reading great books, or applying what you read to your life.
A quick fix to make Goodreads more rewarding:
1) Reflection, which is about taking notes from the material.
2) Imitation is just to read and not do anything with it immediately.
3) Experience, which is hard, it’s about taking the knowledge quickly to use.
In life, work, and game development, we end up with single-loop learning.
At the start of this article, I talked about my fellow game developer and his experiences from figuring out shooters for mobile phones.
His approach was to follow the single-loop learning methodology.
1) His team did the imitation (played other shooters), reflection (discussed what they’d observed in the shooters), and experience (building prototypes), but their produce was the end of the loop.
2) Once the prototype was exposed to the company-wide playtest, the results could have been a coin toss, or instead, it seems, a 20-side die-cast, where you’d only hit success with lucky 7. Why was this?
- The team went the route of minimal effort.
- The team had the autonomy to develop a dream game, but they had outsourced the analysis and the final decision making to the company staff.
- The barrier that needed to be passed with the company-wide playtest. And the team never crossed that barrier.
Now, let’s focus on double-loop learning, which questions everything that is thought conventional and opens up new possibilities for game developers.
The process of double-loop learning is that instead of making the experiences a habit, you put more effort into reconsidering the habit.
In the fellow game developer’s example, one place to start would be to question the game company staff’s usefulness as a decision-maker.
I recently conducted a test at a local Helsinki-based games company, which had instated the company-wide playtest process. There I asked the developers to run similar tests with successful free-to-play games and then ask the same scaled questions. The majority of the experiments yielded results where successful games wouldn’t get past this barrier and would get killed.
Why is this an important finding? If the fellow game developer would have broken out of his single loop of learning and questioned their systems (which had been instituted in good faith,) the democratic approach for killing a project would have become under question.
The double-loop learning approach is about breaking out of paradigms, questioning where conventions come from, and pivoting or making reassessments if necessary.
How would you apply double-loop learning to a gaming startup?
Let’s imagine a small gaming startup, with one development team, working on a prototype. The team has decided that they’ll only kill games if they don’t show good results in actual playtests with real players from the App Store. They manage to get one game out, but the metrics aren’t looking good.
It’s often hard to pinpoint an exact cause for a game not being enjoyed by players in gaming. But this team won’t relent. They break down the entire game, to understand all the attributes that make up for this particular part of the game.
As Rahul Vohra, former game designer at Jagex, explained, games are made up of features: goals, emotions, controls, toys, and flow.
Here, the team starts questioning their process of making decisions, based on each feature. Examining each feature individually, then examining each feature concerning each other. They believe that the way to make better decisions is to ask why a decision was made in the first place, and why it turned out to be a mistake.
- Why did we decide on this goal for the game? What was the expected outcome if this goal was in place?
- What was our assumption on how the goal relates to flow?
- With the decided controls, what kind of emotions were expected?
Such examination is just a simple example, and a team with a more massive game might end up with dozens or hundreds of questions. The point here is to identify blind spots, reflect on those findings by discussing and finding unusual remedies to move forward.
If you start asking questions earlier in the process, you will uncover more knowledge and insights as you move forward.
Ways to improve double-loop learning
Here are the methods that I’ve become familiar with, which can improve our abilities as game developers to make learning a daily practice.
- Ready yourself for people will become defensive. It’s never easy, especially for seasoned game developers, if their conventions and ways of working become under threat.
- Some handy questions to get going: What is the current theory in use? How does it differ from proposed strategies and goals? What unspoken rules are being followed, and are they detrimental? What could change and how? Forget the details; what’s the bigger picture?
- Decide on reflection intervals. When working on a game, I’ve seen the most impact from reflecting and learning from the experience, when teams set regular reflection meetings at fixed intervals. Once a week is a great cadence. Either at the beginning or the end of the week, pref. in the morning.
- Run pre-mortems, where you deliberately come up with reasons on how the project will fail.
- Create a Decision Journal on your assumption. When you make decisions or assumptions, especially before you start developing them into a prototype, ask yourself: 1) What were the matter and the decision? 2) Why was the decision made in the first place? 3) Why did you decide as you did? 4) What is the expected outcome? When you reach the next reflection meeting, pull out the Decision Journal, and see if any decisions have impacted. Reflect on them, especially if the expected outcome wasn’t as expected.
- Rapid game development. Prepare to have your prototype ready for playtesting in one week. Then, if you give the green light to the prototype, give yourself three more weeks to launch the prototype on Google Play.