Intrinsic vs. Extrinsic rewards in Klei’s latest game: Don’t Starve

Intrinsic vs. Extrinsic rewards in Klei’s latest game: Don’t Starve

Greetings! Klei Entertainment is working on a game that has been recommended to me by just about everyone who has played it, and when Jamie Cheng offered to write up a post about player rewards and the thought behind them, I thought it was a great idea. Also, I’m traveling and needed the content. I learned some interesting things reading this this post, and I’m happy to share it with you. Enjoy!

Back in 2010, Chris Hecker presented a talk about Intrinsic vs Extrinsic rewards, titled: Achievements Considered Harmful? You could say that this article is about our own, direct experience in the matter. It’s about how we nearly ruined our newest game by adding one of the most common game tropes: the quest system.

In January, we started working on a new game called Don’t Starve. This game is a big departure for us at Klei, considering our recent history with linear, narrative-driven action games. Don’t Starve is a randomly-generated, open-world survival game - the player tries to survive for as long as they can in a complex, hostile environment. It’s a “lost in the woods” simulator featuring cannibalistic pig people and trees that fight back when you try to chop them down. It’s about being alone, and surviving on your wits while growing a magnificent beard.

We were inspired by building games like Minecraft and Terraria, as well as simulation games like Dwarf Fortress. Add in a splash of old-school adventure gaming and a hint of roguelike, and you have an idea of what we were going for. Beyond a playground that Minecraft offers (oh, but WHAT a playground), we wanted to build a rich world that interacts in surprising ways, and challenges the player to explore and understand it.

With that, I’m going to start with some background, and then get into the surprisingly pointed consequences of using external incentives in our game.

Also, because we’re Canadian, we’re going to stick the letter “u” in words all over the place.

Behaviourism in games

At their most basic level, many video games can be reduced to simple behaviourism. The player sees a scenario on the screen, and tries to affect it via some form of input. The game reacts in either a positive or negative way, and that reinforces or inhibits certain behaviours on the part of the player.

You press a button to collect a coin. A bell rings, and a number gets bigger. All of a sudden, you have a powerful urge to press more buttons, and so it turns.

Meanwhile, a well designed game can elicit a much more complex emotional response from the player than simple reinforcement learning. Do you remember the sense of wonder that you felt when you first heard about minus world in Super Mario Brothers? Or the growing sense of unease that took hold about half way through Shadow of the Colossus? How did you spend your first night in Minecraft? Games can be powerful stuff.

So while behaviourism looms large in the physiological landscape of gameplaying, it is possible for designers to reach for something more. The psychological techniques that a game designer chooses to use in a particular game can have far-reaching effects not only upon the way that the game is played and why players choose to play it.

The design problem with behaviourism

In the large scheme of things, behaviourism in games is a useful tool. The real problem comes when the tool of behaviourism—the extrinsic reward—is either the only reason we’re doing something, or perhaps more tragically, replaces the reason we’re doing something.

Studies about the damage of rewards are well-known, and yet it seems common to hear a design (or business) talk about how effective arbitrary rewards are in motivating players. Take an old, 1987 study of 2 girls drawing for the fun of it:

“Young children who are rewarded for drawing are less likely to draw on their own that are children who draw just for the fun of it.”

In this study, the girls who were rewarded actually ended up drawing faster, but worse, than their counterparts who simply enjoyed drawing. Then, once the rewards stopped, so did they. In this instance, we’ve managed to replace the joy of drawing with the expectation for reward. This phenomenon is reproducible and has been repeated by multiple studies; the examples of the negative effects of rewards are abundant, and its clear that rewarding people, even for things you want them to do more of, can have the exact opposite effect.

Let’s put aside the moral implications of behaviourism for now, and make the case for why this approach is problematic for our game design goal: actually getting people to care about our game deeply. Specifically:

1. Rewards can cause players to do less of what you want
Players given rewards will stop doing the thing you’re rewarding them for once you stop the rewards, even if that action was fun in the first place.

2. Rewards can cause players to care less about everything else
Players looking for a reward become so focused on the reward, that any distraction is exactly that—a distraction.

3. Rewards can cause players to perform worse
Players who are trying to optimize for the reward will often stop taking risks, in fear of losing the reward, even if this risk taking behaviour could actually improve their performance.

All of these behaviours are backed by studies. To see many more examples, I direct you to further reading.

What we unintentionally found is, Don’t Starve became a poster child for what can go wrong. As we learned recently in our studio, you have to be careful when you’re making these decisions. If you choose poorly, you might end up subverting the whole point of your game in the first place.

When we as designers give an arbitrary reward, what we are saying is “what you are doing is not intrinsically interesting, so we’re giving you this carrot instead.”

Why we implemented quests

Our initial playable prototype of Don’t Starve took a couple of months to build. It opened with Wilson, the main character, waking up inside a clearing. He would play a little getting up animation, and then the user interface would appear and the player was given control. That’s it. We wanted to see if players would grok the system we created for them, and start exploring.

Instead, players would watch the introduction (such as it was), and then turn to us and ask “What am I supposed to do next?” and “I don’t know what to do.” And then they’d simply be stuck.

Ouch. All of our work was going to waste, because people didn’t know what to do! With a bit of verbal coaching, they were able to start playing, but as a guided experience the game lacked the sense of discovery and accomplishment that we were trying to impart.

But all was not lost. Once we had taught players enough about the game’s basic mechanics, they would start experimenting on their own and would start to have a lot of fun. The problem was getting them to that point without coaching or prompting.

A couple of brainstorming sessions later, we decided that best solution was to provide more structure to the experience. The players didn’t know what they could do in the game, so we were going to tell them! If you’ve played any triple-A game on the current generation of consoles, you can probably guess where this was going: Quests!

Now, when players woke up, they were confronted by an evil-looking man in a pinstripe suit who challenged them to complete a laundry list of goals, like “survive for 5 days” or “find all of the phonograph pieces”. Completing the tasks would knock them off the list, and they would be replaced by more complex tasks. We figured that we could teach all of the core systems in the game this way, from exploration, to crafting, to combat. By the time they were done all of the challenges, players would be ready to take on the world on their own terms!

The unexpected result

The actual result was exactly what I described previously. First, players would optimize for the goal—in this case, it was surviving for four nights. Except, they now did this to the exclusion of all else! Players simply cowered in a corner, hoarding some food, and waited for the time to run out. This was:
a) very boring,
b) actually ineffective and
c) once they reached the goal, if we didn’t write another goal, they would stop playing.

Our solution had taken a confusing - yet fun - game and made it feel like a list of chores. There was also the problem that players would eventually run out of quests, and then stop playing entirely because the game was ‘over’. It was an incredibly eye-opening experience of how just a simple UI change (the objectives didn’t actually give you anything) could so effectively destroy player motivation.

Embrace the intrinsic fun

What we had initially thought was a failure of instruction was actually a failure in how we presented the game’s goals. In structuring the game as a series of explicit tasks to be completed, we taught the player to depend upon those tasks to create meaning in the game. At the highest level, the game that they were playing was “complete the task list”, and the actual game play became an obstacle to be overcome.

This was a clear example of not us as designers not understanding the underlying motivations for players, and allowing the rewards to overshadow the game itself. It was poisonous to Don’t Starve because as a systems-exploration game, its charm lies in letting players figure things out for themselves. Our itemized lists and rewards made the game seems as about as charming as a couple pages of math homework.

Once we realized what the underlying issue was, we doubled-down on focusing on the one thing that mattered: allowing players to have fun exploring and experimenting.

First we removed all of the explicit quests. This brought us back to where we started. Our goal as developers then became to find a way to teach players about the early game in such a way as to allow them to figure it out for themselves.

Designing this way is a lot harder. We could no longer simply tell people what to do, but instead, after dozens of playtests and many UI passes, created an interface which gently and neutrally showed them what they could do created an environment where players could enjoy the game exactly as they felt was correct.

In Don’t Starve, the player’s progression in the world closely follows their progress through the crafting menu. You collect stuff so that you can build things that let you collect more exotic stuff that leads to even better things. We don’t explicitly tell you that you should collect the ingredients build an axe so that you can chop down trees so that you can build a fire so that you can survive the night. We just present you with a prominent, navigable list of crafting recipes, and we kill you if you get caught outside for too long without a light source. Oh, and we also delete your saved game, because we’re kinda mean.

After a player experiences this loss first-hand a couple of times, they catch on. By the time they’ve figured out how to survive the first in-game night, they have a pretty good mental model of how the game works, and are busy coming up with their own goals for the next day. They have learned how to learn about the game, and hopefully their curiosity about its underlying systems has been piqued enough that they want to continue playing.

Ongoing development

Our game is currently in Early Beta, and it has been incredibly rewarding watching waves of players go through this learning process and teach themselves how to play. Since the game is not yet complete, players tend to hit the content wall around day 50 or so, where they have seen most of what is currently in the game. 

It was thus really amazing to see the power of internal motivation, finding groups of self-motivated players creating their own ‘challenge runs’ on our forums once they got to that point. They were coming up with new sets of self-imposed goals that changed the nature of the game, such as trying to play without eating any meat, or by starting fights with every creature that they find. They weren’t doing this to get achievements or even to get a high score.

They were doing it because they were having fun.

Don’t Starve is currently in Early-Access Beta. You can try a demo via the Chrome Webstore at the official site. Don’t Starve will be released on Steam in the next few months.