State machines and emergent behavior

Another candidate video idea on the backlog is the way I've been approaching game AI design, and more broadly, gameplay design. A very natural approach in action gameplay design is to separately design enemy characters before assembling them into encounters to take advantage of emergent synergies between them.

For example, enemies that usually require careful timed inputs and movement have to be approached very differently when placed next to enemies that demand immediate attention, enemies that can be stunlocked in 1-on-1 combat might be more dangerous in large groups, ranged enemies can pose a constant nuisance that needs to be addressed ahead of other enemies, etc.

The neat thing is that these synergies naturally appear whether or not they're explicitly designed, and it allows for the encounter design discipline to be more of an exploratory process of discovery that can then subsequently be refined and polished as needed. It also nicely breaks up the monolithic problem of gameplay design into more approachable sub-problems that can, at least initially, be worked on relatively independently.

Taking this idea one level deeper, enemy AI design itself can be broken up into the various systems that govern the final player-visible behavior. For example, an awareness system, an animation/movement system, and a high level decision-making system can all be conceptualized as separate state machines or systems. Not only does this decoupling make implementations more modular and reusable, it also takes advantage of the combinatorial scaling when these systems interact with each other.

Interesting things start happening when we build each of theses systems to solve their own goals, then let them communicate between each other and play out in parallel. For state machines, this might look like using the state of one system as part of the conditions for a state transition of another system.

Sometimes, these interactions are explicitly designed and implemented, but given enough freedom to act independently, sometimes the interaction between systems produce desirable emergent behaviors for free that appear during testing.

As a case study, I'd like to talk about some of the behavior design of the spider turret enemy in my game: when the player sneaks up behind the spider turret, it will recoil backward in surprise. This is not an explicitly designed behavior, but one that emerges naturally through the interaction between a few systems. Individual systems have rules that govern their evolution:

• The awareness system can detect nearby sounds, see objects within the turret's cone of vision, and extrapolate the movement of objects that leave or are occluded from the cone of vision. Between these different inputs, it keeps track of the most interesting location.
• The movement system rotates the turret head to look toward the direction of the most interesting location reported by the awareness system. Separately, it computes body and foot placement to move toward a target position and orientation.
• When the enemy is visually tracking the player, its behavior state machine can search for a safe position for the movement system to move to, to start attacking the player
• When the enemy is not visually tracking the player, its behavior state machine can set a movement target for the movement system to move toward the most interesting location reported by the awareness system

When the player sneaks up behind an enemy, the interactions between these rules play out in quick succession:

• The awareness system is alerted to the player movement sound which becomes the most interesting location, and the movement system rotates the turret head to look at it
• When the head sees the player, the behavior state machine detects the distance to the player is too short, and transitions to the distance management state
• In the distance management state, the behavior state machine sets the movement system's target position to maintain a safe distance between itself and the player
• The movement system reacts to the new target position, but because of its second order dynamic model, takes some time to initially overcome its inertia

The end effect is that the enemy is alarmed by the sound of the approaching player, turns its head to look, and scurries away to a safe distance.

Under a different set of circumstances, this same set of rules can produce completely different, but also meaningful behaviors:

• If the player shoots an arrow while hiding behind cover, the awareness system is alerted to sound of the arrow's hit location
• The movement system rotates the turret head to look in the direction of the arrow hit location reported by the awareness system
• If the player remains hidden from the cone of vision, the behavior state machine transitions to the investigation state, and sets the movement system to move toward the arrow's hit location

The end effect is that the enemy is distracted by the sound of the arrow, and turns to investigate it. This is especially compelling if the enemy was previously chasing the player's extrapolated position, such as when the player runs behind cover.

In both of these cases, no explicit sequence of events were designed a-priori: there is no custom animation sequence or state corresponding to being surprised or being distracted. Instead, these are behaviors that emerge from the interaction between the different systems that as a whole define the enemy AI behavior.

What's really interesting about this approach to behavior design is that each new rule can potentially interact with everything that came before, and through testing, the emergent behaviors can then be refined by limiting any messy interactions. These sorts of systems can theoretically scale to all sorts of different scenarios:

• Can we produce the effect of intelligent teamwork just by allowing the awareness system of enemies to have some limited communication with each other?
• Can we produce interesting gameplay options by allowing the player to disrupt the function of certain systems, such as crippling the movement system, or hacking/jamming the awareness system?
• Likewise, can we design "support class" enemies whose primary purpose is to buff the function of certain systems, such as a sentry that boosts the awareness system of nearby units in some way?
• How much variability in expression can be achieved just by tweaking the parameters of individual systems, and connecting them in different ways?

The examples I've given so far treat the different systems in an enemy AI as being connected in a graph-like structure, but there are also cases where a tree-like structure naturally suits the problem. For example, a movement system for the body can drive a foot placement procedural animation system for a legged creature, with a clear parent/child relationship. Any systems that only exhibits 1-way coupling like this is also a prime opportunity for optimization: child systems can often be disabled or simplified when their effects are not visible to the player, such as when an animation system is offscreen. A lot of these ideas are strongly related to the GDC talk about animation in Rain World.

Anyways, this seems like it would be a pretty good, approachable subject for a video. There are probably a few different angles that the subject can be approached from. I could contrast this method against other methods like behavior trees, I could look for analogies in other aspects of gameplay such as systemic gameplay design, I could try to find good visual representations to get across various ideas, What do you think? Maybe it's a little off brand. More GMTK than t3ssel8r. haha.