Reflecting on Planning in Games | 08/05/24

The AI and Games Weekly newsletter is hosted on our dedicated Substack, as well as right here on LinkedIn. Visit our website to subscribe and have it appear in your inbox every week

Hello all, and welcome to this weeks newsletter update. I’m feeling a little nostalgic this time around, and while I’m working on several stories, a background task has been to re-upload ‘classic’ AI and Games episodes to our new home on Substack. As I was putting this together, I noticed a recurring theme of ‘AI planning’, and given we just had the F.E.A.R. retrospective ship last week, I figured let’s talk about that some more, and why planning has been so popular in games!

But first, some announcements, and a word from our sponsor…

Automate Game Tests with AI Agents

Special thanks to this weeks sponsor of our newsletter: Regression Games

Automate Unity smoke tests with AI agents. Get started quickly with our agent SDK which features no-code test building UIs, playback recording tools, and LLM-driven behavior trees.

Try it out by visiting: regression.gg

Weekly Announcements

Alrighty, let’s get some announcements out of the way for current and future events and projects!

• We have a new updated format for subscribing to the AI and Games Newsletter on Substack. The newsletter is now treated as a separate ‘section’ on Substack. As such, you can now subscribe to the weekly newsletter separate from the rest of the updates on AI and Games. Don't worry, if you just like to read these updates on LinkedIn, it's business as usual.
• A reminder that the Barclays Games & Creative big 2024 event 'Barclay’s Games Frenzy' is just next week on May 16th at their Canary Wharf HQ in London. I’m excited to be hosting an AI roundtable which is - last I checked - now full. But I do believe the team are working to get extra seats in there. Either way, I’ll be around and happy to chat to people wanting to talk some more about AI for games.
• As mentioned last week, my talk at Develop Conference 2024, “A Hype-Free Overview of Generative AI for Games” is now scheduled for 2pm on Thursday July 11th. Critically, it will be available in the free ‘indie bootcamp’ track. So be sure to sign up for the free track, as I'll have some fun stuff to showcase during that talk.
• Plus, the academic game/AI conference season begins with Foundations of Digital Games coming up later this month. Sadly I won’t be in attendance, but it would be great to hear about the interesting developments coming out of the event. Hence I am creating an open call for researchers attending the conference to pitch their papers for a future FDG 2024 round-up issue. Please get in touch by DM on LinkedIn. Provide us a short paragraph about what your research paper is about, plus some links to relevant assets (the paper/videos/images etc.) that we can share with the audience.

Planning in Games

So yeah I’ve been spending a lot of time recently looking at the use of AI planning in games. This is actually a recurring point not just of course in my recent retrospective on F.E.A.R., but it will also crop up some case studies that are scheduled for later this year. While my exploration of planning in future case studies will be focussed on ‘old' games (i.e. over 10 years old), even now in 2024 it continues to have relevance, with Will Chambers of Oxide Games presenting a talk at GDC 2024 on use of planning AI for the upcoming strategy game Ara: History Untold.

The adoption of planning in games is, in many ways, a reflection of the needs of the game designers versus the ongoing trends within other technology sectors. Planning is far from in vogue when we consider the excitement surrounding AI. Most planning approaches rely on symbolic encodings of the problem space such that a plan of action can manifest through informed search. In essence finding a series of actions that achieve a goal while respecting hand-crafted rules.

This isn’t particularly exciting when put in perspective of current AI trends, with deep learning developing intelligent policies that learn what to do in any given situation. These trained models either lean directly on human experience, or ascertain good practice by experiential learning within their chosen problem domain. For a good example of this approach applied to games - though arguably not one that is reflective of actual game development practice - I’d encourage reading my article on how Google DeepMind’s ‘AlphaStar’ learned to play the popular strategy game StarCraft II.

But in truth, planning carries a lot of value for game development. As discussed at length in content on AI and Games, the vast majority of decision making systems in games for enemy opponents or non-player characters (NPCs) rely on symbolic AI techniques, be it the likes of Finite State Machines, Behaviour Trees and more. But most of these approaches are largely reactive, and can’t look ahead (or y’know, plan) for future outcomes. As such, planning seemed like a sensible approach to embrace in games, given it would allow for AI opponents to strategise at a deeper level than was previously deemed plausible. Plus, when planning was first adopted in the games industry in the early 2000’s, it satisfied the needs of game developers for several reasons, namely:

• The planner’s world logic, aka the domain, is described in high-level language, often using propositional logic that is (relatively) easy to read for non-programmers.
• Actions can have a number of constraints placed upon them, dictating the circumstances required for their successful activation. These preconditions can either be dictated by game logic, or simply designer need. Offering designers a lot of options for customisation.
• Valid plans can be quickly verified against the pre-defined domain to ensure that they make sense and against the current state of the game world to ratify they’re still plausible.
• Game designers can dictate how AI characters perceive the world and act upon it. By simply adding or removing available actions from the character, it can influence how it will behave in existing circumstances.
• It requires no training data from the designer or developer to manifest. Rather it requires a collaboration between the designers and gameplay programmers to ensure that the domain makes sense in context of the game. But ultimately, you can design the AI to know how to play the game before you’ve finished building it!

But of course it’s not as straightforward as is described. Planning AI can often be quite computationally expensive, as the number of possible states within the space of all possible plans can explode quite dramatically. This is what made the introduction of planning AI in F.E.A.R. so significant. Prior to its introduction in games, planning AI had seldom been used in systems that require real-time response, given the time taken to compute the outcome. To mitigate this, F.E.A.R.’s planner made smart concessions to the STRIPS approach of planning to make the search process computationally tractable. This has since been mitigated by a combination of smarter approaches like Hierarchical Task Network (HTN) planning, and more computational resource on contemporary hardware.

So given that I’ve had planning on the brain, it reminded me I needed to do continue my work in migrating old(er) episodes of AI and Games to the new website. Sadly over the years we’ve had some headaches with old websites being shut down and not successfully migrated. So I decided to focus on the games that include planning in them! Let’s go take a look!

F.E.A.R.

Sure, we’ve already discussed the AI of F.E.A.R. at great length last week courtesy of my retrospective episode, but it’s important to give more detail on how the AI of F.E.A.R. actually works.

Goal Oriented Action Planning as it was named is the first application of AI planning in games. Planning conceptually isn’t complicated, it’s about finding sequences of actions that make sense to have executed one after the other, but all the while respecting the constraints that exist on those actions and whether they’re applicable in the current state. But in practice, it can prove computationally demanding.

As we discussed in the retrospective the GOAP system plans to solve a problem and then translates it for execution with a finite state machine. This makes a lot more sense for game designers to get a grasp on, given FSMs are well-known and embraced throughout the industry.

The actual inner workings of F.E.A.R.’s AI was covered in a separate article, that I have only now brought over to the AI and Games Substack.

Horizon Zero Dawn

Guerrilla’s Horizon franchise isn’t their first foray into the use of AI planning. In fact, the Dutch games studio was the first to adopt HTN Planning into their games with Killzone 2 - a topic I need to write a dedicated episode for sometime!

However, Horizon Zero Dawn introduced numerous challenges as they transitioned for focussed first-person shooter combat design, to an open world full of smart, yet dangerous mechanical animals. The ‘machines’ are designed to operate either individually or in herds. These herd configurations rely on planning systems to delegate how individual characters respond to a particular scenario. Hence if the player is spotted approaching a herd, then the more docile ‘worker’ robots may fall back, even flee, while the defensive and hunter-like machines push up to find the player and eliminate them.

Find out more in the dedicated article, which summarises the two YouTube episodes linked to above and below.

Gears Tactics

2020’s Gears Tactics by Splash Damage takes the tactical strategy gameplay of X-COM: Enemy Unknown and transposes it onto the Gears of War franchise. The visceral and intense cover-based combat of ‘Gears’ translates really well into a turn-based strategy game. But it presents an interesting challenge, in that when in the heat of battle, how do you successfully communicate everything that is happening such that the player can follow along?

A driving philosophy of Gears Tactics is an idea known as ‘tactical clarity’: that players can always successfully interpret the actions of the enemy forces without it feeling overwhelming. Hence Gears Tactics adopts AI planning in a way that we seldom discuss in games, by figuring out how to schedule actions in ways that make sense for gameplay. This includes not just the notion of planning in partial-order, meaning that the final order of actions (or sub-sequences of actions) is finalised towards the end of the decision-making process, but it also finds ways to group actions to execute concurrently in ways that players will find easier to interpret.

Check out the dedicated article, or watch the YouTube video!

Halo Wars 2

Keeping with the theme of Xbox shooters as strategy games, Halo Wars 2 is a brilliant example once again of adapting popular IP to a different genre. Brought to you by Total War developers The Creative Assembly, players take charge of an army of UNSC soldiers stranded in deep space as they face off against ‘The Banished’ and their commander Atriox, who later appears as the big-bad of 343 Industry’s Halo: Infinite.

Perhaps the most exciting aspect of Halo Wars 2’s AI is the ‘commander’ system, while the game relies on a strategy planner that takes the current context and figures out how to control the individual units, there is a separate data-driven framework that allows for the behaviour of the strategy systems to be customised. This can lead to more passive or aggressive behaviours depending on context, and it is used largely to help shape the personality of each of the commanding officers across the UNSC and Stranded armies.

Once again, a dedicated article on Substack, plus the YouTube video!

Façade

Last but most certainly not least is Façade: a game that still carries a lot of excitement and enthusiasm across the internet almost 20 years after its release.

Players are introduced to the characters of Grace and Trip, a couple who you know from back in your college days. They’ve invited you round for drinks one evening to catch up and reminisce. But, things don’t seem to going so well for the married couple. The longer you spend time with them, it becomes clear how unhappy they are in their relationship, and that a lack of communication between them will come to a head this very night. Depending on how you interact with them (using typed text), they’ll rekindle their love for one another, decide to separate, or simply kick you out with the situation largely unresolved.

Developed by just two programmers over a period of 5 years, Façade was an indie-darling that received multiple awards for its unique and powerful application of AI technology for the purposes of interactive drama - and later became a cult classic courtesy of ‘Let’s Plays’ by popular content creators on YouTube and Twitch. But in order for all of that to even work, the planning technology used had to be invented! Dig into the episode to find out more.

Wrapping Up

It’s safe to say there’ll be a lot more games using planning in the future, given it has a long track record in building reliable systems for handling complex decision making in games. Of course many of the existing games that use planning technology are still topics I’d love to explore right here on AI and Games. So keep your eyes peeled!

There are also more episodes of the series dedicated to planning in games that don’t have write-ups but are available on YouTube, notably Transformers: Fall of Cybertron, and Empire: Total War.

We’ll be back next week talking about some more contemporary events in and around AI for games! But in the meantime, thanks for reading and I hope you’ll subscribe and stick around for future updates.

Thanks once again to our sponsors Regression Games. Be sure to visit their website at regression.gg to find out more about their all-in-one test automation framework.

To find out how your brand can sponsor AI and Games content, please head to our dedicated Sponsorship Enquiries page.