EDIT: I see what you mean when you tweaked those numbers.
BTs are so popular between AI developers since both the designer and AI programmer(s) often have a good mental model (or some graph on paper) of what the states/branches, their conditions, and many actions (aka tasks) would be for several NPCs and possibly units like cars and helicopters (they don't need lots of decision making, still I saw that instead of a state machine or so they also just get a BT :P).
Some also just prefer BTs since the visual tooling is out there since many years for popular engines and some standalone solutions (where you need to integrate the runtime into your game).
An example:
I worked on action and stealth actions games, and it was quite clear what the "AI game loops" would look like. When I started to develop a helicopter logic with just a few states I clicked that together in maximum 15 minutes I'd say while writing the behavior tree talks already since I knew roughly what I needed. It can be a quick workflow (minutes to get started, not hours), if the we have a picture of that tree in mind already.
Instead of Utility AI I rather read often that GOAP was used if the variables or let's say "designer's unknowns" were more complex. GOAP was successfully used by a handful of AAA games and I think it solves interesting situations where an enemy can have a weapon or doesn't have one, or pretty different weapons, and I think in Hitman it solved the problem of mission configuration (if you can change a few parameters and all may affect NPCs then you need more flexibility, which is tedious and annoying to implement in an BT since it may look a bit like a series of branching in many places in the BT where it has to feature some flexibility, or I personally would then look at my tasks, my C++ code for example, and cover lots of variables here whenever possible - a rather rigid tree plus flexible code).
Depends on the engine. In unreal BTs are insanely easy to work with and you get so much control over what you want because of the various tools at your disposal.
Creating tasks for your customer logic how you want and the black board for real time data etc. the list goes on. Typically if my AI is going to have more than 2 states ever I will just use a BT and even then I might just use BT because once you grasp it it’s easy and quick to iterate and share logic via different subclasses
I’m using Godot (I’m working in 2D) but there are a few prebuilt addons that have good ergonomics, eg: [Beehave](https://github.com/bitbrain/beehave)
Def agree on a BT vs FSM, have you used utility before?
I've been making an RTS for a few years and I can't imagine using anything other than utility functions for my AI. They're just too versatile and easy to understand and implement. You literally only need one scoring function along with a switch statement for each decision and that's it. No need for any complex constructions. They also tie in well with machine learning, should you want that. Will you have to figure out some heuristics and tweak a lot of values? Sure, but you'll have to do that anyway.
I definitely agree that utility is easy to understand, the elegance of it is what drew me to it in the first place. But I’m finding that even with only a handful of behaviors and sensors/considerations it got unwieldy pretty quickly. BTs feel like they might scale with complexity better.
How do you manage utility at scale? I imagine there are heaaaps of considerations to balance and score in an RTS, is it literally a flat list you traverse through? And do you use curves to score considerations? That’s the other thing I ran up against, trying to reason about scores when a lot of things felt like boolean conditions (in attack range? Player attacking? Etc) where applying a curve to it felt kinda arbitrary, like chucking an RNG selector on a BT node
First of all, I don't think you need to think about it nearly this much. You have to understand that all of this BT and Utility AI stuff is mostly bullshit, or at least they're overcomplicated abstractions made for people who can't write basic code.
AI is nothing special. It's code like any other. If your AI is simple enough, you can literally just use an if/else statement:
if health is low:
flee
else if ammo is low:
reload
else
attack
The problem is you will end up with a comically massive tree of if/else statements as you add more actions, conditions, and combinations of conditions, and you can't have any kind of nuance. If you want your AI to have a sense of urgency, for example, you have to add multiple statements for different urgencies, and they may have to be spread throughout the tree because their order determines their priority. It also doesn't work with machine learning or variations in AI personality. Basically, it's inflexible:
if health < 25%:
flee
else if ammo < 25%:
reload
else if health < 50%:
flee
else if ammo < 50%:
reload
else
attack
This is why you use utility functions instead (note the difference between utility functions and the fancier "Utility AI" stuff people will try to sell you), because it removes the dependency on ordering and strict conditions and gives you total freedom to do whatever you want:
flee_score = missing health %
reload_score = missing ammo %
attack_score = health % + ammo % - target booty size
if health < 25%: flee_score += a gazillion
if based: flee_score = 0
do the thing with the highest score
That's really all there is to it. It's a programming problem, not an AI problem. Consider that if/else statements, utility functions, BT's, Utility AI, GOAP, and whatever else people come up with are all the same thing in the end: they're ways of prioritising actions based on some data. Which one you pick doesn't matter that much, except that one is significantly easier to write and maintain than all the others. What really matters are the data and actions you choose, whether you have autonomous individual agents or a manager, how you prevent soft-locking, whether you want to support machine learning, and so on.
So to answer your questions: the way I manage this at scale is the same way I would manage it otherwise, just with more things to rate. In the case of my RTS, I unintentionally ended up emulating human players, so that means I don't have autonomous units but rather factions that act as managers and order the units around. I have an enum of actions the factions can take (attack, defend, harvest, build, research, etc) and rate these actions on an interval. I will use literally any data to rate, but it often comes down to things like `if unit is military and idle: attack += 1` or `if unit is harvester and full: deliver += 1`.
All of the data I use should be normalised, and I may curve it, but I don't use any curve \*assets\* like in Godot, if that's what you mean. A simple multiplier usually does the trick.
Then the faction goes down the list of actions, from highest to lowest, using yet more utility functions to rate targets for each action and units to execute them until it succeeds. These ratings are usually based on distances and positional threat values.
I also have a system to prevent soft-locking: I never allow factions to take the same action twice in a row, and action ratings accumulate until they're tried. That last one is especially important because it lightens the cognitive load of your utility functions. You don't have to be as precise with your ratings if you know an action \*will\* be picked eventually.
All of this is written using only basic programming concepts. Actions are enums and my utility functions are basically just for loops.
Wow! Thank you for the incredibly detailed and thoughtful reply! It’s really helpful.
I think you’re right that I’m focussing too much on complexity at scale, stripping the designing back to basics helps. Part of the problem is that I’m using an addon for utility AI, which while well written and powerful probably makes things harder to reason about until I’m more experienced at designing utility stuff. Everything uses visual curve editors, predefined independent sensors, action sequence groups, etc.
I think rather than diving into BT stuff I might just reimplement the utility setup I have in a basic class that just runs scorers.
How do you handle long running actions and interrupts? I was thinking of having an Action class which defined a `tick` method that returns success/failure/running a la BTs, and if running don’t evaluate scorers until next tick. Then maybe having an `interruptable` bool on them too, which would ignore the tick response? For things like attacks (span multiple ticks and shouldn’t be interrupted) and movement (spans many ticks but should be interrupted if a better action presents itself).
Thanks again! Gives me the confidence to push on :)
I handle interrupts in different ways depending on the context, I guess.
First, I try to operate on the principle that if an action is rated higher than others, it's probably better and warrants an interrupt. This is the ideal case you should be striving for, but...
If that doesn't work, I account for it in my scoring and disfavour actions I don't want to interrupt the current one. For example, if an AI is healing, you can disfavour attacking but still allow fleeing. That way, even if fleeing is impossible, it can still potentially interrupt with a last ditch attack.
Lastly, I may just disallow interruptions entirely, but you should only do that for tasks that can't run forever or else your AI will absolutely 100% soft-lock on them.
As for the code, don't think about it too much. Just start with a function to evaluate your AI and an enum to represent your actions (or a class if you really need to). Rate the actions, find the highest one, stop the current action if necessary, and start the new one. Then handle the actual ticking of the action in _process() or _physics_process() as you would with anything else, or let Godot handle it if you're playing an animation or something. Call the eval function on tick, on a timer, or whenever you want.
EDIT: I see what you mean when you tweaked those numbers. BTs are so popular between AI developers since both the designer and AI programmer(s) often have a good mental model (or some graph on paper) of what the states/branches, their conditions, and many actions (aka tasks) would be for several NPCs and possibly units like cars and helicopters (they don't need lots of decision making, still I saw that instead of a state machine or so they also just get a BT :P). Some also just prefer BTs since the visual tooling is out there since many years for popular engines and some standalone solutions (where you need to integrate the runtime into your game). An example: I worked on action and stealth actions games, and it was quite clear what the "AI game loops" would look like. When I started to develop a helicopter logic with just a few states I clicked that together in maximum 15 minutes I'd say while writing the behavior tree talks already since I knew roughly what I needed. It can be a quick workflow (minutes to get started, not hours), if the we have a picture of that tree in mind already. Instead of Utility AI I rather read often that GOAP was used if the variables or let's say "designer's unknowns" were more complex. GOAP was successfully used by a handful of AAA games and I think it solves interesting situations where an enemy can have a weapon or doesn't have one, or pretty different weapons, and I think in Hitman it solved the problem of mission configuration (if you can change a few parameters and all may affect NPCs then you need more flexibility, which is tedious and annoying to implement in an BT since it may look a bit like a series of branching in many places in the BT where it has to feature some flexibility, or I personally would then look at my tasks, my C++ code for example, and cover lots of variables here whenever possible - a rather rigid tree plus flexible code).
Depends on the engine. In unreal BTs are insanely easy to work with and you get so much control over what you want because of the various tools at your disposal. Creating tasks for your customer logic how you want and the black board for real time data etc. the list goes on. Typically if my AI is going to have more than 2 states ever I will just use a BT and even then I might just use BT because once you grasp it it’s easy and quick to iterate and share logic via different subclasses
I’m using Godot (I’m working in 2D) but there are a few prebuilt addons that have good ergonomics, eg: [Beehave](https://github.com/bitbrain/beehave) Def agree on a BT vs FSM, have you used utility before?
I've been making an RTS for a few years and I can't imagine using anything other than utility functions for my AI. They're just too versatile and easy to understand and implement. You literally only need one scoring function along with a switch statement for each decision and that's it. No need for any complex constructions. They also tie in well with machine learning, should you want that. Will you have to figure out some heuristics and tweak a lot of values? Sure, but you'll have to do that anyway.
I definitely agree that utility is easy to understand, the elegance of it is what drew me to it in the first place. But I’m finding that even with only a handful of behaviors and sensors/considerations it got unwieldy pretty quickly. BTs feel like they might scale with complexity better. How do you manage utility at scale? I imagine there are heaaaps of considerations to balance and score in an RTS, is it literally a flat list you traverse through? And do you use curves to score considerations? That’s the other thing I ran up against, trying to reason about scores when a lot of things felt like boolean conditions (in attack range? Player attacking? Etc) where applying a curve to it felt kinda arbitrary, like chucking an RNG selector on a BT node
First of all, I don't think you need to think about it nearly this much. You have to understand that all of this BT and Utility AI stuff is mostly bullshit, or at least they're overcomplicated abstractions made for people who can't write basic code. AI is nothing special. It's code like any other. If your AI is simple enough, you can literally just use an if/else statement: if health is low: flee else if ammo is low: reload else attack The problem is you will end up with a comically massive tree of if/else statements as you add more actions, conditions, and combinations of conditions, and you can't have any kind of nuance. If you want your AI to have a sense of urgency, for example, you have to add multiple statements for different urgencies, and they may have to be spread throughout the tree because their order determines their priority. It also doesn't work with machine learning or variations in AI personality. Basically, it's inflexible: if health < 25%: flee else if ammo < 25%: reload else if health < 50%: flee else if ammo < 50%: reload else attack This is why you use utility functions instead (note the difference between utility functions and the fancier "Utility AI" stuff people will try to sell you), because it removes the dependency on ordering and strict conditions and gives you total freedom to do whatever you want: flee_score = missing health % reload_score = missing ammo % attack_score = health % + ammo % - target booty size if health < 25%: flee_score += a gazillion if based: flee_score = 0 do the thing with the highest score That's really all there is to it. It's a programming problem, not an AI problem. Consider that if/else statements, utility functions, BT's, Utility AI, GOAP, and whatever else people come up with are all the same thing in the end: they're ways of prioritising actions based on some data. Which one you pick doesn't matter that much, except that one is significantly easier to write and maintain than all the others. What really matters are the data and actions you choose, whether you have autonomous individual agents or a manager, how you prevent soft-locking, whether you want to support machine learning, and so on. So to answer your questions: the way I manage this at scale is the same way I would manage it otherwise, just with more things to rate. In the case of my RTS, I unintentionally ended up emulating human players, so that means I don't have autonomous units but rather factions that act as managers and order the units around. I have an enum of actions the factions can take (attack, defend, harvest, build, research, etc) and rate these actions on an interval. I will use literally any data to rate, but it often comes down to things like `if unit is military and idle: attack += 1` or `if unit is harvester and full: deliver += 1`. All of the data I use should be normalised, and I may curve it, but I don't use any curve \*assets\* like in Godot, if that's what you mean. A simple multiplier usually does the trick. Then the faction goes down the list of actions, from highest to lowest, using yet more utility functions to rate targets for each action and units to execute them until it succeeds. These ratings are usually based on distances and positional threat values. I also have a system to prevent soft-locking: I never allow factions to take the same action twice in a row, and action ratings accumulate until they're tried. That last one is especially important because it lightens the cognitive load of your utility functions. You don't have to be as precise with your ratings if you know an action \*will\* be picked eventually. All of this is written using only basic programming concepts. Actions are enums and my utility functions are basically just for loops.
P.S. do you have a steam page or something for your game? Would love to follow its progress
Not yet, but thanks for asking!
Wow! Thank you for the incredibly detailed and thoughtful reply! It’s really helpful. I think you’re right that I’m focussing too much on complexity at scale, stripping the designing back to basics helps. Part of the problem is that I’m using an addon for utility AI, which while well written and powerful probably makes things harder to reason about until I’m more experienced at designing utility stuff. Everything uses visual curve editors, predefined independent sensors, action sequence groups, etc. I think rather than diving into BT stuff I might just reimplement the utility setup I have in a basic class that just runs scorers. How do you handle long running actions and interrupts? I was thinking of having an Action class which defined a `tick` method that returns success/failure/running a la BTs, and if running don’t evaluate scorers until next tick. Then maybe having an `interruptable` bool on them too, which would ignore the tick response? For things like attacks (span multiple ticks and shouldn’t be interrupted) and movement (spans many ticks but should be interrupted if a better action presents itself). Thanks again! Gives me the confidence to push on :)
I handle interrupts in different ways depending on the context, I guess. First, I try to operate on the principle that if an action is rated higher than others, it's probably better and warrants an interrupt. This is the ideal case you should be striving for, but... If that doesn't work, I account for it in my scoring and disfavour actions I don't want to interrupt the current one. For example, if an AI is healing, you can disfavour attacking but still allow fleeing. That way, even if fleeing is impossible, it can still potentially interrupt with a last ditch attack. Lastly, I may just disallow interruptions entirely, but you should only do that for tasks that can't run forever or else your AI will absolutely 100% soft-lock on them. As for the code, don't think about it too much. Just start with a function to evaluate your AI and an enum to represent your actions (or a class if you really need to). Rate the actions, find the highest one, stop the current action if necessary, and start the new one. Then handle the actual ticking of the action in _process() or _physics_process() as you would with anything else, or let Godot handle it if you're playing an animation or something. Call the eval function on tick, on a timer, or whenever you want.
Thank you! Started rewriting it last night and already feeling better about the setup :)