On the Implementation of Entity-Component Systems

[sourmìlk]

So, I was reading up on entity-component systems, and they're all the rage today in game development. It makes sense why as well; I can see how using this paradigm will help me a lot. But there's something I fundamentally don't understand:

I was reading up on entity-component systems and an article said that components should be completely unaware of each other. This makes sense, as I'd like to be able to mix and match components as I please, and because encapsulation is generally a good practice, but there's a problem: what do I do when the state of one component depends on the state of another? For example, if I have an AI Component that decides to move to a certain point, how do I tell the Collision component that my location has changed? How do I tell the animation component that it should start walking? Aren't the components supposed to be unaware of each other?

[Divinas]

Having the components completely unaware of each other is possible - through some kind of events system, but that usually creates (much) more problems than it solves.
So while you should strive for as much independence as possible, don't try to do some stupid stuff just so that you can keep them independent. Try to poll state as much as possible, but don't be afraid to push it when it needs to happen.
A somewhat standard approach is having the AI system control the animation system, which in turn modifies the location of stuff in the world, and the physics' fixed updates poll the current state and modify it.

Also something you should keep in mind is granularity: how complex should one component be, and how specific, and how generic: which brings you to two camps: the very small, very reusable components, that create huge dependency chains, or the bloated component that takes care of only one thing and can't really be reused anywhere.

[sourmìlk]

I like the tiny ones better, though I'll probably go with more of a middleground. Maybe I'll make large components out of common combinations of tinier ones or something. Anyways, what, if anything, should be contained in the entity itself? Should it have location information or what? Or is just a list of components sufficient, with the components containing the state information? I kind of like that, but it requires more interdependence between components.

[Divinas]

It really depends on what you're trying to accomplish. A lot of the 'modern' engines go with the approach, that an entity is something located somewhere in the world, having coordinates and some basic generic info (for example, update frequency - some objects need to be updated much less often than others. If you have some kind of phasing system - stuff that is active in only some states of the game, for example after you've passed some quest, you can put into the entities as well ). That has the benefit of making your editor much simpler - you can just walk around the world and look at the different things that make up your gameplay. The other approach is to have an entity without a position, and having a transform component. Fact of the matter is that you're most probably going to put a transform on just about everything, except on some gamesystems (like UI). There's no reason those two approaches to be mutually exclusive - game systems can be components, but a part of your 'levels', not entities per se.

[sourmìlk]

I don't think it makes sense to make a UI component an entity. I'll go with the location data, I think.

[Sc4Freak]

An "entity" should be an actual object in your game world - like a player character, an enemy, a projectile, a powerup, etc. Things like UI elements don't fit into that.

The idea behind components is that each entity should really be nothing more than a collection of components. Each component contains some state regarding the entity. Neither components nor entities actually contain any game logic - they only contain state. All of your game logic goes into component managers - which each take a list of components and updates them all.

Often you will need to pass information between components - which is usually done via an events system. Each component can publish or subscribe to certain events - for example a CollisionCompnent might have an event to signal when the entity has collided with something in the game world. Or a HitPointsComponent might publish an event to signal when health gets low. Other components can then subscribe to these events which can trigger certain behaviors.

[sourmìlk]

Wait, why shouldn't my components have logic in them? What benefit is there in creating a component manager to do what I could encapsulate within the component?

[Sc4Freak]

Because it defeats one of the major strength of component systems: you minimise communication overheads and you can easily "batch" updates.

Say you have a CollisionComponent that handles collision detection between components. Naive collision detection performs in O(n²) time - simply checking every object against every other object. Smarter collision detection using specialized spatial data structures allows you to perform it in O(nlogn) time. There's no clean way to do this in pure-OO style, because to efficiently perform collision detection you need access to all objects that can be involved in collision detection. If every CollisionComponent has an Update() method which independently updates each component, how are you supposed to build your octrees or whatever to perform efficient collision detection?

Performing updates in a manager makes this easy. Components hold nothing but publicly visible state - in C++ parlance, they are merely PODS (Plain old data structures). A component manager takes a list of components and updates them all at once. This makes things like collision detection simple - when the time comes to update collision, you have access to all CollisionComponents. There are plenty of other examples where this is useful. Rendering is one - you need to sort and batch your render operations for maximal efficiency.

Another benefit of this approach is easy parallelism. A very common example is a particle system - you can have thousands of particles which make up various effects in your game (eg. fire, smoke, explosions, etc) but which are very computationally complex to update. Particles are an ideal target for parallelization because each particle is typically independent - it can be updated without touching the state of any other particle. In a traditional OO system, you would have a Particle class with an Update method which would update the state of the particle. That's very hard to parallelize - the logic for updating each particle is encapsulated in the Particle class. So if you're inside the Particle.Update() method, what can you parallelize? Exactly nothing. The gains from parallelization will come from updating multiple particles simultanesouly, not by updating each particle in a multithreaded manner. But if all the logic is inside the component manager, this becomes easy - the component manager simply spins off N threads, each of which updates (total / N) particles. It's also safer to do parallelism this way - all your Particle update logic is in one place, managed by one class, and is far easier to reason about.

[sourmìlk]

Those are some convincing arguments. I'll try to overcome my distaste for singleton classes. I feel like there's not much use coding something if you're only going to use it once

[Divinas]

Although I agree on the general idea that Sc3Freak is advocating, and agree that the examples he gave are really good point, fact of the matter is that doing everything in this manner is rather counter productive. It really is a great idea for graphics and physics - that's why I said you should poll state as much as possible. But making this into a goal in itself forces you to devise some really obscure architectures, that only make code less legible, and provide little (if any) benefit.
As an example, an AI <-> Animation system. Having an AI component, which reacts to events, that puts it into a given state (most probably via some kind of behavior tree), then a manager, that parses that state (most probably, via the same behavior tree), and depending on it fires event to the animation system is a bit like doing the same thing twice. Of course you could have a behavior tree manager, and reuse the tree from there, and whatnot, but fact of the matter is that most of your AI would be have to be updated on the entity level, and you won't really be able to batch it the way you can with collision. So you would split all of your AI updatables into groups of entities that go in a given thread.

So, as a summary: what Sc3Freak said, but don't get too carried away on that and make EVERYTHING work that way. As everything in programming: have a large toolbox, and use the tools that are proper for the job at hand, because there is no silver bullet that solves everything.

[sourmìlk]

This is all excellent information. I'm not going to be implementing this for a bit (probably not until I rework my Model classes and std::thread comes out on MinGW), but I rather like this information.

[Divinas]

I would advise you to have a look at the free available engines out there - as of recently, quite a few adopted a 'free for educational purposes' policy - the Unreal Engine, CryEngine, If you have access to the Sony Games Development network - Phyre. Unity is also noteworthy. If you want to roll your own - by all means, go ahead, I guess every game developer has one behind his back, and that's a good thing. But see what the industry standard is, because the problems you're going to face have been already faced and thoroughly thought about, and you can only win by seeing how they have been solved - even if you decide to do something different.

[sourmìlk]

Well, I've seen the solutions. That is to say, I've played a lot of video games. I really should take more of a look at others' code bases though. I know how I want my engine to work and what features I want it to have, but there are a lot of specifics in there that I could learn from others from.

[Divinas]

Well, I didn't mean necessarily the code base of how they work - more like how they have organized their pipeline. How their components work, what components are there, how do they interact with each other - just running the editor and doing some really simple things with the data exposed things can help you learn a lot.

[Sagekilla]

Also something you should keep in mind is granularity: how complex should one component be, and how specific, and how generic: which brings you to two camps: the very small, very reusable components, that create huge dependency chains, or the bloated component that takes care of only one thing and can't really be reused anywhere.

You can actually get away with the small reusable components when you're doing general application development (not games).

If you use Dependency Injection, you can have a IoC container wire up all the dependencies for you. So when you say "I need a
Foo that depends on Bar and Baz", you just do something like Resolver.Resolve<Foo>() and it gives you a Foo that's all wired up.
Of course, in games this isn't always feasible because DI isn't cheap to do. Most application development can get away with it though.


Real example: I'm developing a web application in ASP.NET MVC3 and I set Ninject up as my dependency resolver. In my controllers
(stuff that passes data from the domain model to the view) I just tell them that I need something like an IEmailService, IAccountService,
etc in my constructors and it resolves them for me. Then I can happily just use the methods defined in the interfaces and have a fairly
loosely coupled system.

Everything is defined in terms of interfaces which say what functionality is required and it's extremely easy to swap out for a new component.
If my IAccountService maps to MyAccountService and I need to have it map to TheirAccountService, then I can just change one line of code in
my dependency resolver wiring code and it propagates through the whole system.

Of course, that's not really applicable for games because DI is fairly expensive compared to just calling new.

[sourmìlk]

I just thought of something. Don't Entity-Component systems with external component managers sort of defile the corpse of the concept of encapsulation? Good programming involves hiding data, and the whole point of components only containing state and being managed elsewhere necessitates that data is exposed.

[Ubik]

I don't think for proper OO you need to encapsulate every single bit of information. Sometimes tight coupling for very interrelated classes can be okay. A pretty classical example is a linked list - the list class can have full access to the listnode class. The classes are hardly usable without each other. You could consider the components to be something the manager owns and the component entity merely links together.

[Sc4Freak]

I just thought of something. Don't Entity-Component systems with external component managers sort of defile the corpse of the concept of encapsulation? Good programming involves hiding data, and the whole point of components only containing state and being managed elsewhere necessitates that data is exposed.

Yes, that's right.

But generally speaking, it's not a problem. The only point of access to a Component is through its manager - it's really the only one who can read and write to a component. Encapsulation is important in OO because objects can hold references to each other willy-nilly - in a component entity system, only the component manager can (or should) access a component. Components themselves don't hold references to each other because they should communicate via a decoupled message (or events) system.

[sourmìlk]

Well in C++ I can preserve encapsulation via use of friend classes, but what about in Java or some other language? I get that only the the manager class should bother with the more private data of the components, but I could easily make a class that accesses the private data of the components without needing that access.

[Sc4Freak]

The answer to that is "don't do that". You're trying to protect against murphy, not machiavelli. That is, your coding practices should aim to prevent mistakes, not to protect against sabotage.

For example, it's a widely accepted idiom in C++ to pass large objects by const reference into a function instead of by value. This is more efficient as it saves you a copy. But the semantics of pass-by-const-reference aren't the same as the semantics of pass-by-value, and you can do more dangerous things to a const reference (like making assumptions about the reference's lifetime) than you can with a value. For example, passing by value makes absolutely no assumptions about the lifetime of the object - whereas passing by const reference allows you to do bad and possibly unexpected things (like taking its address and storing it somewhere).

But does that mean you should always pass by value instead of const reference? No, because while it's possible to come up with a pathological case, it's highly unlikely anybody would come across that case in practical usage. Of course, it's easy to construct a case where this would fall down when a hostile programmer was determined to break your code, but that's not a very useful case to consider.

[sourmìlk]

That explanation works just as well to explain why we don't actually need member access protection at all. Why does abandoning the private and protected keywords suddenly become safer in an entity-component system than in regular old OO programming? Also, I think the point of murphy is that, given enough time, the effect ends up being the same as machiavelli.

[Ubik]

If you use the manager also as a factory of sorts in languages like Java or C#, you could have it return only a reference to a limited interface or maybe to a component base class to the user, but internally handle the object as the "real" type with public members. Anyway, not all OO supporting languages even have access modifiers (Python and JavaScript at least), but I understand how it doesn't feel like a good idea in a language that does have those constructs.

[Sc4Freak]

That explanation works just as well to explain why we don't actually need member access protection at all. Why does abandoning the private and protected keywords suddenly become safer in an entity-component system than in regular old OO programming? Also, I think the point of murphy is that, given enough time, the effect ends up being the same as machiavelli.

Okay, so let me try and explain this another way.

The fundamental reason why encapsulation, information hiding, type-safety, and other software engineering practices exist is to make it easier/cheaper/faster to write and maintain software. That is, we do these things because they provide a return on investment (ROI).

In traditional OO programming, it's common and expected for objects to hold references to each other willy-nilly. That's why things like encapsulation and access control are so important. We do these things in an OO environment because they provide a solid ROI.

In an entity-component system, it's widely understood that only the component manager should have access to the components. Of course, it's *possible* to construct a case such that you manage to circumvent this, but so what? In reality, it's not going to happen. Or at least, the probability of it happening is not worth the trouble of constructing and maintaining a convoluted system to enforce this rule. That's why people don't usually bother with encapsulation on components - because it doesn't give any appreciable ROI.

Besides, it's possible to construct a pathological case for traditional OO as well - just write "#define private public". I'm sure you can find a way to protect against that, too - but is it worth it? No, because nobody's going to do that in the first place!

[sourmìlk]

Code: Select all

#define private public


I am so going to use that sometime >:)

Anyways, I have on remaining question. How logic-free should components be? Is it okay to have a function that isn't called every frame (like a RenderComponent that loads a .md2 model containing a load() function) or should they be straight structs?

[Sc4Freak]

There's not really any compelling reason to use member functions. All fields are public, meaning a member function has no special access. And the only thing that will be able to call that function will be the component manager. So you may as well just put it in the component manager (so you can potentially reap the benefits I outlined here).

Although when it comes to loading, you probably want something a little more sophisticated anyway. You certainly don't want every particle in your game to be loading a separate copy of its mesh and texture. And you may also want to consider making a separate MeshComponent that holds data relating to the mesh itself, leaving the RenderComponent to hold information on how to render it.

[EvanED]

Besides, it's possible to construct a pathological case for traditional OO as well - just write "#define private public". I'm sure you can find a way to protect against that, too - but is it worth it? No, because nobody's going to do that in the first place!

Hah... I should totally start putting

Code: Select all

#if defined(private)
#error "Stop being a dick"
#endif

at the start of all my header files.

[sourmìlk]

Wow so just... straight structs? I may as well be programming in C :\

[Sc4Freak]

Yes, that's the idea. Like I said before, Components should basically be PODs. Use common sense, obviously, but that's the general idea of it. One of the main goals of Component systems in games is pure unadultered performance which is a requirement typically unique to game development. Components manage to achieve that, without sacrificing much in the way of readability and maintainability.

[sourmìlk]

That's a little uncomfortable for me. I think that, for the purposes of my own sanity and what I perceive to be the readability of my code, I am going to attach constructors to components. But in general I'll try to get past my love for OO programming and be a little more comfortable using component managers.

[Yakk]

The goal of most modern programming paradigms is to reduce the amount of knowledge you need to have in your head to understand what each line of code, and collection of lines, does.

OO encapsulation is one attempt at it, but it requires a particular kind of restriction on the data you are working on.

As an example of something that (java/C++ style) OO is bad at handling, look at the double dispatch problem. The collision of two objects can easily be a function of the type of both.

Properties with setters and getters are another kind of restriction that keeps things simple -- but there are a myriad of cases where there is a 2 or more dimensional state space, and you cannot go from one valid location in the state space to the other by moving along one dimension at a time. Setters that enforce data coherency thus cannot operate on a single dimension of the state space at once -- which means that your setters cannot be single-dimensional in nature.

Entity-Component-System models try to break down complexity in a different way. Entities are collections of (typed) Components, Components are collections of (typed) data, and Systems are things that operate on some subset of Components, often on each Entity in the world each "frame". Systems restrict what set of Components they work on, and what properties they work on: so long as you don't have too many Systems accessing the same Components, and/or you enforce strict state restrictions after a System modifies a Component or Entity, you can keep the code sane and maintainable. (you could imagine a Sanity System, whose job is to do such sanity checks in a "debug" build between each other System iteration).

Now, part of the reason why this kind of thing is tempting is that many games have exceedingly uniform objects -- or, at least, the above works really well for uniform objects. A person running, an alien spacecraft, and an energy blast are really similar things, as is a networked player, a local player, or an AI controlled opponent or ally.

[Divinas]

To be honest, I tend to disagree with the POD components idea. Although it sounds good on paper, I know of basically zero commercial engines that go that way for all components. For some, this is a great idea, and works marvelous. For some, it becomes a huge burden to support, and you start creating new events just to get System A to talk to System B, which in turn talks back to System A, and this is special cased for 1% of the components of that type because some game designer came up with a crazy idea that has nothing in common with how the rest of the game behaves. If you have perfect knowledge of all your game design, all your features before you start writing anything - then maybe yeah, it's a good idea. But that absolutely never happens.

[sourmìlk]

My impression was that most engines nowadays went for entity-component systems.

[Divinas]

Yes, they do, but all that I know of have a little (or in some cases, a lot) of logic inside the components themselves. I don't think I've seen an engine, where components lack update procedures where they hold some logic. Having all logic updates in a manager that only polls and pushes state to the component sounds very neat, but very rarely works for all cases. Adding a new component becomes a huge burden. Adding a new component, that does almost the same thing as another component, with a few tweaks begins to require either a new Manager, and extracing the code from the old manager someplace where it can be reused, RTTI, or some absurd branching in the manager code, that depends on some state variables.
That doesn't mean that such approach can't work - maybe it can. I'm pretty sure there are such engines, as well. But most poll state when useful (as you said, rendering, physics, pathfinding) and let the components handle themselves when it is unwieldy for a manager to handle em (AI, decision trees, animation driven movement) - stuff that can be paralelized at the entity level, usually in several steps - pre-animation, pre-physics, post-physics, and so on.