This reminds me of a portion of a talk Jonathan Blow gave[1], where he justifies this from a productivity angle. He explains how his initial implementation for virtually everything in Braid used arrays of records, and only after finding bottlenecks did he make changes, because if he had approached every technical challenge by trying to find the optimal data structure and algorithm he would never have shipped.
"There's a third thing [beyond speed and memory] that you might want to optimize for which is much more important than either of these, which is years of your life required per program implementation." This is of course from the perspective of a solo indie game developer, but it's a good and interesting perspective to consider.
It's also notable that video games are programs that run for hours and iterate over large sets of very similar entities at 60 frames or more per second repeatedly and often do very similar operations on each of the entities.
That also means that "just do an array of flat records" is a very sane default even if it seems brutish at first.
I think when he said "just do an array of flat records" he meant as opposed to record of arrays (i.e. row oriented vs column oriented), as opposed to fancy data structures which I think you're assuming he was implying. Separate arrays for each data member are common in game engines exactly because they're good for iterating over, which as you said is common.
That's also great for letting the compiler unlock auto-vectorization opportunities without delving into the world of manual SIMD.
Even storing something simple such as an array of complex numbers as a structure of arrays (SoA) rather than an array of structures (AoS) can unlock a lot of optimizations. For example, less permutes/shuffles and more arithmetic instructions.
Depending on how many fields you actually need when you iterate over the data, you prevent cache pollution as well.
I haven't watched his videos on his language for ages, but this was a big thing he wanted in his language: being able to swap between array-of-structs and struct-of-array quickly and easily.
I would like to add that Braid is not a normal video game.
To me, it is a piece of art. It contains so many mind-bending mechanics that I think it is a real possibility that you could have worked on some of that stuff for years, making it unlikely to ship at all.
So, in addition to the performance requirements of a video game, Jonathan also had to find ways to implement all those fancy mechanics.
It's easy to see this outside of the perspective of a solo game developer. We all have deadlines to manage even in the regular 'corporate world'. And engineering time spent on a problem also translates to an actual cost.
I'd be careful extending learnings from games (solo or team efforts) to general programming as the needs and intent seem to be so different. We rarely see much code re-use in games outside of core, special-purpose buy largely isolated components and assets. Outside of games there's a much bigger emphasis on the data IME, and performance is often a nice-to-have.
Yeah - A game is (generally) a one and done enterprise. Like a start up it's all about getting it out the door, there's little to no expectation of having to maintain it for any real length of time.
This comment assumes game companies throw away all their code and start from scratch on their next title. Which is completely untrue, games are built on decades old code, like most software. There is absolutely a need for maintainable code.
I think that there might be a distinction on indie vs big publishers (like EA) - I would expect someone like EA to enforce coding standards, but they do so whilst living off the income of previous hits.
We can probably add to that the fact that the nominated titles almost certainly started out with spaghetti code that had to be refactored, reworked, and gave maintainers nightmares in their sleep
Roblox didn't have server functions until 2014 and they weren't mandatory until 2018. Anything the client did automatically replicated to every other client.
That meant I could attach Cheat Engine to Roblox, edit memory addresses to give myself in-game cash, and inject whatever code I wanted to everyone else.
Thankfully it was sandboxed so I couldn't give people viruses.
This was probably a good decision because most of Roblox's best games from that period weren't updated to use server functions. It's too difficult.
It's been mixed moving to normal code: I haven't had to low-level optimise for ages now (man I miss that). But performance in the O() sense has been the same.
Game engine development is very much about processing of data. The pipeline is long and the tree is wide. Being able to reason about complicated data processing topologies mapped very easily across.
That's very much the norm in the game dev world, especially for someone like Jonathan who's been doing this since the early era of PC gaming.
You've got 16 ms to get that next frame rendered, so stuffing everything into fixed sized tables and linear walks is a simple way to structure against that limit. You've got an entity budget of so many, and there's no unpredictable delays from dynamic allocation or similar.
Even when using scripting languages that offer more flexibility, like LUA or C#, you see a tendency towards this pattern.
Wow! I'm shocked Limbo took 3 years and had a larger team. I've played a bit of the game (but didn't complete it). Looking at the Steam reviews and a quick search shows me it takes 3-10 hours to beat the game. :O
IIRC the Limbo team grew over time, it shows in gameplay, the second half feels less personal and more like a generic platformer. They hired more people to get done with it. It don't mean it is rushed, but to me, the second half lacks personality compared to the first half.
Also, I don't like the idea of using gameplay time as a value statement. Maybe I say that because I don't have money problems, but I find that the tendency certain gamers have of judging games in terms of dollars per hour of gameplay is pretty damaging, as it incentivizes developers to focus on gameplay time more than polish. 3-10 hours is already plenty for that style of game. Note that there are many AAA games in the 10 hour range.
The funny thing is even data structure is really an array of memory depending how low down deep one looks. All the people who took an intro computer science course knows, those who had to implement a heap allocator, it's all really just arrays underneath...
"There's a third thing [beyond speed and memory] that you might want to optimize for which is much more important than either of these, which is years of your life required per program implementation." This is of course from the perspective of a solo indie game developer, but it's a good and interesting perspective to consider.
[1] https://www.youtube.com/watch?v=JjDsP5n2kSM