Is it really not more expensive? I haven't operated a large scale Spanner cluster, but the numbers would suggest it's quite a bit more expensive. I suppose it's workload dependent is what you're saying?
I ran some rough numbers a while ago and I found single-region GCP Spanner to be the most cost-effective relational DB option (out of the ones I looked it) if you don't mind that it can't scale to zero (min $65/mo), if synchronous replication for durability is a must and you need serializability. It's more cost effective than any Aurora offering including DSQL (which isn't serializable afaik but even then).
I concede that we do have a big lock. But that is only because we did the alternative first and it performed worse, which is what OPs article is about.
Reposting what I posted below regarding the strn.cat article:
I'm a cofounder of SpacetimeDB (and the author of OPs article). The https://strn.cat/posts/spacetime/ article has several substantial errors. I've spoken with Vicent directly about them.
Most notably, almost the entire commentary about durability is incorrect. SpacetimeDB does not acknowledge anything before data is fully persisted to disk, even though he claims it does. Clients CAN chose to listen before that, but you can do the same thing in Postgres if you want.
There is no 50 ms delay to writing to disk. The article is mostly nonsense.
He spent 15 minutes looking at our code (by his own admission), having never written a database storage engine before AFAIK, and made a pronouncement that SpacetimeDB wasn't a good database. Crazy stuff.
> I don't see anything that indicates how spacetime solves the core problem CRDB does, which is guaranteeing that single node failures can be tolerated with zero data loss or loss of availability
I think people get hung up on high availability (HA), and ironically don't test their backups. Single node with streaming backups to S3 (simple in a single writer system) can give you a system that will at most lose a few seconds of data (interestingly RDS has a 5 minute window of dataloss).
Multi node systems tend to have more failure modes, so fail more often, high availability is not free.
If you replicate data between two nodes (that are note async backups). You have to maintain strong data consistency. So writes to the database are considered successful only if the data is written to both nodes. If either node became unavailable you'd be unable to write anything without sacrificing consistency, reducing our overall availability rather than improving it. Also increasing latency.
This is why the whole multi node HA push by the industry is so laughable. A lot of time it makes systems more fragile.
> which is guaranteeing that single node failures can be tolerated with zero data loss or loss of availability
We solve this with distributed state machine replication. You don't need multiple writers to solve the single node failure problem. You only need multiple writers for a write throughput scaling problem.
I am the author of OPs article (SpacetimeDB cofounder). These are all fair caveats/criticisms.
> The benchmarks are completely meaningless for that use case.
I wouldn't say completely meaningless, but it isn't a game benchmark that's true. We feel very comfortable that it's the most performant backend for persistent games though. We were trying to show it's also more performant for web use cases.
> 90% (or more) just waiting on timers to finish
This is not correct. It's mostly processing player movement transactions. We do about 50 million an hour.
> We feel very comfortable that it's the most performant backend for persistent games though.
Depends how persistent you really need it to be. If rolling back a few minutes is tolerable in exceptional scenarios then I disagree. It will be more performant to keep state in engine and snapshot whatever you need to save every few minutes. That's pretty much what most games do now.
> This is not correct. It's mostly processing player movement transactions. We do about 50 million an hour.
I'm a cofounder of SpacetimeDB (and the author of OPs article). The https://strn.cat/posts/spacetime/ article has several substantial errors. I've spoken with Vicent directly about them.
Most notably, almost the entire commentary about durability is incorrect. SpacetimeDB does not acknowledge anything before data is fully persisted to disk, even though he claims it does. Clients CAN chose to listen before that, but you can do the same thing in Postgres if you want.
There is no 50 ms delay to writing to disk. The article is mostly nonsense.
He spent 15 minutes looking at our code (by his own admission), having never written a database storage engine before AFAIK, and made a pronouncement that SpacetimeDB wasn't a good database. Crazy stuff.
We originally did MVCC and it was actually worse performance (in our implementation, I grant), but that's what OPs article is about. We spent a lot of money finding out that a lock is more performant.
Calling it a "hashtable" is something that only someone who hasn't built a DB engine would do. It's incredibly naive. It discounts the complexity of execution, atomicity, durability, constraint validation, migrations, query planning, incremental query evaluation, down to zero. It really makes it sound like he has absolutely no idea what he's talking about.
Fresh reader here. I am very interested to learn more about your sentence
> We spent a lot of money finding out that a lock is more performant.
I want to hear about that journey.
> Besides, it's a btree.
I guess it's not a hash table, but I think the point of Vincent's post is still worth exploring. You and he both say that essentially a key/value store is mutexed, and the user's code runs inside that mutex. That is fascinating. Why would that be better? Clearly it is, or you wouldn't have spent the money. What controls did you put in place to ensure user code didn't blow up the performance, or did you even feel the need for such controls? Is WASM VM execution fast enough for this? Is there even a market for that? Who pays to put their code inside that mutex and why? I want to know more.
I don't have the whole story for you, but the key is that SpacetimeDB transactions are not interactive. The TigerBeetle team talks about this a lot as well.
The TL;DR is that because you're not holding locks across the network (as is the case in Postgres), your server code can complete transactions in single digit microseconds, rather than milliseconds. And the practical effect is you can do many more transactions per second as a result.
Would it be correct to say that this is an example of the “Actually Serial Execution” strategy for implementing serializable transactions in the terminology of “Designing Data Intensive Applications”?
(I mention this mainly as a keyword that people can look up for more information)
I know people aren't going to put more than a few minutes into verification, so that's why I suggested it. I'm not really sure what else to do. It really is all there in the code.
Oh I'm sure it very enjoyable. This is more on the question of "could it have been a better game with better reviews" if more time had been spent on gamedev versus spacetimedb?
I suspect the answer to that is probably nuanced. Spacetime probably gets more VC funding based on the DB rather than being a game. Trying to make a successful indie MMORPG without pre-existing IP is going to be an uphill battle in 2026 with or without fancy backend innovations.
> Of course, there’s not a lot of studios building MMORPGs right now, and the ones that are building any kind of multiplayer games really tend to prefer their own in-house backends
So which one is it, most mmorpgs build their own backends or clockwork shouldn't have wasted their time building their own backend?
They had to build it anyway, might as well sell it. Similar to Epic and their Unreal engine, right?
As a SpacetimeDB dev, I absolutely agree. However, these benchmarks are more than fair. Please read our full technical writeup on the benchmarks: https://spacetimedb.com/blog/benchmarking
Doesn't really cover the types used in the beta APIs. Nor would I expect it to though.
Is there a more technical explanation post somewhere? I'm definitely curious how it works, and I like the core idea quite a bit - I'm fairly convinced that WASM uses like this are going to grow immensely, and "it's just a global mutex/single thread" reduces complexity so much that it's always interesting to see where it's possible to use.
The latency+throughout brings many questions though, like "are you holding responding to writes and any reads that occur after until changes are written to disk" and "if so I kinda assume that change log is constantly streamed to disk with a moving high water mark for responses". I can generally see how that'd hit such benchmarks, especially with non-dirty reads being near instant, but details are always interesting.
Is there anything on how a multi-node cluster works (I'm kinda assuming there's something since it makes "single" mentions a few times)? Application-level sharding is always an option (and multiple things I've worked on have used it to great effect), but anything automatic seems likely gnarly and fun to learn about, if it recovers some of the naive perf loss of "just hold the lock while doing rpc to other nodes".
