I find this to be a bit of an oversimplification. It's certainly not the case that there's only one type of storage, at least from a hardware perspective: there will always be storage that is faster than other storage, and managing that will always be required to optimize performance. I agree with the author that you should program against the simplest abstraction you can, but that's not always clear. Just because your environment is one in which you don't have to doesn't mean you can't or shouldn't in the right circumstances.
I'd argue that it's actually more complex now. I programmed in the early 1980s, and did have to worry about where my data physically resided inside the machine, but I never had to worry about whether it was in a different room on a different machine, or sitting on a hard drive in some Amazon Data Center a continent away.
As others here note, the idea of using the same abstraction for all storage is at least as old as MMUs and Multics (mid 1960s). What is different is that programmers in the 60s and (early) 70s were usually coding pretty close to the bare metal. Nowadays, Moore's law has allowed us sufficient power to permit programming on top of a big pile of abstractions that try very hard to hide the actual hardware from us.
That's a luxury afforded by the sheer power of what we're working with, but the people writing those abstraction layers still have to pay attention to the layer beneath them, and if you go down far enough, you'll find some code that needs to pay attention to what class of memory something is stored in and how to optimally move it around. It's just that work was probably done for you by someone else who wrote your operating system.
Just because your programming language doesn't require you to use pointers doesn't meant that indirection isn't being used. You just don't have to deal with it (until it rears up and reminds you it's still there). Joel Spolsky's Law of Leaky abstractions (https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...) is relevant here.
> I'd argue that it's actually more complex now. I programmed in the early 1980s, and did have to worry about where my data physically resided inside the machine, but I never had to worry about whether it was in a different room on a different machine, or sitting on a hard drive in some Amazon Data Center a continent away.
Oof, I hear that. That’s just the tip of the iceberg. We also design for those computers being deleted or disconnected spontaneously. Even more complex than designing distributed systems (in my opinion) is designing the continuous deployment systems which get our code from git to running in production (without running incompatible versions of services even for a moment). It certainly makes the idea of writing single-host software seem simple and romantic.
But the whole point of the article is that given the way modern hardware and kernels work, you don't actually have the ability to program against lower layers of abstraction, and attempting to do so can degrade performance.
I agree completely that you shouldn’t be digging into that stuff unless you have a reason to and possess the appropriate competence. But the reasons for doing that is the same everywhere: (1) complex systems are difficult and subtle, and (2) “premature optimization is the root of all evil”.
In fact, Knuth’s aphorism is a much better description of why one should avoid the complexity of this issue than a clearly incorrect argument proposing that the distinction between storage classes doesn’t exist, that 1975 programmers were ignorant of storage abstraction, and that this issue has become less complex over time. (Edit: corrected typo. Programmers in 1075 were definitely unfamiliar with storage abstractions.)
The post should be marked as [2006]. A lot has changed since then. For example it doesn't talk about NUMA but it should, if it were written today.
I would love to hear some opinions from the authors of more contemporary web servers like one of H2O or Envoy. (The latter may have slightly different use cases though.)
NUMA is interesting, I've had a lot of fun with it on a system to control the adaptive mirrors in ESO's ELT telescope, and it is /amazing/ what you can do with a modern server, even on a vanilla kernel.
However, it is a lot harder to exploit NUMA in an application like Varnish, because the requests arrive as a randomized stream, and the effort to figure out which NUMA pool is best situated to handle the request, is not significantly different from just handling the request to begin with.
We used to serve a cache hit in seven system calls, an accept(2), a read(2), a write(2) and four timestamps, and as a result, Varnish servers ran with CPU's >70% idle.
If we had added NUMA-awareness on top of that, we would on average Nnuma-1/Nnuma of the time add at two system-calls to migrate the request to another NUMA domain, and that made little sense.
Since then NICs have become NUMA aware and since we use scatter/gather I/O, that matters a lot more than which CPU core did the overhead processing.
Where NUMA may become mandatory is 100G and 400G networking, but very few people seem interested in running that much traffic through a single server, for reasons of reliability, but I'm keeping an eye on it.
It will filter incoming TCP connections to listen sockets owned by threads bound to the same NUMA domain as the NIC that received the packets. This is part of the Netflix FreeBSD NUMA work described here https://papers.freebsd.org/2019/eurobsdcon/gallatin-numa_opt...
The idea is that you can keep connections local to a NUMA domain. We (Netflix) have a local patch to nginx to support this (basically just setting the socket ioctl on the listen socket after the nginx worker is bound)
Why did the timestamps require syscalls / do they still do now? (I know CPU counters were a mess back then, but was there any other factor about that?)
Well if by contemporary you mean written by Google builds with Bazel and runs in Docker, stuff like Envoy is pretty good and it uses Joyent's HTTP parser which is more readable than the pigeon C parsing Varnish likes to do. But Varnish understands UNIX much better than Envoy does, as evidenced by its portability and the things you've read in the blog post. Compared to Squid then hands down Varnish is the best. I'm not sure if it's better than open source GFE but it's pretty good.
None of them go as fast as redbean of course. The blog post talks about how they only need 18 system calls to serve a local asset. redbean only needs one system call: writev(). That's because redbean cheats by having a novel design. The zip central directory means we don't need file system system calls. The invariant RDTSC instruction means we can also have timestamps without context switching.
Even for 2006, what would make this argument for me is actual numbers. Is it plausible that some programs worked well like that then? Sure. But I can promise not everything did. The kernel's approach to caching is optimized for the general case, but each program is specific. It's plausible to me that a caching service like Varnish is a very close match. But that's an empirical question, not a theoretical one.
I would say nothing, except maybe for how many mentions it gets in HW-review articles.
Unless you are in numerical HPC or kernel programming, you do not need to think about NUMA, but it is not black/white, it is a slope.
Varnish tries to do certain things "NUMA-friendly", without actually being "NUMA-aware" as such, and the kernels seem to do a really good job with that.
That makes sense to me for the entire computing landscape, like if we're including windows, various 32-bit x86 unix like OS's, etc. But varnish was used on Solaris, IRIX, and HP/UX. All of which had cc-NUMA support fairly well established by 2006. I suppose, though, it was still new-ish.
A webserver, as bare minimum, should be able to serve static files from the disk - but you are correct, there is no such definition, so this is an opinion.
Regardless, calling envoy a webserver is misleading: it's both more and less, than a "traditional" webserver. It's a proxy; call it a proxy, please.
As a historical note, I'll point out that 1975 computers weren't as primitive as the article implies. Virtual memory was introduced commercially in 1961 with the Burroughs B5000. Cache memory was introduced on the IBM System/360 Model 85 in 1969.
On what design features came to market when, two examples:
(1) In 1972, I was at Georgetown U., and IBM was proposing that we buy a 370/135 with virtual memory.
(2) In 1973 there was the IBM 360/67 with 31 bit addressing, virtual memory, and virtual machine.The virtual machine software was CP/67 (Control Program 67), and the user interface was from CMS (Conversational Monitor System). I programmed it in PL/I to schedule the fleet at FedEx.
RCA/Sperry Univac had reasonably large scale time sharing systems running basic, FORTRAN, apl, algol and 360-compatible assembly language. They had both physical and logical I/O APIs and supported vm to both disks and drums. Fill in the rest of the old time computer room as you’d like. Did a chess program and various other things in assembly, wrote the required Star Trek game in basic etc. As they provided the source code to the OS, even messed around with changing things like adding 2-step logins and other hacks.
Basically, that decade saw the continuation and growth of the hacker culture that started in the early 60’s on trivially small machines (google spacewar). As there were no small machines, things were more social / collaborative than now out of necessity. And yeah, the hardware is bazillion times faster but the software less so as code bloat is real.
Just from the title, I would have expected the opposite approach: not treating RAM as a disk cache, but using RAM only and ignoring the disk. One of the biggest differences between now and then is that we've gone from RAM deficit to RAM surplus. (The VMs that contain so much of today's software can be seen as a way of cutting too-big physical RAM back down to slices that match problems.)
This means that batch orientation makes sense for a much smaller slice of problems. Compilers, for example, come from a world of scarce RAM. When my dad started coding, one iteration of change-the-code-and-watch-it-run took days. Now that loop can be seconds: if I pause typing, my IDE starts running my unit tests automatically. It seems to me that things like compilers and runtimes should by default keep everything hot, incrementally updating as fast as possible.
> Now that loop can be seconds: if I pause typing, my IDE starts running my unit tests automatically. It seems to me that things like compilers and runtimes should by default keep everything hot, incrementally updating as fast as possible.
Is it only me that finds this incredibly distracting?
When I write code I pause often. It might be in the middle of a sentence, or a function definition, the most recent example I can think of is because I realised I needed another function, and this function's state is dependent on it, or I needed to look something up. Getting my mindstate trashed every time I pause because it's found a dozen irrelevant errors sounds horrifying.
Because there's latency. It wouldn't be distracting if there's no latency at all because it feels natural even if the code is in stale shapes.
I use Wallaby[0] to run JavaScript tests, and it shows the result on the fly just near every line of test code. The latency is very small so it's hard to notice it.
After I was used to this approach the idea of press the test button, and waiting for 1s to compile, another 3s to run tests is simply tormenting. It breaks the flow because I have to repeatedly wait for the computer to catch my thought.
I think one could make the same argument for a zillion things a modern IDE does, from syntax highlighting forward. It's more of a problem in theory and on early use than it is for sustained use.
For unit tests in specific, if the interface is designed well and one is used to TDD anyhow, I think the distraction goes away pretty quickly. When I'm used to TDD, I always have a bit of my mind on what the test impact is, so if the errors showing are the ones I expected anyhow, then it's not disruptive at all. Indeed, it's welcome confirmation that I'm not off track.
That's how syntax highlighting and feedback feels for me too. I may pause at a point with unparseable code, so some things will be the wrong colors and may be marked with issues. It's fine, because that matches my mental syntax parser. And when it doesn't, I'm glad to learn as early as possible that I have an issue somewhere.
We have come a long way since 2006 in terms of both storage and RAM. There are a lot of Varnish servers out there running straight out of RAM, and some of them have a /lot/ of RAM.
But the most important question is your data set and your traffic pattern, and whatever that is, Varnish can deal with it, given suitable hardware.
As a general rule, I would suggest people refrain from trying to express what my viewpoint is, because, quite frankly, N-1 person on this planet suck at it :-)
There still is a cost differential between 1TB RAM and 1TB disk, and that matters in the real world.
Varnish lets you decide where you want to spend your money, if you want to run straight out of RAM, it can do that, if you want to have a bigger cheaper and slightly slower cache on disk or ssd, you can do that too.
I didn't put enough caveats, I should have said, "the viewpoint that I think phk is trying to express _in this article_ ..." :)
You have written enough publicly that we could probably cook up a GPT2 level simulacrum with your posts. Useful for code reviews, discussions, etc.
Aren't we saying the same thing? The RAM is there to support the disk and the disk holds the data (the asset), so your article is disk centric. If disk was fast enough, there would be no ram. The ram is just a tool to support a latency target.
Remember 2006 ? You know, back when Twitter was a new thing :-)
Back then you couldn't buy machines with 1TB of RAM, most systems were 32bit and limited to about 3GB usable RAM.
Varnish was written with 64bit in mind from the start, but it also had to work on 32 bit systems, so we had to be able to use disk-backed VM to get past a couple of GB.
This post elicited a response from antirez back in the day. Then he and "the varnish guy" /g had a meeting of geek minds. You'll learn quite a bit from both.
This article is a bit strange and conflates disk storage with swap and RAM. Modern systems don't necessarily even have swap (looking at you, k8s), so the whole article falls on its face.
> And then there were the secondary store, paper tape, magnetic tape, disk drives the size of houses, then the size of washing machines and these days so small that girls get disappointed if think they got hold of something else than the MP3 player you had in your pocket.
> This article is a bit strange and conflates disk storage with swap and RAM.
The author is pointing out that many programs duplicate OS virtual memory functionality by paging temporary data to persistent storage and loading it when needed. This duplicates the operating system's built in virtual memory capability and has negative effects on the system. The whole idea of virtual memory is to allow a system to handle loads where memory allocated exceeds the size of RAM.
> Modern systems don't necessarily even have swap (looking at you, k8s)
Modern linux does have swap, and it is quite useful. Proper support for swap is coming in k8s (looks like 1.23). Quite a few workloads need swap to run safely, so adding this to k8s will be an improvement.
>> Modern systems don't necessarily even have swap (looking at you, k8s)
> Modern linux does have swap, and it is quite useful. Proper support for swap is coming in k8s (looks like 1.23). Quite a few workloads need swap to run safely, so adding this to k8s will be an improvement.
Ah, I see the HN pedants have arrived. I was not claiming that Modern Linux does not have swap. Note the words "don't necessarily". I was claiming not all machines have swap.
Most of mine do not have it enabled, and I take effort to ensure on machines with 16GB or more RAM that it is disabled. I do run a K8S cluster as well, and those machines do not have swap enabled, not because K8S requires it to be off, but because having it on would be adverse to the health of the flash storage they use.
TBH, this is the kind of vapid pedantic response I've come to expect of the average of HN comments these days, and why I'm considering just not using it anymore at this point.
It would have been funny if I was still 15. But then again, this article is from 2006; The blogosphere was a different place in those days, and I've matured since then as well - hopefully the author has too. Still: Bad joke was bad.
Yeah, that is a really bad take. Remind me again why women drop out of engineering teams. To frame this triggering statement, imagine if he had somehow used breasts to make the same analogy. He’d be in a heap of trouble, prob LIFO’d out of his job
The challenge is to be able to understand how a particular post would appear to various other people who are unlike yourself. So I think you should answer that question yourself. Expend some effort or imagination, or else there’s no point.
Thank you, but there’s hardly any point in arguing with a rock. You might be making great points, but at the end of the day, observers see a man talking to a rock. The rock may roll at some point, through forces beyond yourself
Your comments aren't offensive, but they seem like externalizing your own issues. Asking someone to consider what's offensive isn't constructive, when you aren't able to describe what you think is offensive and why.
I just imagined what if we have to consider other civilizations for our posts and what they might be offended by.
For example, the civilization of Dune is offended by any mention of complex "thinking machines", so any programming topic to them would be like an article full of dick jokes.
Imagine we get visited by aliens, but no matter what we do, they get gravely offended, and we can't understand why.
The joke denies that the reader may be not male. It also shows women where their place is: near the man’s genitals. I can’t see how this is not offensive.
This is probably the thing that annoys me the most about golang: It abstracts everything as if you were on a 1975 computer, so there's allocations, allocations everywhere! And trying to keep them under control requires intimate knowledge about go's implementation details. Failure to do so dooms your program to run 3-4x slower since allocations are horribly slow to begin with, and now your data is spread all over the place. And even then, there are still some things (like []byte <-> string) that you simply can't get around (unless you want to delve into unsafe land and risk your code blowing up once the runtime implementation changes).
To be fair to Go, it provides an escape analysis log and a memory profiler in the standard distribution, so while fixing the heap allocations can require good knowledge of implementation details, finding out that you have heap allocations is really easy.
Go is also not that clever, so you can often intuit fairly easily where the allocations are taking place and how to avoid them. As far as optimizing goes, it’s a lot easier to optimize away allocations in Go than in many other languages. I also think this is the right tradeoff—most code isn’t that sensitive to allocations so why make the programmer think about the 100% of the time, rather than opting into thinking about them in order to optimize the hot path?
I think this comes down to the right tool for the job.
If you have a lot of programmers churning out code, languages like golang makes it harder for them to make mistakes, sometimes at the cost of performance, but 99.9% of the time, performance does not matter.
If performance matters, there is no better language than C to bend the hardware to your precise will, but that comes at a cost of programming difficulty.
>It abstracts everything as if you were on a 1975 computer, so there's allocations, allocations everywhere!
Care to elaborate? Generally you don't care about memory management in Go at all, so I wouldn't call it 1975 programming where you certainly would. Sure, taking the address of a variable heap-allocates but that's an implementation detail and I don't see how it's related to the article at all. It's automatic and you don't even think about it. The article talks about userland software reinventing what's already implemented in a modern operating system. How is it the case with heap allocations in Go, escape analysis etc.? Operating systems don't have means to do escape analysis for userland variables. Or did you get the conclusion from the article that slow=1975 programming? I don't think that's what the author meant.
Abstractions do have costs, no doubt. I just don't see the connection between "Go heavily uses interfaces which force heap allocation in the current implementation" and "1975 programming". How would one go from 1975 to current year? By stopping using abstractions? Isn't that exactly what 1975 programming was to begin with?
Varnish seemed really popular a decade ago, but I wonder how it fits the modern web and who's using it today and for what purpose.
The lack of built-in HTTPS seems killer to me. On the client-facing side it needs a separate daemon for TLS termination and on the upstream side there's no TLS support at all unless you're using the paid version.
The reason why Varnish does not have built in TLS, is a matter of security design and flexibility.
First, as to flexibility:
If we added TLS support to Varnish, we would have to pick a TLS library, which means our users would automatically be locked into that library.
This would violate one of our core principles, which is "Tools, not policies".
And given the state of TLS implementations, I /really/ dont want to make that choice, I want to leave that to the person responsible for the security at the site running Varnish.
Second, I think it is sound security engineering to confine the server certificate in a process which does that one thing and does it well.
I know several sites that have two different TLS frontends in front of Varnish, using different TLS implementations, so that whenever a CVE hits one of them, they can just shut that half down until the issue is fixed, and still remain in production. That would not be possible if TLS was bolted into Varnish.
I agree. I think varnish remained popular for 2 reasons:
- It was, at one time, higher performing than an nginx proxy cache. That doesn't seem to be the case anymore.
- VCL is very rich and flexible with primitives for reading/writing cookie contents, cache metadata, cache flushing, client and server state, and so on.
So probably the only reason left if that you're doing something really complicated with your cache in VCL.
Otherwise, as you say, the built in HTTPS of nginx, haproxy, nuster, or something else is a big advantage.
My personal estimate is that around 20% of all HTTP traffic pass through a Varnish instance somewhere, many of which are not public-facing.
As to why people use Vanish, I think the main reason is that VCL gives you full and instant control over your HTTP traffic, and people really like that, because it lets unify heterogenous sites and pamper legacy systems as necessary.
There are advantages of using HTTP cache on application side even if you use CDN:
- Varnish can guarantee that a given resource will be fetched at most once every TTL expiration since you can set up one instance to do so (+ some HA solution like hot standby). This complements a large CDN that is highly scalable but at a cost that there is no hard "synchronization" between servers. You will see many requests for the same resource, even when the CDN uses cache hierarchy because they can't risk such bottleneck not knowing in advance your traffic pattern. You can intentionally do that with Varnish and benefit from that.
- Complex caching rules, even including what to do if application is not available/slow. That includes serving stale content, how long to serve stale after TTL expires, guarantee that fetching a new version is performed in background. Example: https://varnish-cache.org/docs/trunk/users-guide/vcl-grace.h...
We've used varnish + some other magic in addition to Cloudflare to withstand Black Friday when marketing insisted the promo has to start at specified time, anuonced much earlier to all customers. The landing page had up-to date info on available products updated almost instantly, thanks to serving stale (in practice < 1s) content and guaranteed prefresh in background. Cache key was properly set up to exclude tracking elements from URLs (from FB, adwords etc) and only include what is necessary.
One thing I've heard a lot is "We know we can always fix it in Varnish".
By this people mean things like printing the wrong URL in a full-page sunday newspaper add or splitting traffic based on first digit of customer-number and stuff like that.
Some of the screwups Varnish-ops have fixed are truly the stuff of legends, but telling them belongs to their heros :-)
Also CDNs became cheaper and easily accessible. They don’t eat anywhere near 100% of your traffic for most sites but it’s likely enough to get a big chunk of the cacheable traffic at lower latency than your own Varnish server can unless you’re running them all over like Fastly. If a significant fraction of the traffic coming back isn’t cacheable the benefits of running Varnish might sink below the level where it’s worth dealing with another layer of production infrastructure.
Adding my voice to this. I know that the article was written in 2006, but still. If one wants to be taken seriously as a writer, one shouldn't insert puerile jokes for 12 year olds into their prose. It made me wince when when I read it, and I was ready to comment here about it, but GP got here first.
Generally, the accusation of "sexist" is used when someone is making unfair or irrelevant assumptions on the basis of sex. It is not generally leveled at technical documentation that assumes the reader has the technical background to read the documentation, for instance.
Technical documentation sometimes does contain humor, it's not that uncommon for developers to include jokes and easter eggs. Furthermore, this is listed under category "Poul-Hennings random outbursts", described as "You may or may not want to know what Poul-Henning thinks.". So while technically included in it, it's not really a documentation of anything. It's basically a blog post. And just like I don't find anything wrong about a blog about cosmetics or fashion assuming that the reader is a woman, I don't find anything wrong in a blog post about technology assuming that the reader is a man.
I propose the article branches at this point as a set of selectable tabs, each labeled over a suitable set of properties describing the reader, and then offering a matching joke when you click the tab.
Doesn't that still have the problem of categorizing the reader, being wildly off-topic from the technical content of the article and distracting the reader by making them wonder, at least a bit, if the technical content of the article is going to be different for them depending on who they are?
The joke barely makes sense in context. Is the physical size of the memory relevant to the point about memory-mapped views of disk - and is it a problem that the physical size of memory is smaller? Should one program an MP3 player the way described in this article, or not? What about USB flash drives (which one is likely to have in one's pocket, and which are roughly the same form factor as the other object alluded to here) - they're much slower than main disk and more likely to fail. Is this an intended reference to such memory devices, and in either case, how does the advice from the article apply to such devices?
The article would be better (more effective at communicating its point) without it.
Arenas are great for latency sensitive programming. But some of the advice in this piece is a bit limited in that it assumes the presence of swap (often disabled on server hardware) and programs whose important "hot" datasets are likely to fit in memory (i.e. the memory use is nice and predictable--an enviable but not always achievable property).
When working with bursty memory demands and code/data you don't always fully control (think a big thick runtime with its own allocation decisions, some of them decidedly "1975") on a system whose swapping behavior you can't predict, hand managing on-disk versus in-memory data residency in userland (via manual memory mapping or something like sqlite/lmdb to take care of it for you) becomes necessary.
TLDR; Varnish is very proud they use memory-mapped files for cache, instead of managing RAM and disk cache separately.
That said, don't overestimate the OS ability to understand your usage of RAM. The entity with highest insight on that usage is the application.
Maybe there's some fortunate overlap in the case of a caching service (Varnish) using a caching service (OS disk cache / virtual memory). Varnish itself has little clue how their entries will be used beyond what the OS sees.
But in more complex applications that's decidedly not the case. Photoshop for example to this day uses a "scratch disk" separate from virtual memory. In fact you better hope you never use swap file with Photoshop, because it becomes unusable.
Thanks for making this point. If you access your memory sequentially, as Varnish does, great. But if you need to stride across a giant matrix in transposed order, you'll bring your spinning disk backed virtual memory to a halt, as it does a million seeks. To use virtual memory successfully you have to write in-memory algorithms that recognize the non-random-access nature of the memory. If you had simply acknowledged there's a storage system, you wouldn't put out much more effort, if any, to get the same performance. In fact, by pretending it's all RAM, you give up the opportunity to overlap your I/O and do something useful with the time the system would have to spend paging in your data.
Edit: btw, happy and satisfied Varnish user for many years!
Sure, but the point remains: If you fight the operating system you WILL lose. Varnish doesn’t solve the same problem as Squid does, the author have said as much in talks when Varnish was first released. Still that doesn’t excuse an outdated programming model in Squid.
We actually do have a persistent `stevedore` in Varnish Cache, but we have marked it "deprecated" because it did not live up to expectations, but we keep it around to keep the internal API's honest.
Redesigning that code has not bubbled up to the top of my TODO list yet, and shows now signs of doing so.
But if you need persistence, there is a commercial version of Varnish which has it, but I have not been involved in that.
Since the moderators seem to have collapsed the subthread about it (but not flagged it, so I can't vouch for it), I'd just like to say that the random puerile joke in this article detracts from its technical merit.
I'd argue that it's actually more complex now. I programmed in the early 1980s, and did have to worry about where my data physically resided inside the machine, but I never had to worry about whether it was in a different room on a different machine, or sitting on a hard drive in some Amazon Data Center a continent away.
As others here note, the idea of using the same abstraction for all storage is at least as old as MMUs and Multics (mid 1960s). What is different is that programmers in the 60s and (early) 70s were usually coding pretty close to the bare metal. Nowadays, Moore's law has allowed us sufficient power to permit programming on top of a big pile of abstractions that try very hard to hide the actual hardware from us.
That's a luxury afforded by the sheer power of what we're working with, but the people writing those abstraction layers still have to pay attention to the layer beneath them, and if you go down far enough, you'll find some code that needs to pay attention to what class of memory something is stored in and how to optimally move it around. It's just that work was probably done for you by someone else who wrote your operating system.
Just because your programming language doesn't require you to use pointers doesn't meant that indirection isn't being used. You just don't have to deal with it (until it rears up and reminds you it's still there). Joel Spolsky's Law of Leaky abstractions (https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...) is relevant here.