It's the other way around. He's framing America's decline in terms of Chinese success.
The Chinese rise is also remarkable in its own terms, but it's not the subject of discussion here. The subject here is how the US is under cutting its former preeminence in science.
We had a similar discussion about computers the other day. A lot of us miss the days of tiny memory and optimizing for that.
It takes all kinds and I see no reason to gate keep either way. It's too bad that we do t often segregate around those opinions and don't share as much
> It takes all kinds and I see no reason to gate keep either way.
Gatekeeping is a good way to describe it. This happens to a lot of niche hobbies that go mainstream. Some of the old ingroup that were into the hobby before it was cool get irrationally angry at the new waves of adoption because they're in it for different reasons.
I don't care why someone is in a hobby. If you want to do it the easy way, good for you. Nothing has been taken away from those who want to do it the hard way. It's much easier to get started building custom machines now than it was when I was starting out.
I don't remember those days with a ton of fondness. Yes, the challenge was fun, but I really wanted to ship it and get my product in the hands of customers. Now I can spend more time thinking about what they want and less time about what the computer wants.
The world turned millenia ago and will turn millenia later.
The problem statement is of applications using up expensive RAM. Incidentally, expensive RAM is just one of the problems we face in the computing space. Forced obsolescence is another, when running hardware needs to be replaced because software is built for only newer CPUs.
> The problem statement is of applications using up expensive RAM. Incidentally, expensive RAM is just one of the problems we face in the computing space.
You need to stop and think about the problem. Nowadays RAM is expensive because many people now want to max out their computers with RAM to run LLMs and AI coding assistants. Today's software is still the same software that ran perfectly well half a dozen years ago. Cloudflare happens to operate a large global computer infrastructure, and it's scale is such that 1% gains are lauded as fantastic cost savers. But that's the bean counter's perspective, pointing out that they saved a bean.
> Nowadays RAM is expensive because many people now want to max out their computers with RAM to run LLMs and AI coding assistants.
Source? I imagine the amount of people trying to run local LLMs is miniscule. RAM is expensive because a handful of companies have spent billions buying all of the compute.
> Today's software is still the same software that ran perfectly well half a dozen years ago.
This statement is quite honestly not true at all. Just take the two largest OS from 6 years ago and compare resource usage between them and you will find you are incorrect, never mind the software running on it.
You are right, but it reads (to me, could be just me) you mean newer specs which used to be true when I shipped software in the 70-90s; you mean faster machines / more memory right? Like installing a new version of software or OS and suddenly all memory is used, system is swapping and you did not ask for that but some obscure feature you didn't need needed to be shipped fast.
That statement can be used to justify anything, to the point it is utterly useless.
— Humanity has been on the decline. People are hateful towards each other, striking their fellow man and poisoning the environment. Despots eventually launched nukes which killed everyone but the cockroaches.
On the contrary. Industrial developing countries saw bigger % increases in their economic growth in the last couple of decades than the fully developed "old" western ones. Mostly due to "the rich" shipping western jobs there to cut costs and increase their margins.
For example the likes of Poland, China, India and Vietnam were and still are growing like crazy when you compare to the likes of UK or Germany.
> And its this obsession with shipping things as fast as possible quality be dammed (...)
You seem confused. Allocating more memory than optimal levels is not a measure of quality. Similarly, a web page is not suddenly lower quality if an image asset is 50kb instead of 25kb. And how much complexity and engineering effort and bugs are you willing to tolerate to halve your memory allocations?
You are conflating quality with mindless minimization, not even knowing or caring that are the tradeoffs. The blog post you're commenting on starts by presenting the case for celebrating small improvements, even 1% improvements at a time. A similar 1% improvement in a mobile app is at like 1MB. Do you ever notice it? How many hours of engineering effort are you hoping to spend on this nonsense? And you prefer to spend it on this or in actually fixing a bug or implementing a feature?
This puerile conflation of minimization with quality suggests your personal notion of quality has no bearing on what quality actually is.
> And how much complexity and engineering effort and bugs are you willing to tolerate to halve your memory allocations?
You seem to be confused about this relationship. It's usually exactly the opposite.
The wasteful applications are generally not well reasoned about and half assed implementations. That's why they're guzzling resources
There is ofc a middle ground, because targeting eg incredibly resource constrained embedded systems will naturally increase complexity, but that's something entirely different to the scenario this discussion was about up to this point.
Exactly, that`s why i refuse to call 99.9% of "software engineers" engineers. Imagine you optimize a aircraft turbine 1%....man you get to drink a pool of champagne with the highest ceo's and aircraft carriers try to buy that turbine as fast as possible.
But with software and a install base of some millions 1% optimization is seen as wasteful, its like software slop is acceptable since forever because hardware gets faster, and electricity is "green" anyway.
Resource constraints have historically been a bastion of quality engineering, though. The relationship is not directly causal, but any time an engineer must not only solve a problem but _fit it within some kind of a budget_ vs having an unlimited budget, it forces them to slow down and think more carefully about what they are producing and how well it actually functions.
Ultimately it is an expression of anti-fragility: some minimal challenge must be met and conquered on many different axes of a production in order to help the full production itself mature to the best possible quality.
It is the same reason we exercise, the same reason that cars and toilets and so many products greatly increase in quality after emissions and usage and waste-related regulations get applied from above, etc.
The appreciation of the refining force of outside limitations is not about min-maxing, but it is at least partly about curbing the min-maxing of other concerns such as "ship the fastest garbage possible to move on to the next opportunity to repeat that process".
Yes, I remember those too. The costs of manual memory management were real and were not low.
But costs on the cloud are real too, especially now. I’ve been living in JVM land for a very long time, but now it’s especially clear how important lean services are. Especially now that the bar for writing lean code is so much lower: let the borrow checker figure it out, etc.
I just spent a couple days wringing out more performance/memory efficiency for our services. Nice gains to be sure, but it’s still so immensely wasteful compared to something well written running native. If it was my money, I’d be going native for sure.
> But costs on the cloud are real too, especially now. I’ve been living in JVM land for a very long time, but now it’s especially clear how important lean services are. Especially now that the bar for writing lean code is so much lower: let the borrow checker figure it out, etc.
I don't think even Cloudflare bothers with this waste of time. If they did, they would certainly not have built their global infrastructure on JavaScript running on V8. They'd have done what Google and old-time Facebook did and built their whole infrastructure on low-level system languages, and hiring the world's leading minds on the subject to milk the last drop of performance from their hardware.
Even Google stopped to look at the problem and came up with Go. Not V8.
> The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt.
All those things happened before the era of decent AI.
Also, it really does matter what you're building. Yet another CRUD app? Don't waste time on (super) lean languages - use Go or something else you like and move on.
Building a Kafka replacement? Making something that processes gobs of data quickly? Perhaps think about using something lean and efficient, as the economics have changed. "waste of time" (or memory) also applies to the cloud and your (or your company's) bill. And if you haven't noticed, you're being ripped off on the cloud, running something 10-50x less efficient has real bottom line impact.
The vast majority of people do not care about memory efficiency on the levels we're talking about. I've never met a non-technical person who has cared. And out of the technical people that I know (which represent less than 5% of the population), no more than half of them have complained about the memory usage of an electron app.
Or, they care but they lack the vocabulary to express what they care about.
You can get a new laptop to run software A or competing software B.
You can pay $1000 for the laptop with 4GB of RAM, and software A either can't run on that or will act like pulling teeth, so that software requires you to upgrade to 16GB of RAM which will put you back another $2000.
Or you can run software B which will hum along perfectly smoothly on the original $1000 laptop.
(All of that of course simplifying right past "everyone runs every app 24/7 with five trillion browser tabs open")
Are you honestly suggesting that the end user doesn't care about having to throw exponentially more money at stuffing RAM into their PC just to make the game all their friends are playing online function?
> Or, they care but they lack the vocabulary to express what they care about.
That's speculative to the point of being conspiratorial.
> Are you honestly suggesting that the end user doesn't care about having to throw exponentially more money at stuffing RAM into their PC just to make the game all their friends are playing online function?
I'm extremely clearly not suggesting that. Don't make up things and pretend that other people said them. Engage honestly or not at all.
Customers often want to pay less, and shareholders often want lower capital costs. A tasteful optimization is a win-win. The opportunity cost should be traded off against new features of course, but tasteful optimization is a good thing for customers.
The fact that it came down to a governor to set these standards means it's going to be subject to even more abuse.
It was up to Flock to figure these things out before they deployed it. Instead we've got fifty separate states coming up with guesses as to the needed level of control, and each of them surely forgetting something. That's what happens when you roll your own security.
We now will have 50 separate state police forces looking for ways around the restrictions. And they will find them, because they are far more flexible in finding holes than governments are at patching them.
"Falsifiability" is generally understood in terms of physical observations. Mathematics is ultimately tautological: they are true or false by their own definition, without reference to the physical world.
It just so happens that certain kinds of mathematics are unreasonably effective in drawing parallels to the physical world, but as far as mathematicians are concerned those mathematics are neither better nor worse than those that do not correspond to anything tangible.
> those mathematics are neither better nor worse than those that do not correspond to anything tangible.
A sweeping generalization. I certainly know professional mathematicians who disagree.
Also: many consider “inter-connectedness”, not “tangible” to be a sign that a topic is interesting. That is, it touches branches of mathematics aside from its own.
Usually, months to years. The economy is a very big ship and slow to steer.
This particular crisis is quite abrupt, caused by a sharp jump in the price of oil. Most presidents don't really deserve either the credit or fault that they receive on the economy, but in this case there's a very clear and direct connection.
I didn't realize that open math problems were a finite resource.
I recall a story about some famous mathematician (Gauss?) dismissing interest in Fermat's Last Theorem claiming that he could crank out problems of equivalent interest.
Clearly Tao knows a hell of a lot more than I do about this, but I'm surprised that math that close to completion.
They aren't, but the problem is that open problems tend to emerge when people are working on other problems. If fewer people are spending time deeply thinking about current problems since a handful of labs are solving them with AI without an eye towards understanding and only on verification, the pool of open problems won't be continuously growing. There is a fear that there will be a chilling effect on the community if people are disincentivized from trying to solve deep problems or study them for understanding as opposed to simply focusing on verification. It's more of a social and community problem than a fundamental problem with mathematics itself becoming "completed".
So we can let the ai generate some math problems based on the solutions found? Other fields (computer science, physics, ...) can generate math problems too.
There's an infinite number of possible math problems, but the things that make these open problems worthwhile is they're interesting to people who have worked in related areas.
They're good to give to new mathematicians, and they're good to help humans understand the shape of the problem space and relative difficulty with the tools we have.
Cheesing these problems with LLMs gets rid of both the training benefit and our ability to create good related problems. There's an aesthetic part of this, too, that LLMs do not capture.
This kinda reminds me of the guys who decided to industrialize digging up dinosaur fossils, in order to feed the dinosaur fossil collector market. They were amazed that paleontologists were so "inefficient" at finding and digging up dinosaur fossils.
But from paleontologists' perspective, they go out looking for dinosaur fossils when they have questions that digging up a fossil may answer. The metric they're focusing on isn't tons of fossil mined out of the ground, it's a developing understanding of extinct life.
These open problem solutions often reveal tighter bounds on prior conjectures. Even if the solutions produced are far from elegant and only machine verifiable, we do learn new information. But I agree that just like writing prose and code, brainstorming frontier math proofs is a perishable skill
It's easy to come up with new open problems. It's hard to come up with new open problems that seem to teach us something fundamentally new about the world. Our current batch of problems went through a complex selection process over decades (or centuries) based not purely on difficulty but also on perceived insightfulness.
I studied math, but I am not a mathematician, so I think I have a slightly different perspective on this than Tao overall. This is certainly the definitive end of an era in mathematics, but I think he's wrong that insightful new open problems are truly non-renewable. They might be non-renewable by humans at the rate at which they are being closed, but I see no reason why AI systems could not also discover insightful new open problems. In fact, once we have Riemann-capable AI mathematicians, I'd personally love to see what the next Riemann hypothesis is, which even these AI systems cannot solve with any amount of available compute.
I think we're about to find that, on the spectrum of mathematical intelligence, the best human mathematicians were only a fraction of a percent forward from the very beginning, and there's a vast universe of mathematical depth that's beyond our ability to imagine or work on directly in any way. We're used to feeling like we're able to directly perceive the Platonic realm, but we're almost certainly going to discover that our own minds, even when joined together over centuries of deliberation, can only interact with a tiny little shadow within it.
I haven’t been following the AI proof stuff very closely, but the impression I got was that these models are producing massive Lean programs that prove the statement one way or another, but are quite difficult to fully understand.
Actually, I have to admit I don’t really know what math is. With physics we suspect there’s a universe, and when we study physics we’re improving our description of the behavior of that universe, right? The universe exists whether or not we know how it works.
Eventually, as you suggest, maybe we’ll hit math that won’t fit in anybody’s head at all. What is the nature of mathematics that doesn’t fit in any human’s head? Does it even exist in some sense?
I think math is compressible structure. That's why we care about something like the Riemann hypothesis but, to use Tao's example, we really couldn't care less about computing the 10^10^10th digit of pi. The first compresses a vast amount of information about the primes, while the second decompresses information that we've already compressed (a few lines of code can define every digit of pi).
Most patterns that exist are incompressible. Math is basically a search for those compressions that do exist. An example I personally really like is the amplituhedron: a geometric structure that humans have just barely been capable of recognizing compresses information about scattering amplitudes and Feynman diagrams. That one happens to be within our reach, but it's right at the edge, and we can only imagine what glorious, wondrous compressions exist in abundance beyond the edge. Math accessible only to superintelligence would exist entirely beyond that edge, compressing patterns whose existence we cannot even detect using objects and constructions that we cannot grasp.
As an aside, I also think this is why AI is quickly becoming superhuman at math: intelligence is essentially a form of pattern compression.
I think part of mathematics is taking things that don't fit in our head and giving them human abstractions so they can.
Take infinity. Infinity can't fit in your head, hell, it can't fit anywhere, but you can abstract away the endlessness and look at infinities of different sizes, et al.
Now, is there a single formula for something actually represented in this world that would take most of a humans life just to read it, no idea.
The models produce both Lean code for formal verification and a traditional-style narrative proof. Like the general long-form output of frontier models, the math papers produced appear to be generally correct technically, but written in an ungraceful and sometimes hard-to-follow style, so they are often polished by a human mathematician as of today.
What you are saying implies that by some technique that hasn't been discovered yet, we can make the models to have the capabilities of extrapolate the information they are trained on and also interpret that what they are extrapolating are Riemann-capable hypothesis.
I do believe it will accelerate the discovery of that "vast universe of mathematical depth that's beyond our ability" but at the cost of removing the "fun part" of solving the problems. Not sure if the community is willing to do that.
I'm surprised nobody has stated the obvious: a hard math problem that has been open for ten years (because many serious people have given it serious thought and been unable to make significant progress) is, in fact, nonrenewable.
The only way to renew it is to make a new problem that is so hard systems and humans will be unable to solve it for the next ten years. And, in the spirit of trees, the best time to plant a tree is twenty years ago, the next best is today: we do need to start posing some hard math problems and deciding if they are interesting merely because there are challenging or because of something else (eg busy beaver problems are arbitrarily hard, but does solving them imply anything other than "another busy beaver problem was solved"?)
Eh, if AI quickly solves most of our mathematics problems that are solvable then it might be time for us to hang up our hat as our little monkey brains aren't very good at this stuff.
Now, I think AI will solve some, but we'll find out that some are just either unsolvable or wildly huge that nothing is solving them any time soon.
And a whole lot of these problems have been around quite some time, when even knowing how to do advanced math meant you were a landed gentry or someone of high wealth. If those problems fall, they fall. They aren't pets we keep around forever. And new problems will crop up over time for both AI and men to scratch their brains over.
I think "close to completion" is not the right framing. Creating good open problems was an achievement because these problems often sit at the edge of known techniques, and solutions require inventing "new math". It's hard to find these problems, and they take decades to mature as they withstand scrutiny by many people.
In another comment below, I likened this to clear-cutting a forest. Growing the forest takes a lifetime; destroying it could happen in the next few months.
his whole point is that specifically problems that have been held as important by consensus in the field are a finite resource. obvious example being the Clay millennium prize problems. seems like they function to shape the direction of future research into useful directions. which is to say, the process of developing a solution itself generates more useful problems.
of course thrrr are tons of problems once you remove this social consensus based filter. if i’m not mistaken Ramanujan left a book of dozens of unproven theorems, for one quick example. i don’t think that that has opened up dozens of fields of mathematical research.
> famous mathematician (Gauss?) dismissing interest in Fermat's Last Theorem claiming that he could crank out problems of equivalent interest.
Web search turns up Gauss's comment, with a bit more nuance:
"I confess that Fermat's Theorem as an isolated proposition has very little interest for me, because I could easily lay down a multitude of such propositions, which one could neither prove nor dispose of." (https://mathshistory.st-andrews.ac.uk/Biographies/Gauss/quot...)
Deforestation might be a better metaphor than mining. Logging is renewable if for each tree you chop down you plant several more. AI companies are operating "in a non-renewable fashion" by chopping down trees without planing seeds. Open problems are a renewable resource, but only if harvested sustainably.
Relevant, interesting problems that we have some immediate hope of making genuine work on might be, if not finite, quite difficult to produce. And it's also plausible that AI will not do as good a job of producing these as it does at solving them.
The other problem that Tao identifies is that math has typically been an unusually open subject in many respects. This openness may not work if big AI labs can afford to throw $X million at a problem to scoop you if the rumor gets around that you think you have something promising. Hence, less collaboration, and less chance of identifying these exciting new problems, infinite though they may be.
I think you can't have read the thread.
The whole point is that there is no end of mathematics, an infinite sea; but the constitution of an 'open math problem' is a delicate piece of mathematical thought, at any moment a small supply of drinking water developed by finitely many human being.
What worked was getting people stirred up to the point that some would turn to violence. Even now a majority of their party believe they were justified.
That helped them take control of politics. And so that tactic will surely turn people to violence again. The violence itself may not be effective but the level of outrage is.
If the extremes are unelectable, the center party should split eventually. They'll find something to disagree about.
As long as they are more terrified of losing to the extremists they'll try to keep it together as long as they can. I hope they don't become too corrupt in the process.
The Chinese rise is also remarkable in its own terms, but it's not the subject of discussion here. The subject here is how the US is under cutting its former preeminence in science.
reply