Does anyone have any understanding of how they do this?
My knowledge of how these models work is basically that they are a black box that you put text into and get text out of. I don't phrase it this way to diminish their capability, but more to ask how, other than using a technique like stenography, are they able to hide their true chain of thought in a recoverable way?
Welch Labs on YouTube has a great collection of videos on how AI models learn. His recent video [1] covers how image models can learn to encode reasoning in the image processing layers when not given an out of band reasoning set of weights to use instead. I suspect that this applies to LLMs and CoT reasoning vs output token weights.
Tao is saying that there is very little insight from something like an LLM counterexample (e.g. Jacobian conjecture counterexample he investigated further on his blog) - you don't learn much about the subject and _why_ a conjecture was true or false from an LLM giving a counterexample. That's why he wrote the blog post - to analyse what the counterexample says about the subject.
Tao does not disbelieve the counterexample (it's seemingly easy enough for him to verify it is a counterexample).
Parent is saying something very different - they're saying they literally don't have any faith that this is a proof. Given its size, it could just be a bunch of completely useless statements that do pass the type checker.
You're putting a lot of words in my mouth. What I'm saying is that whether or not it's a proof, it's useless: it does not improve human knowledge, because the only thing able to consume 10MB of Lean to build upon it is another LLM that's going to build a 50MB piece of shit.
It's very much likely a proof. It's also completely useless.
If we accept that it is a proof then it does improve human knowledge, even if no one can understand how to get there.
If you were navigating a pitch dark cave, wouldn't you find it useful to be able to see the light of the cave opening even if it's not bright enough to illuminate your path to it?
To the credit of the original commenter, that is why they said "give it one or two years". _Right now_ we need the experts to formalize/check. They're saying they think LLMs will reach a point in the near future where that won't be necessary.
All we need experts for right now is verifying the formalization of the statement of the problem is correct. The proof itself, that formalization is checked automatically.
It doesn't mean preposterous or not preposterous. It does mean there is general dispute regarding the proscription.
Appealing to it is not really progressing your argument. Both decisions are controversial, you don't make a more convincing argument as to why one controversial decision should not be considered controversial, by pointing to a different, also controversial, decision.
That's a deeply flawed reading of what I've written. For example, you've disagreed with me in a way that leads me to believe you're more than a little bit dense, but not a terrorist.
1 mil is not generational wealth in the US. It is a big chunk of money, no doubt. It’ll buy a reasonable 2-3 bedroom house in the city I live in, with nothing left over. No one’s definition of generational wealth involves not touching the money for two generations…
We must be operating on different definitions. Generational wealth means you have enough money to meaningfully improve the lives of your kids and give them a leg-up on life. A million dollars is enough to buy a house for each of your 2.5 kids.
If getting a million dollars wouldn't affect how much money you can leave your kids, you already have generational wealth.
> If getting a million dollars wouldn't affect how much money you can leave your kids, you already have generational wealth.
Well said.
Where I live, one million dollars would allow me to pay off my house, open healthily sized investment accounts for my kids, pad my investment account, setup a trust and, overall, set my family up for a comfortable life in the future. I don't see how that isn't generational wealth.
Generational wealth is where you can also do all of the above for your kids and potentially their grandkids as well.
Basically the bar is higher than "something you can pass down". It is enough that the next generation does not need to worry about making money either.
Generational wealth generally is used to mean something like you and at least your children can live very comfortably off of investment income for the rest of your lives, I.e. none of you have to work for a living. That’s why sibling’s def of 1.5-5m per child is much closer to the commonly understood meaning.
We can debate “live comfortably” if you want, but no 4.5 people are doing that from the investment proceeds of 1m.
What you are describing is kind of more like social mobility.
> If getting a million dollars wouldn't affect how much money you can leave your kids, you already have generational wealth.
Generational wealth is definitely not "affect[ing] how much money you can leave your kids." That's an equivocation - if you're leaving your kids a dollar, another dollar will "affect how much money you can leave your kids."
edit: you need a million dollars to securely retire at all, and that's if your parents, kids, or you don't get sick. If they do, a million is not only not "generational wealth" but it may not even last you three years.
Generational wealth is generally used to describe not just a comfortable personal retirement but your children and their children and so on never needing to work if they are remotely responsible with the money.
I have to agree, yurishimo's assumption of returns is wildly optimistic.
At a more sane expected return of 5% annually, you get $50k a year to live off of to just keep what you have (or rather, watch it slowly erode in value due to inflation).
That is basically a one person income, maybe two adults if you pinch pennies and live in a crappy apartment or a low-end house in a midwestern suburb.
You can get a lot more lucky with a million bucks than you can with 10k if you gamble, but there are no guarantees. Risk tolerance is the most impactful variable. For those in the low risk tolerance group, I think you'd need at least 2 mil these days. And that's with being frugal, as well as probably not having much left over for kids.
Cannot pursue your vocation in certain jurisdictions. And there are plenty of adjacent fields I suspect wouldn't be covered by the NC - like education.
> Cannot pursue your vocation in certain jurisdictions. And there are plenty of adjacent fields I suspect wouldn't be covered by the NC - like education.
I doubt the PE firm informed him of that, and I'd bet they made their NC seem bulletproof when they talked to him.
sure there are always options. but they may not be visible or accessible. maybe they can not or don't want to move because of family. maybe there are no alternative jobs in the area, maybe they hate teaching. seriously, the only sane response to non-competes is to make them illegal. everything else is missing the point.
My knowledge of how these models work is basically that they are a black box that you put text into and get text out of. I don't phrase it this way to diminish their capability, but more to ask how, other than using a technique like stenography, are they able to hide their true chain of thought in a recoverable way?
reply