More benchmark noise. They’re already problematic per se, and the error bars only get bigger by aggregating and adding costs on top of it.
Create your own, private evaluation system that reflects your use and your constraints. Focus on hard cases you encountered, find cases that break on one tier and not on another.
Exactly, according to these benchmarks Gemini 3.8 Flash is the best option at that budget level for coding, but in real world testing, I found DeepSeek flash to perform far better at a much lower price.
The tech is very cool but for the love of Gaia stop calling it system one, even Kahneman said this (S1/2) is a framework for understanding the brains inner workings. There is no autonomous system to speak of.
Ok sorry for the unprompted rant but you're absolutely right, though it's far from "new" and not really a problem (IMHO). AI even beat neuroscience to this one, funnily enough! All of the below dyads are congruent:
--- Phil & [Pre-]CogSci ---
| rational | intuitive |
| reason | understanding | (Kant & Hegel's goofy wording)
| deductive | inductive |
| animated | automatic | (from ~Aristotle, ultimately)
| ~higher | ~lower | (as in "higher faculties")
--- CS & AI ---
| logical | analogical |
| symbolic | stochastic |
| Neats | Scruffies | (related academic 'camps')
| ~deterministic | ~non-determnistic | (a common heuristic)
| Good Ol' Fashioned AI (GOFAI) | Evil Datacenter AI (EDAI) | (a common honorific)
--- Modern [Cog-]Neuro[-Psych] ---
| slow thinking | fast thinking |
| S2 | S1 |
...ok I assumed I knew more but maybe I don't? Would love to hear from experts!
--- Colloquial ---
| left brain | right brain |
| intelligence | wisdom |
| intention | instinct |
| me | AD[H]D |
At least that's how I see it. Hopefully it goes without saying that there's plenty of nuance specific to each of those pairs, and that many of the relevant experts would balk at this broad characterization. That said... I'm right and they're wrong, I guess!
Brains aside, it S1/S2 seems like a good description?
System 1 is quick and intuitive. System 2 is slow and deliberate.
Compared to models that compose responses with many forward passes, thinking in token space, these models make a quick decision after processing their inputs.
It’s ok to use it as inspiration, but using the term comes with psychological baggage in an industry that already abuses anthropomorphism. Give it an informative name
System 1 is quite informative about what it's for. It means it's a system designed for bounded time quick classification, vs. agentic unbounded or slow thinking with scratchpads and tools.
Does it inject thoughts into system 2 without system 2 knowing about it? Does it have instinctive social skills? Can it do instinctive arithmetic? The list goes on, there’s a nice book about it…
I'm not sure about the salaries. Vocation-motivated people are already doing it. The rest needs a good reason to put their entire life, quite literally, into one single purpose. It's one of those professions that requires a complete dedication of free time, youth, health, mind, etc.
Nope. There is effectively infinite demand for physician services. If we had more doctors that wouldn't reduce the workload on individual doctors. The actual solution to reducing demands on their time would be through reducing the administrative workload which contributes nothing to patient care. Much of that could be automated or delegated or not done at all.
The time demands also differ widely between specialties. You can't really be a part-time neurosurgeon. But we see a lot of doctors in other specialties like pediatrics, internal medicine, radiology, and emergency care downshifting to part-time schedules after they have established careers.
If you look across countries, you realize that a lot of the sacrifices are part of the pipeline, but not necessarily key to good outcomes. Just like the grueling hours at certain residencies.
It's like claiming that one needs to cram hours upon hours of leetcode practice to be a quality software developer. It might be necessary for some companies to hire you, but that's to meet a filter that is less and less predictive over time, not an actual performance requirement.
Education system was set up in XIX century. It's not clear how much of it is necessary in XXI century.
Back in the day religious books were copied by scribes educated in a monastic tradition. Now printers can print them in a completely godless manner but the result isn't any worse.
I remember some friends working in intensive care during the pandemic, I'll never forget their facial expressions after these 24hr shifts... I really wish someone would come up with a better way to do this
And that's the main reason why lock downs extended so far later than they should have. Our hospitals run at 120% during normal times; during abnormal times they're stretched to the breaking point. Our small local hospital had two nurse suicides during the pandemic. Not early in the pandemic, but well after vaccines were available.
Our societies should have expanded their healthcare capacity afterwards and maintained it to a high standard, but unfortunately it sounds like the political classes continue to treat healthcare as a cost centre that they can starve or pull money from for other projects.
OpenAI has hit $1bil in annualized revenue since launching ads in February. AI search optimization is already a thing. I don't see much of a reason to assume they aren't already getting served information coming from what's essentially advertisements/sponsored content.
reply