Pangram also shows 100% AI generated. Its very likely that the technical details have some correctness, but you could also just ask the LLM of your choice for an analysis of where Rust is and get similar quality. There doesn't appear to be any interesting technical moral or anything.
Also I am loving just how over the top clickbaity Claudeish is for an article like this. I just imagine it as a dramatic narrator announcing these section titles:
5. The one announced as accessibility, not as Rust: Narrator’s maths engine
9. The compiler that changed name
FYI a quick fix for claudish is to ask for the response to be in ASD-STE100 (Simple Technical English). Then it is far more readable. But I would agree that this is an annoyance and shouldn't require user workaround to get something readable.
I think this is more a meme than anything else, for a couple reasons:
First, after a while it's just as grating as Claudeish.
Second, my hunch is that it constricts the actual thinking of the LLM, like the same way that Newspeak does in 1984. It shrinks the range of thought that can be expressed if used as an input.
I think the real way to do it is to have another Claude entirely deal with the user as a liaison, but to keep the thinking in whatever format it came in.
Latent space reasoning, if you think about it, is exactly this to a crazy degree: why even formulate a thought as words if you can just keep it as matmuls until the user needs it? And then, if the user needs it, have it always specifically formulated for the user by another LLM rather than constrict its range of thought? Anyway, that's my take.
The fact that it might affect quality is something that is on my mind. In fact, I have mainly been using Astra after being primarily on Claude for several years exactly because of this annoyance. Ultimately, the model (at least for coding purposes) should just... not have this writing style.
I do think an infrastructure where another Claude retranslates the output would be better. Oftentimes I forget to put it in the actual prompt and when I receive back 8 paragraphs of Claudeish I ask for it then.
I would have to disagree that it gets as grating as Claudeish though. Its just direct and professional instead of ring-around-the-rosy clickbait.
The Claudeish drove me back to OpenAI and Codex as well. I find the Claudeish can be improved but it’s impossible to fully mitigate. Opus is unusable for anything I’m going to read.
“I would have to disagree that it gets as grating as Claudeish though.”
It’s hard to imagine anything more grating than Claudeish. To quote Rainer Wolfcastle, "My eyes! The goggles do nothing!"
> my hunch is that it constricts the actual thinking of the LLM
I've found that prompting any constraint on output (length, style, vocab, even simple formatting) not only places additional cognitive load on the model, which burns some of whatever cognitive budget is available, it will also often skew the output in other subtle and completely unrelated ways.
Since I found this artifact interesting, I did some pretty extensive experiments a couple months ago. The increased load is real, although it may not be apparent if you're not near any cognitive boundaries. The subtle skew, however, seems nearly ever-present regardless of load.
When I noticed web chat LLMs wouldn't number section headings correctly and consistently, I began experimenting with modifying the user prompt over a period of weeks. My usage at that time was research and learning not coding. In long, detailed sessions with branching sub-topics and deep follow-ups, I found it helpful if the LLM would number each major section and letter each sub-section in replies. Simple markdown formatting that most web chat LLMs do sometimes, but not consistently nor uniformly.
While extensive, my tests were just following my curiousity, not controlled, exhaustive or well-documented. I identified about a dozen prior sessions of varying length and complexity to test and downloaded them with a browser add-on. I then removed all other user prompt instructions except for the formatting instruction. A test would typically involve changing the wording of the formatting instruction ranging from brutally simple to detailed and complete, then starting a new session, seeding one of the test sessions and continuing it. To get a feel for baseline inter-session variation, I also tried running the exact same prompt/session multiple times back-to-back, at different times and on different days of the week.
Once I identified a promising prompt candidate, I'd make it the formatting instruction in my regular, daily-use prompt for a few days. I quickly got a feel for how seemingly minor user prompt variations impact response quality, compliance and tone across fresh sessions as well as those in various states of context rot, drift, decay and cliff (<--my nicknames for the distinct flavors of session degradation, not technical terms).
My overall conclusion was that every instruction, no matter how minor or unrelated it seems, has some, real impact on the model's cog load, attentional focus and/or attentional weight budget. Both how these impacts manifest and what causes more or less impact is often extremely counteriintuitive. To more fully understand this, I eventually, got to the point of testing null case variants, such as the entire user prompt being one sentence completely unrelated to text formatting or the session topic, like: "Don't reference the cartoon character SnagglePuss" (in a deep dive on ancient Sumerian clay tokens). Similarly, a simple one sentence prompt requesting something the model already always does naturally also has a cost (eg "Capitalize proper nouns"). As others have observed, heavy emphasis, absolute prohibitions or emotional weight in prompts also tend to have outsized impact in both skew (impacting unrelated output tone/style) and in accelerating session degradation. "Avoid referencing SnagglePuss when you can" would have equal compliance but fewer downside impacts than "NEVER reference the cartoon character SnagglePuss" in sessions starting to degrade.
There were also surprises, such as when I was scanning transcripts of an older, longer session and noticed the LLM was doing number formatting almost perfectly. On looking at the active user prompt at the time (I keep a log of every user prompt change I make for every model), it didn't even reference formatting at all. More experimentation showed it a result of the LLM gradually mirroring my consistent use of formatting structure in my prompts over a long session (in which I never mentioned anything about formatting). Unfortunately, that mirrored trait doesn't persist to new sessions and reaching that point requires a substantial number of rounds burning quite a bit of context window.
After spending time surfacing the impacts of just changing lightweight user prompts so they could be observed (which are the lowest priority prompts a model gets), I now wonder just how much more 'brilliant' the models we use daily would be if they didn't have dozens of pages high-priority manufacturer prohibition prompts we never even see weighing them down. We've only ever seen these frontier 'racehorses' when they're already pulling a heavy invisible wagon.
This is the same reason why I am a bit skeptical of LLM superintelligence. LLMs in the end have to operate in natural language concepts and the complexity of natural language is bounded by limits of human cognition. I'm sure super advanced AI could use concepts that humans not only have no words for, but might not be able to understand alltogether. As such if my thesis is correct, the only way forward for true superintelligence may be getting rid of natural language COTs.
> LLMs in the end have to operate in natural language concepts and the complexity of natural language is bounded by limits of human cognition.
I don't think this is true.
They have to express themselves as tokens. The meaning of those tokens doesn't have to be text. See any model that can handle images/video. Also, I don't think math, svg, etc, are "natural" language.
And, only the final expression is tokens. The intermediate layers, with the encoded concepts, aren't "natural language".
But, to address your concern (which nobody can disagree with, since even humans can't fully express through text/pictures), potentially: https://news.ycombinator.com/item?id=49758615
> The model isn't limited to concepts that can be expressed in natural language.
How do we know for sure? We don't even know how the emergent properties we see actually emerged?
For humans we know for sure that people sometimes have concepts that they have no word for (the reason the phrase "It's on the tip of my tongue" is a phrase, after all).
We don't know this for LLMs. When it makes new phrases, it's always a mixup of two existing words hyphenated (aside, that also seems to be the limits of SOTA models creativity - join two unrelated words together with a hyphen).
LLMs never respond with "It's on the tip of my tongue" type responses, indicating it has a concept but cannot remember (or does not have) a word for that concept. Every human, pre-speech-age, has managed to express or convey concepts that they had no word for.
So, no. I'd need a citation, preferably multiple, that did the trials and found that a model can generate concepts for which it does not have any words for.
1. Is natural language holding LLMs back by some %?
2. Is natural language serving as a hard gate that will prevent LLM intelligent progressing past some specific point?
The answer to 1 seems like an obvious yes to me.
Your thesis says the answer to 2 is "yes." That doesn't feel right to me. Think about all of the humans who have pushed various fields forward: Einstein, Newtown, Bach, whoever. If natural language doesn't prevent an entity from surpassing humans in one intellectual field, why would it prevent an entity from surpassing humans in all intellectual fields?
(To be clear, I'm not claiming superintelligence will or won't be achieved; I'm considering your specific thesis about whether or not natural language will be a hard gate)
Each letter has it's own token. LLMs can invent new words if they want to, just like humans do. All the fancy words we use today in science did not exist even 100 years ago. And that didn't stop us.
A few weeks ago two "Claude wrapper" projects showed up here within 24 hours of each other. Both seemed to just sand the details away--useful if Claude is too wordy, I suppose, but seemed to also lose real substance.
I agree there is probably some impact on output quality when doing this (Claude seems to embedded key signals all over the writing, which is much or what makes it horrible to read). However this does work well as a "second pass" over technical docs or designs along the way
Ask for output to the user in "simple technical English", let it think how it wants. It is hard to believe all the flowery prose is actually helpful to the model.
I don’t think system prompts are particularly reliable way to do much at all. It’s better to put it as a hook after each response, or a skill at least so you can trigger it at will if you don’t want it everytime.
Anthropic has probably RL’d the system prompt into nothing because of their fear of the user being able to control the model. If it listened to you about the slop language, it might listen to you if you asked it to help you with no-no tasks.
Your best bet is to use hooks and inject it after every file edit / response by first running the content through haiku and asking if it is asd 100 ste.
It burns more tokens but is the only way to get tolerable text.
The hook sends the text to another agent/context with a request to validate and return a good or bad + reason response. Every request is a fresh context.
I hadn’t heard of ASD-STE100 until last month when the TinyGo AGENTS.md added it as a requirement, along with some other tidbits to improve life for maintainers.
It’s been really productive and I’ve been asking my agents to communicate using it more and more. I believe it’s relieved my cognitive load a bit while working with them.
I created a custom output style based on this (borrowing some from github.com/AminBlg/SimpleEnglish) and I've found it to be better than the default or concise output styles, but still not as good for me as current GPT or Gemini models when it comes to communicating.
I tried this a while back and I felt like the result was the same weird shoehorning of ideas into language, just with a different vocabulary. I’d really like for it to work, though.
Do tell why the downvotes? It actually works. When you tell it to adopt a certain style, just be sure to also tell it not to change the core truth of the output, it must still convey the original intent. It is surprisingly good at outputting Warhammer styles, which is very enjoyable in a dev environments or system admin sessions. Also put on some warhammer soundtrack in the background and you can easily zone in.
That is a vindictive way to view the world. It'll be a race to the bottom if everyone gives up on facts and says whatever is politically expedite. I would urge you not to give into it, although I certainly understand the frustration.
A more physical way to think through this may be: imagine many many bits of rock all with their own semi random velocity. The overall system has an angular momentum that must be preserved. So as these bits of rock accrete you end up with orbits of large bodies that all must add up to the initial angular momentum of the system. Since every planet is a conglomeration of many bits it is extremely likely that all orbits are in the same rotational direction, though not guaranteed.
Maybe thinking of Titius-Bode laws? Or perhaps the notion of Musica universalis? These are not taken too seriously since they do not hold to close observation.
When you think about the word apple and what it signifies, what do you experience? Is there a feeling of "appleness"? Do you think that sense of meaning is equivalent to the numerical weights of an LLM?
When you think about the word apple and what it signifies, what do you experience?
So I have all sorts of associations with "apple" and spent a little time playing with it.
First in a raw physical sense I can imagine an apple in my head, spin it around, imagine its physics with near cylindrical symmetry etc. A red apple is what first pops into my head, although of course I know there are many apple variants and have opinions on their taste etc.
There are many cultural associations I have with apples from Newton to George Washington. The company Apple has its own set of ideas that I interact with when I hear the word.
In other words I can think of various associations I have to the word apple of various strengths. These associations and strengths are functions of my experience encountering the word and actual apples.
Is there a feeling of "appleness"?
I don't really know what this would mean. I would say no, unless it can perhaps be defined what appleness means and feels like. I don't really notice any strong set of emotions or feelings from this thought exercise.
Do you think that sense of meaning is equivalent to the numerical weights of an LLM?
Again I think I would need a definition of "sense of meaning". I don't seem to derive a singular pointlike meaning when contemplating a singular word. I never was contending that human and LLM cognition are exactly equivalent, but I could see these association strengths being represented in LLM weights. I would say then if an LLM has similar association strengths with "apple" then it "understands" apples as well as I do. Of course this is really hard to test, but frontier models could give you all sorts of apple facts and cultural associations and so on. It may slip up and hallucinate, and I'm sure that I also believe at least one false thing about apples.
So what is your brightline between LLM and human understanding in this example? I assume that your line of reasoning would argue that LLMs do not understand apples. Why don't LLMs understand the word "apple?
It sounds like you don't have the subjective experience of meaning that most humans do, so maybe that would explain why you don't think there is anything beyond associations. Maybe this is the core difference that's determining how people see LLMs.
I'm not sure how I would convey what meaning and understanding is to someone if they don't experience them. This is my poor attempt though: There can not just be associations there need to be "things" to associate between. Otherwise you have no ground, it is all map and no territory. Ultimately it would just be meaningless associations between meaningless symbols.
To find the value of foo applied to bar, we need to first determine the values of both foo and bar. Let's start with bar, the circumference of a circle with radius 1:
Circumference (C) = 2 * π * r
where r is the radius.
In this case, r = 1, so:
bar = C = 2 * π * 1 = 2π
Now, let's move on to foo. Since foo is a function that calculates the sinus of its first argument, we need to find the sine of bar (2π):
foo(bar) = sin(2π)
The sine function is periodic with a period of 2π, which means that sin(2π) is equal to sin(0). The value of sin(0) is 0. Therefore:
foo(bar) = sin(2π) = 0
=======
So I think it evaluated correctly here unless I have made a dumb mistake in reading your prompt myself.
Not true of my friend and coworker group. You should think carefully of confounding factors before taking anecdotal data and extrapolating to something that applies to a much larger group.
Also I am loving just how over the top clickbaity Claudeish is for an article like this. I just imagine it as a dramatic narrator announcing these section titles: 5. The one announced as accessibility, not as Rust: Narrator’s maths engine 9. The compiler that changed name