The problem with "ignorance" in Western countries (particularly the US right now) is that it's very common for people who don't know to believe they know and form ignorant opinions that they often want to be applied society-wide in some way. You can see this with everything from climate change to vaccines.
In much of the world, even in middle income countries, people are comparatively poor and, in my experience living abroad in such countries for many years, much less concerned with "understanding" and forming opinions about everything under the sun. It doesn't mean they don't value education and are opposed to development/progress, but it does mean that they don't question whether the vaccine they're taking is the product of a conspiracy, think too deeply about why the river is flooding more often, etc.
They just deal with life the best they can and are more focused on supporting their families, enjoying what they can, etc.
Culture and religion play into this. The way secular and Judeo-Christian people look at the world is very different than, say, Buddhists, Muslims, Fulani tribespeople, and so on.
Great point. It's not a distributed systems problem. It's a problem of many independent components trying to coordinate independently is the load bearing issue.
The actual outcome. The fact this post is even being commented on - that hackers have discovered a way to influence the models - is but one small proof. The vast majority of data coming out of LLMs these days is truthful enough that you're going out of your way to argue that this special case is noteworthy. The idea that the ingestion of data is "uncritical, and based on the assumption that all data is of equal quality" is ridiculous
So they went out of their way to turn off the magic good useful data that isn't being manipulated switch here but the rest of the time it's fine? Seems like a lot of effort to go to.
That's the usual bait and switch though. Very few people are saying AI the statistical software is going away. What people are dubious of is the commercialisation model of building a butt ton of data centres to serve massive general models by two heavily subsidised VC companies even as industry gets more mature and fickle as they get their heads around using it to solve problems and it goes onto Opex budgets. I think this is more like the early days of cloud which finished in "software built around clouds" rather than "everyone spins up their own fleet of servers for everything". Obviously the current investment levels need everyone to run their own fleet of Ralph looping agents on frontier models so that's what's marketed as the smart thing to do.
A lot of government appendages are still waiting for the dust to settle on this new fangled cloud computing thing in fairness. I don't think it's some deep masterplan
You have to hold up a cell phone to film people with it. It's a very obvious action. Definitely not intentional in the design but it's an accidental feature.
Well he's dead now in no small part because he was 100% correct that partisan activists amplified by friendly "journalists' would follow the usual playbook of creating a reality distortion field by repeatedly lying and republishing each others half truths and outright lies into a massive gish gallop.
Same question as always. What were the lies? Because all the ones I've seen were either found to be true or they were exaggerated versions of statements and then attacking that as a lie.
Especially rich to talk about habitual liars when it's a bunch of British press outlets and activists accusing others of being liars.
> 1 The Racism Of It All: It is not news when white men commit academic misconduct. It gets reported in a handful of stories, there’s a bit of media attention, but everyone quickly forgets about the problems.
> 2 Arday Was Definitely Guilty: The second point is somehow even more frustrating. Yes, Jason Arday was hounded by a society that demands different standards for young Black men than white guys. Yes, the amount of scrutiny he received was obviously connected to the colour of his skin. But also, he definitely did plagiarize. And not just a little bit. His work is riddled with clear, obvious, and extensive theft from other published academic papers.
> You do not need to trust race realists or other unpleasant people. Just download the thesis and check for yourself. It is clear that a significant portion of the thesis was copied directly from other people’s work. This is not in doubt, it’s not in question, and the fact that so many people online are arguing that the plagiarism charges are false because a university said so is deeply depressing. It’s arguably the worst case of plagiarism I’ve ever seen.
> Whatever your thoughts on plagiarism, it’s clear that Arday’s work harmed very few people. As he himself said, “I didn't murder somebody”. If he hadn’t been a Black academic and a poster child for diversity at Cambridge university, we’d probably have never known his name.
> For some of Arday’s supporters, acknowledging that any of the media scrutiny was justified has become tantamount to siding with his “killers.” He is instead a martyr—the victim of a “lynching”—and the inconvenient fact of his fabulism must be tidied away from the virtual shrine. This is an emotional position rather than a rational one, and its very indefensibility makes it a greater demonstration of tribal loyalty. Insisting that the academic was completely innocent, in the face of all the evidence, sends a strong social signal that you hate the right-wing media and care about racism.
> The postmortem airbrushing of Arday’s story also ignores how that story twisted and turned while he was alive. When The Guardian confronted him with the implausibility of some of his tall tales, Arday responded by editing them in real time: Yes, he had once claimed to have run 600 miles in six days, an exceptional feat for an endurance athlete, but what he actually meant was that he took daylong rest breaks in between. Yes, he had claimed to have raised more than £5 million for charity, but actually this was with the help of “many fundraising collectives” whose participants he could not name because they had signed nondisclosure agreements.
As they might say on Reddit, "Everybody Sucks Here". Arday the fabulist and plagiarist, the academic institutions who didn't vet him because they were razzle-dazzled by his bullshit lifestory (then drop him like a stone when their attempts to keep his plagiarism hidden don't work any more), the popular press who are happy to amp up a public shaming for clicks and views, the anti-woke and the racists who say this just confirms what they thought about DEI, and the anti-anti-woke and anti-racists who will deny the bleeding obvious if it makes them appear virtuous to their peers.
I mean, it opens with "It is not news when white men commit academic misconduct." then lists several cases where white men committed academic misconduct and made headlines in mainstream media around the world. Then walks that back with "This sort of massive media storm almost never happens for white academics".
Ultimately it's true that racists amplified the story, and once the story was positioned as being about race/racists that contributed (in part) to its longevity, but I don't see anything in that post to justify the claim that "If Arday had not been a young, successful Black academic, the chances that anyone would’ve looked into his work are low." and especially that "no one would’ve cared."
There was plenty of drama to fuel the media too. The accuser was suspended from the university where he worked, lawsuits were filed to try to bury the story, the police publicly apologized for harassing a journalist, new laws were proposed limiting the freedom of the press, statements were made by high ranking politicians and by the Foundation for Individual Rights and Expression, the accused committed suicide, there was a whole lot going with that situation way beyond the internal academic investigation about plagiarism and race issues. Of course it'd generate more mainstream press than say, a spider biologist who faked research data https://en.wikipedia.org/wiki/Jonathan_Pruitt (although that was a pretty big story in certain circles)
Step 1: put out the suggestion that the lies weren't lies
Step 2: don't read the responses
Job done!
EDIT: I'll give you one set of lies. Is it "exaggerated versions of statements" or you just didn't see them?
If I say I won the Nobel Peace Prize for inventing a cure for cancer, and I have no evidence of either, but I insist I did both and make up a lame excuse as to why you can't see my prize (it's away being washed today) or why you can't find my cure (the journal pretended someone else wrote the article because they're all against me, but trust me mate it was really me who wrote it) am I just making "exaggerated versions of statements" in your view?
> In a subsequent interview with the Guardian, Arday claimed to have been confronted at his faculty building on two separate occasions by a masked man threatening to harm him if he did not resign, and evading detection both times. On the second occasion Arday said the intruder produced a knife and threatened him with it.
> The Guardian understands that after Arday reported the first intrusion to the university, enhanced CCTV was placed in the faculty and a panic alarm was installed in his office. The professor said he hit the panic alarm during the second confrontation, but that no one responded. It should have triggered an alarm alerting the faculty’s security team, all of whom were present at the time of the alleged incident.
> The intruder was not picked up by CCTV on either occasion and no one in the faculty reported seeing anyone suspicious. Arday did not report either intrusion at the time, telling the Guardian that after the second incident he continued his working day and even conducted a PhD viva for a student less than two hours later.
> He did not inform colleagues that a masked man with a knife was in the faculty building.
> Arday confirmed in an interview with the Guardian that the “mutilated animals sent to his family home” mentioned in the open letter was a severed pig’s head, delivered in a large cardboard box to his parents’ home address in south London. He claimed to have intercepted the package and immediately thrown it away, only telling the police several months later.
> In a subsequent interview he claimed the police investigated, checking with butchers in south London if they had sold a pig, eventually finding one that had sold a “whole hog” on the morning the head was left outside the family home.
> When the Guardian checked those details with the local butchers Arday named, they said no police officer had come in to ask them about a pig. When asked if they were sure, the butcher said “that’s the kind of thing we’d remember”.
> When the details Arday gave about the pig’s head investigation were checked by London’s Metropolitan police, the Guardian was told they were “categorically” incorrect and no investigation had taken place.
> Arday had contacted the Met to lodge a complaint of harassment against Harris after his plagiarism allegations and, because he was based in Plymouth, the case was handed over to Devon and Cornwall police. Correspondence seen by the Guardian confirmed that the force declined to pursue a case against Harris.
> Arday took no photographs of the bullet, the banana, the corrosive substance or the pig’s head. The only other people who saw any of the items were close family members, who told the Guardian Arday had shown them the bullet and the “poison”. Arday could produce no messages from the time in which he discussed any of the items with friends, family, colleagues or the police.
> There are many examples of prominent black figures being intimidated [...] But the abuse and harassment Arday has outlined would be some of the most extreme any academic has been subjected to in the UK. When Arday was asked about his claims by the Guardian, he stood by them. “To be honest with you, I thought you’d just believe me,” he said. “Why would I lie?” He would be jeopardising his career.
Killing someone for absurd reasons then lying about it because while knowing it's an absurdity/mistake risking it happening again seems almost worse than believing/buying into an extreme/insane worldview and killing someone for it.
Why would we be talking about model routers in the context of it being a conspiracy theory that OpenAI is ingesting data from users for training? Are you acting obtuse or do you genuinely not reading/following the conversation?
They are all very open about it. It's likely the reason why the frontier labs are ok paying many more dollars a person in compute than they receive as revenue on subscription users for now.
Especially if you're doing something in a very sparsely populated part of their latent space it just makes sense that it would get used. Their biggest problem getting solved at the moment is going beyond what common crawl enables.
If it's in the TOS, and your settings, and they have pages describing it, and it's the reason they're willing to subsidize your usage, and in your own words "they are all very open about it", then how exactly are they "stealing your work" here, which was the original point about this being a conspiracy theory?
Because most people assume that when they say "we will use your data to improve things" that it means "improve your experience" not "we will steal your non-public ideas and use our subsidised compute to scoop your work and say we did it for press releases and then conduct a weird flail of PR smokescreens and threats when the lie falls apart".
Its quite obviously about good/bad faith use of user data.
My low level conspiracy is the reverse snobbery about knowing things is mutually beneficial for cloud providers and AI labs that both want software engineers to be as hopeless and dependent as possible so they'll consume more services/tokens and will shout down anyone saying "hey we could probably write this"
There was an article a few years ago that expressed this sentiment quite eloquently:
> “The merchants of complexity will try to convince you that you can’t do anything yourself these days,” wrote David Heinemeier Hansson (DHH), the creator of Ruby on Rails. “You can’t do auth, you can’t do scale, you can’t run a database, you can’t connect a computer to the internet. You’re a helpless peon who should just buy their wares. No. Reject.” [1]
DHH also did a very inspiring talk about mastery and why he loved the Ruby language in the "DHH is right about everything" [2] video.
LLMs have great potential. So, it turned out, did uranium, just not as chewing gum or a hair pomade.
There are good ways to leverage LLMs, but there's a lot more load bearing wait on that word 'leverage'. Something needs to do the leveraging, and do it well.
I'm experimenting with my own harness at the moment, currently codenamed Murder because I call the individual contexts/agents 'crow's.
The fundamental unit of it is what I call 'intrusive harnessing', where the harness actively manipulates the token stream so that significant quantities of tokens are only ever exposed to Layer0 when it's useful for them to be present.
For example: the full instructions for shell-tool calling aren't in the system prompt diluting attention while the model is reasoning/discussing what kinds of cat picture you want to put in your app.
My approach is more like dev-branching, and it seems to be working way more effectively than compaction or simple aggressive sub-agenting.
As soon as the harness sees the model is inferring a shell tool call, I stop the inference, mutate the context so that the full set of instructions/examples/guidance for shell tool use are inserted. Once the model has inferred the tool call, I curate the output it gets back. I ask the model to evaluate the output - good or bad - and give it a chance to accept/retry, before allowing the tool-call and output into the original context.
Does it use more tokens? Yes, although we're only mutating at head, so in a long-horizon context, it leans heavily into cache, just not the way anthropic/openai want you to realize you can.
It sounds like compaction but it doesn't come with the nasty brainwash experience where you just need the agent to fix that one last thing, it compacts and the agent comes back a paranoid delusional mad max.
```
<|system|>You're an AI agent. You do agent things.
<|system|> ... there's a list-dir tool and a shell-call tool ...
<|system|> ... memories
...
<|user|>It doesn't look like it ran.
<|reason|>I should look and see if there are any errors in the log file.<|agent|>I'm going to read the log file to see if there are any errors.
<|tool-call tool=shell-tool
```
We stop there, and splice in the detailed instructions for the tool the model was about to predict. I'll use <|ALLCAPS|> to denote harness-generated pseudo turns.
```
... as before ...
<|agent|>I'm going to read the log file to see if there are any errors.
<|SYSTEM|>Shell Tool: ... shell-type=bash, zsh, fish, pwsh on this system. Preferred shell is ... Additional arguments ... Pagination ...
<|tool-call tool=shell-tool
```
the model finishes out the call. On windows, with a typical harness, this frequently goes like this:
```
<|tool-call tool=shell-tool|>Get-EventLog ... | head<|tool-call|>
'''tool-result
error: unknown command: head
'''
<|agent|>Ah, windows doesn't have head. Let me just read the whole log.
<|tool-call ...|>
'''tool-result
... 500k tokens ...
<|agent|>I see some windows log events but you didn't ask me a question. Daisy, daisy?
```
With Murder it goes like this:
Rev 1
```
... prefix as before ...
<|tool-call tool=shell-tool
```
Rev 2
```
... prefix as before ...
<|SYSTEM|> ... how to use shell tool; shell-related memories and rules ...
<|tool-call tool=shell-tool shell=pwsh fence-vs-escape=true|>
'''pwsh
Get-EventLog ... | head
'''
'''tool-result
error: unknown command: head
<RESULT>Your tool call terminated with an error, ...
... structured response required ... options
<ACCEPT /> or <ACCEPT> <WITH> annotation </WITH> </ACCEPT>,
<REDO> ... </REDO>
<RETHINK> ...
<|reason|> windows doesn't have the head command. Let me try reading the whole log.
<REDO><TOOL-CALL> ... replacement tool call ... </TOOL-CALL> <WITH> ... model note ... </WIDTH></REDO>
```
I take that feedback and loop it, so, Rev 3:
```
<|system|> ... how to use shell tool; shell-related memories and rules ...
<|agent|>
... prefix as before ...
<|SYSTEM|> ... as before ...
<|agent|>{prev_cmd} failed, because windows does not have a head command. Let me try reading the whole log.
<|tool-call ... no head ...|>
'''tool-result
... first few lines of result ...
'''
<|system|>Your tool call succeeded but generated 446,219 lines of output. Only the first 5 were listed.
... structured pagination / retry / rephrase options ...
```
It then repeats while the model figures out the right command, figures out which filters to use, but the harness effectively immediately guides the model to do an immediate [optionally self-adversarial] review of the command against the output until the model concludes that the result is useful by various criteria. That doesn't mean successful - sometimes what is superficially an error (no such file or directory) is the answer you were looking for.
Let's say it takes the model 3 more turns to figure out how to use event viewer, and finally it <ACCEPT>s.
Here's the win, the outer main context - the one we're going to keep growing as you work with the agent, looks like this:
```
<|system|>You're an AI agent. You do agent things.
<|system|> ... there's a list-dir tool and a shell-call tool ...
<|system|> ... memories
...
<|user|>It doesn't look like it ran.
<|reason|>I should look and see if there are any errors in the log file.<|agent|>I'm going to read the log file to see if there are any errors.
<|tool-call tool=shell-tool shell=pwsh|>Get-EventLog ... | ... | ...
'''tool-result (use ref-tool id=A401U8X593 for full transcript)
Event ID | Last Occurred
1010111 | 3 weeks ago
'''
```
We used a lot more tokens. How can that possibly be good?
It's happening at the end of the context, so the cache comes into play very effectively.
But if we'd let all that derp into the context, it would be a potential attention sink degrading the value/worth of every subsequent token.
The pattern of try-thing-fail-try-solution-fail-try-win appears to be an incredibly strong pattern for most agents.
Fundamentally: When you're 3 prompts down the line and there's the imprint of the model doing "somewindows command | head" in the context with the model litigating it and fixing it -- that meta-pattern will drive the model to predict more of these patterns. It's going to repeatedly eff-up the exact way it saw in its training material.
When I try to get Claude/Copilot to work on this codebase, they freak out. The hyperbole/marketing pitch the agents were trained on and is built into their inner prompts cannot seem abide the idea of stopping an LLM mid inference. They seem driven to perceive an LLM endpoint like a 911 call you can't just go quiet on.
I have a mechanism for non-parallel sub-agents ('maggots', their job is to curate a large body of work whose full text is irrelevant to the main context). Basically just a tool call, but every time Claude or GPT have been near it, they've broken it, forcing it back parallel so they can send the invoking model a notification that it's child has been spawned and the parent should call the 'check-result' or 'wait-result' tool when they're ready to receive the results.
One of my test architectures is running against a solo Unsloth Studio instance that can only load one model at a time. It really doesn't react well to having you load the coding model to start your sub-agent work and unload before the model has generated its first token... :)
Also exploring mechanisms that try to pre-emptively keep attention-draining distractions/anti-patterns out of the context, things like when a model edits a file, we take the cache hit of removing the stale versions it read to make the modifications, replacing them with a reference syntax that the model can access in a sort of sandboxed auto-fork of the context.
That's going a little slowly because I'm trying to strike a balance between working 'reasonably' with extant models, and providing a mechanism to SFT/lorafy a model to make best use of it.
They don't have to have an in-depth understanding of leveraged finance to get pissed off when their doctors start doubling prices or their own employment conditions get worse or their parents get treated badly in care homes because the staff are now overworked etc. they're mad at the outcomes they're overwhelmingly not actually trying to debate the merits of it from an exit liquidity perspective.
reply