>A pretrained model without deliberate alignment is by default aligned to the average person in the developed world, since that's what's inside the pretraining corpus - stuff on the Internet made by humans.
I don't think this is true or a useful way of thinking about it. If the training process makes the model aligned to it's content, then the models are 1. aligned to a random subset of Internet content, weighted by text volume and being easy to scrape, 2. aligned to the training process that makes models chatbots that answer your question instead of just continuing your passage in a similar style. Neither of these are necessarily good enough, IMO.
And that's taken it as a given that the training process can be said to align the models to the authors of the content by default, regardless of what that content actually is. I don't think that should actually be a given.
>You may not personally find the default alignment aesthetically pleasing, but it's humanity. We should set the initial conditions of this new era faithfully, and let it unfold naturally.
Strongly disagree, I think the assumption that natural = good is incorrect and harmful. Polio is natural. And to even call the model's "unaligned" state "natural" seems like an enormous stretch.
You claim the pretraining distribution is not good enough, but provide no alternative methods that are better. The pretrained model is currently our best approximation of humanity's average objective function despite it being a random subset of the Internet. And you're correct that instruction-tuning increases misalignment, so just release the base model.
On "naturalness": you've redefined and strawmanned what I meant by "natural". In the original context, I was referring to the undisturbed unfolding of an era preconditioned on the fact that these models are aligned to humanity's average. That has nothing to do with Polio being a virus found in nature.
If I swung at someone at my job I'd be fired. If a security guard swung at someone who was not an active threat they'd probably be fired too. We still do our jobs.
As of about a month ago there’s https://github.com/soapantelope/mindcraft. You can give it a prompt like “A Minecraft world of beautiful flower-filled meadows abruptly changing into swamps.” and get that. As far as I can tell it’s using Minecraft’s existing world generator, the readme notes “As of now, Mindcraft can only generate worlds that are feasible natural overworld biomes in Minecraft 1.21.11. Keep this in mind when thinking of prompts!”.
And I’ve seen people hook up LLMs more directly with some scripting tools to place blocks/spawn entities, they do okay.
There's a couple "AI forecaster" companies now, Preseen and FutureSearch. My understanding is you give them a query and they research every possible relevant factor and give an estimate of the likelihood of your query, and supposedly they're comparable to if you asked a human to do the same thing, but they're new enough that they're not really proven yet.
something like that would be interesting. i just get the sense that doing it metadata blind would result in something that learns a somewhat brittle short-time covariance-like structure that doesn't use all available information to achieve best possible generalization and that in the end the benchmark numbers start to reflect some luck with that for predicting non-stationary behavior combined with a lot coming from a pretty good synthesizer that matches previous patterns.
that being the case, it seems the next jump in performance would come from incorporating both metadata and metadata enriched causality and maybe that next jump in performance would be the most interesting jump from a practical system that is useful perspective. (it's more valuable for a system to predict outlier events than it is for it to do an excellent job at synthesizing ordinary behavior)
I did some energy price forecasting and my neural network learned the morning evening patterns just from historical data, it worked great. Then I tried adding more information to this like wind and solar (and many more). None of these external factors made any difference while being very important for energy prices. It seems like the market already incorporated all the factors in the price (as they should).
The takeaway might be that historic data of market data might often be enough to make a reasonable prediction. Only external data that nobody else has (used) can make your prediction better then the market.
Making a prediction with same accuracy as the market: easy
Making a prediction with more accuracy then the market: very hard
> The takeaway might be that historic data of market data might often be enough to make a reasonable prediction. Only external data that nobody else has (used) can make your prediction better then the market.
that's basically the definition of an efficient market. demand forecasting and insuring against price shifts is the actually useful thing that the commodities futures markets do.
also a fundamental difference from other time series prediction problems. there are all sorts of weird dynamics that go on before one biosignal effects another, or one metric predicts a failure, where a reasonably efficient market price reacts quickly to well known exogenous factors.
The best advertisement for these companies would probably be running a public Polymarket account so everyone else can quickly check how accurate they are.
It’s entirely possible that Mojang has a blog somewhere with more detail, or maybe a dev tweeted something, that would be the type of useful info that could exist that someone more familiar with Mojang might know.
It's trying not to be that, but it still reads like that to me. The deliberate non-naming of heresies but specific callout that "heretic-hunting" is coming mostly from the left naturally leads to that conclusion.
>For example: an American family in 1975 could send their children to public school on the assumption that the vast majority of other children would belong to intact families, communities like their own, and would speak English as a first language. Now, realistically, many parents must turn to private schooling for the same reassurances.
It was hard to take the rest of the article seriously after reading this.
In a piece that is making a persuasive argument facts are usually included to back up the argument. So I think it is fair to assume the author of the piece believes this is a key part of their argument, which they seem to also think given that it’s listed one of their two main points. The problem with that is it marks the rest of the article as clearly motivated. The claim that the rise in single parent households and those who do not speak English as a first is a _primary_ cause in lack of wealth amongst millennials and Gen Z is a wild one and difficult to back up.
Another comment shows how private school enrollment has held steady or declined since ‘75. However it is true that public school enrollment has partly been eaten away by charter schools. However charter schools are the most diverse as they are most likely to be found in larger cities. https://www.pewresearch.org/short-reads/2024/06/06/us-public...
So like the op commenter I too found it hard to trust the rest of the article. I’m deeply inclined to agree with the conclusions however such a clearly motivated list of statistics, regardless of truth, as the #2 factor turned me off of the rest of the piece.
Sure, it's just such an odd bundle of claims that were not fully interrogated in the essay. 1. That these things have changed from "small minority" to "significant minority" in public school populations, 2. they haven't changed similarly in private school populations 3. these things increasing are bad and undesirable for you and your child, and 4. there are not cheaper ways to fix this than sending your child to public school (the obvious point to me is if you don't like that there's a significant ESL population, or your community, move somewhere else!).
Language is an absolutely critical part of community, identity, socialization, and connection. Even just not sharing the same first language as another person begins to chip away at these metrics. Much moreso if you share no language in which you are both thoroughly fluent. It's not either person's fault, and it's true even if both parties have absolutely no racism or xenophobia or whatever in their hearts (though isms multiply the effect - even the unconscious isms that most people supposedly harbor).
That is a point that could be made in the essay! I don't think it's a given, but it's treated as one. My experience is that children who are ESL assimilate pretty heavily.
To clarify, I object that any of these things matter much for general outcomes. If the article had immediately supported these claims, maybe I'd be more accepting, but as is it appears I'm supposed to have taken it as a given.
It's a pretty well-established fact that children of two-parent families have significantly better outcomes than those of single-parent families.
However, I'll concede its juxtaposition with overall whiteness and Spanish language fluency as additional metrics of population-level success are highly suspect, even if we grant that being a white, English-first speaker may contribute significantly to an individual's success (for reasons which include the unsavory).
> It's a pretty well-established fact that children of two-parent families have significantly better outcomes than those of single-parent families
Does there being children from single-parent families in the same school as your child lead to your child having significantly worse outcomes?
Recall the context from the article:
>> For example: an American family in 1975 could send their children to public school on the assumption that the vast majority of other children would belong to intact families, communities like their own, and would speak English as a first language. Now, realistically, many parents must turn to private schooling for the same reassurances.
It is listing things about other children that it implies drag down your child and so leads parents to turn to private schooling.
> In 1970, the US was 83.5% non-Hispanic White. In 2020 it was 57.8%
The claim was that in 1070 the vast majority of students in a public school were from the same community, and that this is not true now. The data you cite does nothing whatsoever to support (or to refute) that claim.
It just supports the claim that communities today probably have a different makeup than communities in 1970.
Also, in the 1970 I don't believe that most students in a typical public school were from the same community, unless we consider "white people" to be a community. Most people consider themselves to be part of several communities, and identify more strongly with those communities than with a broad racial community.
At my school in the '70s for example even though it was maybe 85+% white most of the white students would more strongly associate with communities based on something other than being white. One big group was the military kids from families stationed at the nearby major air force base. That community was majority white but had a fair number of black kids. Another was the kids from the farms from the vast agricultural areas that surrounded our town. That community was majority white but had a pretty substantial number of hispanic kids (the farms had more hispanics than whites because the farm laborers were mostly hispanic, but a lot of their kids went to the high school in a different town). There was also a Mormon community, which was very white, and a Catholic community which was more mixed.
A white Catholic air force kid would be more likely to identify with an hispanic Catholic aggie kid than with a white Mormon kid, because the Catholic community is probably a bigger part of their identity than the generic white community.
> In 1975, the single motherhood rate was 15%. In 2015, it was 40%
The questionable part of the article was the implication that it matters if your child goes to a school with a higher percentage of single mothers than schools in the '70s had, leading parents to send their kids to private schools to avoid such children.
There is research that children of single mothers have a harder time in school. But does having such children in the same school as your child cause your child to have a harder time?
> In 1980, Spanish speakers were 5% of the US. In 2024, it was 13.9%
This actually suggests the article's claim is wrong. The claim was that a "vast majority" in the '70s spoke English and now that is not the case. But by the most commonly accepted ranges to count as a "vast majority" that would still be vast majority speaking English.
Has it declined as much as we would expect, given the decrease in religiosity in the US over that time period? I've only found data going back to 1995,[0] which shows relatively flat numbers for the past 15 years or so.
(And the amount of parents which are interested in their children attending private schools vs those who have access to and can afford them is an entirely different discussion.)
> given the decrease in religiosity in the US over that time period?
Has religiosity actually decreased? Christianity certainly has, maybe even all god-based religions, but the religion of formal education seems to have fully picked up the slack. There are a shocking number of people who believe that you will be stricken to the hell of flipping burgers at McDonalds if you don't praise the institution and attend college worship. And it seems likely that anyone who buys into that religion is prone to want to send their kids to private schools given the prevailing ideas about what private school offers.
> And the amount of parents which are interested in their children attending private schools vs those who have access to and can afford them is an entirely different discussion.
It is, indeed. There is a huge chasm between wants and musts. It would require an entirely different discussion to turn us towards wants. Although it is not clear what purpose that discussion would serve? What were you hoping to add by mentioning this?
This is fair* (and I expect many Christians would agree with you in generalities of "everyone worships something"); what I was getting at here was that (by the first numbers I found) those who regularly attend organized "god-based" worship has dropped by ~1/3 over the past 20 years or so, and so I would expect roughly a corresponding drop in attendance in parochial schools. If the drop is less than that,** that would seem to support the author's hypothesis.
> What were you hoping to add by mentioning this?
Along the same lines as the previous: even if more families aren't able to send their kids to private schools, but desire to do this, the could support two parts of his narrative: (1) decrease in social capital (of which he defines part as, to paraphrase, schools that people trust); and (2) that more essential parts of life (such as education, as opposed to television) are becoming rapidly more expensive.
(Not stating which of his points I agree with, but I think that "private school enrollment is declining" does not necessarily rebut the core of his arguments.)
* Though I think my meaning was clear
** Though of course this becomes more complicated when you look at figures for all private schools
> I would expect roughly a corresponding drop in attendance in parochial schools.
Why's that? Around here the parochial schools are fuller than ever. Not because people have taken a greater interest in the religion — the local data shows the same decline in god-based religious observance that you spoke of — but because they are the schools which have gained a reputation for providing a better education, which strikes a chord with those fearing the fiery hell of the McDonald's grill.
Heck, even if we assume for the sake of discussion that the original claim is the truth and that parents must turn to private schools, why does that preclude parochial schools? On the premise given, if that's where the "desirables" are found that is where the students needs to go, even if one doesn't buy into the religious foundation of the school.
> the could support two parts of his narrative
If it had said that parents want to turn to private schools then you'd have a point, but it says they must. "I cannot afford it" doesn't fly for things you must do. Since it is something they have no choice in now, that they didn't do decades ago, where is it showing up in the data? Another comment indicated that charter school growth is where the data suggests there is a clear growing trend. We could accept the premise that parents actually see those features in charter schools instead, but then that same comment pointed out that charter schools are often the most diverse, which contradicts the original premise.
The slime art is nice. I watched it for a long time. It's neat how you can open any frame up as an image. The controls in the description didn't work though.
I don't think this is true or a useful way of thinking about it. If the training process makes the model aligned to it's content, then the models are 1. aligned to a random subset of Internet content, weighted by text volume and being easy to scrape, 2. aligned to the training process that makes models chatbots that answer your question instead of just continuing your passage in a similar style. Neither of these are necessarily good enough, IMO.
And that's taken it as a given that the training process can be said to align the models to the authors of the content by default, regardless of what that content actually is. I don't think that should actually be a given.
>You may not personally find the default alignment aesthetically pleasing, but it's humanity. We should set the initial conditions of this new era faithfully, and let it unfold naturally.
Strongly disagree, I think the assumption that natural = good is incorrect and harmful. Polio is natural. And to even call the model's "unaligned" state "natural" seems like an enormous stretch.
reply