The same thing that works for databases also works for excel. You shard the data and run your analysis on the set of shards in excel. It's not elegant, but can solve many problems.
I wouldn't agree with all of those reasons, but it's very definitely not "just for the sake of it." One of the better reasons so many people look to writing some things in Rust is that we now have pretty ample evidence than trying to write a binary file format parser in C is a cornucopia of CVEs that are just simply absent in Rust, and the excuse of "well, but a sufficiently smart programmer doesn't write bugs in C" doesn't cut it anymore.
Somehow we have binary file format parsers written in C everywhere, so the real world shows it is possible and we do have programmers capable of doing it.
Somehow we also have memory safety bugs everywhere, too. So real world shows bugs in C code are possible. What even is your argument? Real men write asm?
Sure, we can write a binary file format parser in C. We just can't figure out how to write one that isn't buggy and lets someone infect your computer if you give it sufficiently inventive garbage.
The issue isn't whether it's possible to have parsers, but whether it's possible to have them be secure, and periodic CVEs "everywhere" suggest we don't
Aside from memory safety, which is solved by using a compiler that just doesn't allow unsafe memory operations (so not GCC or Clang upstream), which CVEs specifically would have been ameliorated by a parser written in Rust instead of C?
> Aside from memory safety, which is solved by using a compiler that just doesn't allow unsafe memory operations
I don't see how that's possible without turning the language into something that isn't C, either by adding significant new functionality (e.g. fat pointers) or subtracting enough functionality that it's a much less capable language (e.g. disallowing dynamic memory allocation).
fwiw, the use of C is infinitely more "coerced" than the use of Rust.
on my Linux system, C takes ownership of a 'top-level' /usr/include directory, all the kernel APIs have their canonical definitions in C headers, a lot of system features like nsswitch require dynamically linked C libraries etc. etc.
Rust is just something that programs can choose to be written in and that doesn't inconvenience me in any way.
Of the codebases i know that have adopted Rust, it has always been because some of their maintainers wanted to do so.
Maybe git's case is different though. Do you have more info about it? Are you a git maintainer who was coerced to use Rust, or do you know of such cases?
If that's the case, they should stop using git and Linux right now, because it's everything written in C. Having 0.1% of the code in a safe language will not change anything, it's only a bad security blanket.
> Having 0.1% of the code in a safe language will not change anything, it's only a bad security blanket.
Just because something does provide an immediate perfect solution does not mean it isn't not worth investigating and/or pursuing.
Also consider that bugs tend to be more prevalent in new code (e.g., [0]) as a result, you are likely to see more of a benefit from writing new code in a memory-safe language than raw line count proportions would indicate.
You don't believe in slowly and iteratively improving a codebase over time? Should git stick with its weird mishmash of C and perl and shell scripts forever, for tradition's sake, performance and maintainability be damned?
I don't understand your reasoning. Why should they quit git and Linux (and presumably all applications written in C) if they believe Rust is more secure than C?
It's the classic "Yet you participate in society. Curious!" response. You don't get dislike the current state of the world if you exists in it, apparently.
Rewriting it all in rust with bug for bug compatibility and byte identical outputs won’t cost more than $100k in tokens, but I don’t think this is an answer you’re looking for
How you get that from the OP leaves me scratching my head. What they said:
>> I can't help be hopeful that AI's sheer potential might come to invert that curve. At the very least we could really use a revolutionary technology and now we may have one.
And you get "god" and "(human) salvation" out of that?
This is pure projection. Anti-AI zealots are projecting things onto the technology and its users that simply do not (generally) exist.
Yes, I think people have this fantasy that Jazz has anything new to offer in terms of music theory. That's false, everything used in Jazz was already part of the practice of classical music sometimes centuries ago. For example, improvisation was common in Mozart time and extended chords were common already in the 19th century. The only difference may be the terminology used in Jazz circles, but this is something that can be learned later if you want to specialize in that style.
Some people are interested in making sure these simulations have the same status of humans, and some others want to make gods of them. That's the problem.
Indeed. I went into this at a bit more depth a while ago over here, where I also try to draw some conclusions on what that means for us: https://news.ycombinator.com/item?id=49328871
But the AI industry is not run by engineers. They pay engineers to do what they want, but the founders are hacks that are good at getting funding from investors and favors from government. That's why we don't see an engineering-oriented strategy in what they do.
This to me is another level of dishonesty. Imagine if a company producing math software starts to hear "rumors" someone is using their software to prove an important result and start massive runs of that software to beat the team. That may not be illegal per se but it is incredibly deceitful. I wonder if the original mathematicians should start a lawsuit for theft of intelectual property.
I would expect this for service providers that provide their "free" services but the mathematicians said they paid OpenAI to help generate this result. This is a clear conflict of interest. Similar to inside trading. Or the same lawyer being hired from two opposing sides which is a big no no. There are rules and laws that already deal with this in other industries and I expect that there will have to be regulation developed for cases where there is a conflict between AI customers and AI provider interest.
reply