Just that an actual rubber duck doesn’t do anything. You solve the problem you have by talking, and in doing so, thinking, to come up with a solution, an idea, or gain better understanding.
After that you either implement something yourself or have learned something.
With an LLM you offload all of that, the only thing you still do is tell it what the problem is. The agentic duck does the rest and you look at the output.
Even if you have to argue, you argue without having gone through the steps to gain anything yourself.
I basically spend way to much time with RISC-V related things.
But the easiest way to get more info about RISC-V developmemts is by watching the youtube uploads of the RISC-V summit talks.
Every SemiCon usually has a RiscV forum. Obviously there are also RiscV events. Wesemibay usually has several RiscV processors and GPU's from new vendors and you can then go and check what features they support. But basically as with everything else you have to show up and talk to people, or just look at the summaries afterwards.
You'd need hundreds of GB alone for the KV cache of each user. For something like LLama 3 405B you need ~67GB at ~130k tokens.
A single CS-3 has 44GB on-chip sram.
So, afaik, Cerebras are optimizing for ultra-low latency batch=1 inference.
Congrats on the great job with the Altar II. If I didn't already have too many keyboards, I'd hop on the Kickstarter!
FWIW, I fully agree with your opinion on including a `half working` fingerprint sensor.
if you assume that training requires about 3x the compute of inference (one forward pass, one backward pass, parameter updates), and we take DeepSeek-V3 since their numbers are public.
they used ~14.8 trillion tokens with about 2.66 million GPU hours.
14.8 * 3 = 44.4 t inference tokens.
obviously, this is back of the envelope math, but at 100t/s you would need like ~14k years. scale this to >100k GPUs and your in the hours to a couple days range.
Hasn't changed at all since AI agents became a thing. tmux, nvim with a few plugins, mainly fzf and LSP support. If I do use an AI agent, I just run it in another tmux window.
> I'm not sure you can prompt a full, accurate, copy of a nontrivial codebase out of them. Even with zero temperature their accuracy is just not that high.
> For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright in the U.S.: Harry Potter and the Sorcerer’s Stone and 1984 (Section 4).
That paper is basically using the LLM as a compression algorithm: it's prompting with some section of the book and it's reprompting if it doesn't give the right output. Notably this only works if you already have a copy of the book in question!
You misunderstand my point: the LLM is not a losslessly compressed version of the text: you need to supply additional information from the original in order to 'extract' it from the LLM (and from that point of view, the extra information would be the compressed form).
There is actually haha. I've always hated sushi. And sushi is now on my shortlist to try again. I can't wait.
(My handle comes from graffiti I found on the booth of a hot dog stand in Worcester MA called Coney Island[1]. I thought it was a cute oxymoron and adopted it on a silly whim. I only later learned that some sushi is indeed cooked.)
Man, the origin of your screen name is the same level of lore as Rust being named after fungi, and not corrosion. Love it! Glad you're in better health again. Been using your software for nearly 10 years spanning before and after my career started. Thanks for all the work you do in open source.
Man that's awesome. That's motivation to get one of my vehicles in good enough shape for a day trip (I'm about 3-4hr north depending on choice of route). I have fond memories of Worcester, and of all the places I've been since I'd be most surprised if it's changed much :p
reply