sulam
Born on July 11, 2012•5193 Karma
Either way, I don't see much point of intentional austerity in times of extreme growth. There will be time for austerity once the growth ends.
You’re making some decent points here, but you’re either forgetting or ignoring the major thing that people usually neglect to mention when they want to make a case for this (crazy) idea — weight. Unless SpaceX is going to completely redesign hardware such that it is optimized entirely for its mass, it requires many (many) launches to even get a small set of racks into space. I don’t normally get up in arms about the CO2 emissions of data centers, I think there is offsetting value created by their use, but I would absolutely protest trying to put data centers in space and do my best to shut down the hundreds, if not thousands of launches it would take to achieve even a tiny fraction of an AI data center.
I will upgrade the "why it matters" to "and now AI output is part of the training data". A day is coming when the punched-up AI verbiage will be the norm and hard to distinguish unless you're from the previous generation. Sort of in the way that I miss some aspects of Usenet.
I could only follow up with, "that is a genuine insight."
Not a single person visibly flinched in pain.
Seems stifling. We'll need someway to reward human creativity and out-of-bounds thinking before our greatest corpus of human intellect is a bounded by whenever and whatever was trained on.
The LLM is bounded by it's training data, and relying on it means we are as well.
As a much more immediate practical matter, LLMs trained on LLM output makes them worse overall, they degrade from doing that. So the more LLM-prodoced content fills the web, the less useful it is as a data source for future LLM training. In addition to just being increasingly boring and vapid.
I think I'll stick with 1996 though :)
As opposed to other technologies; microchips do not grow on trees.