☆ Yσɠƚԋσʂ ☆

  • 641 Posts
  • 740 Comments
Joined 7 years ago
cake
Cake day: January 18th, 2020

help-circle
  • Compilers are incredibly complex things and the often have undefined behaviors that slip in because the compiler has to interact with the operating system at large, and that means the code actually doesn’t run the same on every platform, or even every version of the same operating system. So, while they are technically deterministic, the reality is that you get surprising and unexpected behaviors in practice.

    Also worth nothing that the probability distributions for an LLM are fixed at training time, and that is why the weights are static. When doing inference, models are selecting from a top-k of probable tokens at each autoregressive loop for your typical LLM. When you adjust top-k to 1 or simply set the “temperature” to 0, it will always give you the same result from the same input.














  • Which is that they don’t understand how statistics and probability work. The LLM generalizes patterns that get imprinted on the neural network during training, and then these patterns are expressed in the given context during inference. The stochastic aspect of the system is what allows the LLM to adopt generalized training to the specific problem being solved. So, the outputs LLMs produced are clearly not random in a sense of playing bingo because randomness is not the dominant property of the system.





  • But the incentive structure is only present in the corporate versions of these tools. I’m advocating for building these tools in the open, and we don’t have to mirror what corps are doing. I disagree that these tools are anti-worker at their core, they’re just another form of automation. And we have a concrete example of what happens when this technology is introduced in a rational fashion with China.

    Every form of automation will be used to alienate and atomize workers under capitalism, but it’s obviously worse if these tools are also controlled by corporations which will decide who uses these tools and how they use them. While we continue to live under a capitalist regime, open source approach at least helps mitigate the worst ills. On top of that, it directly cuts into the business model of these companies.

    Open models are what actually scares them and they state is plainly. We only have to listen to what these companies are saying themselves. They don’t care about boycotts, or some online activism, but they are lobbying to ban open models, and that tells you everything you need to know.


  • What part of distributed systems are you still struggling with here? Also, models are getting incredibly more efficient every few months. Qwen 3.8 that runs on a laptop today is comparable to Opus 4.6 from around half a year ago which needed a whole data centre.

    People working on local models don’t need to compete with whatever the big corps are doing, they just need to make tools that are good enough to use on your machine. Just the same way Linux didn’t have to do everything Windows did to become usable.

    And yeah, Luddites had lists of demands just you do, how’s that work out for them?

    And if the US somehow did ban open models, I’m sure the vassals would as well, but again who gives a fuck? The whole west is like 13% of the global population. And it’s imploding economically and politically in real time. Who cares what goes on in this genocidal cesspool?