☆ Yσɠƚԋσʂ ☆

  • 6.84K Posts
  • 7.77K Comments
Joined 6 years ago
Cake day: January 18th, 2020
  • ☆ Yσɠƚԋσʂ ☆@lemmy.mlOPtoWorld News@lemmy.ml•Macron Criticizes Trump and Calls on Allies to Unite Against US
    0·
    2 hours ago

    EU is not really an export power in any meaningful sense. But even if that was true, you could make exact same argument with the US which directly competes with Europe on exports. Europeans have had absolutely no problem sticking their heads up American ass as far as possible.

    Meanwhile, everybody is free to run their internal economy the way they see fit. The whole premise of free markets and capitalism was efficiency. It was supposed to naturally outcompete planned economies like China. If the model isn’t actually working then Europe could acknowledge that and emulate what China is doing instead of whinging. Or accept that they have an inferior model of economic development.

    Finally, the EU very obviously doesn’t give a fuck about protecting its economy. If it did, it would’ve never allowed itself to become so dependent on energy imports from the US. The EU kept paying lip service to the idea of going green, but refused to invest into stuff like wind and solar because it came from China. Now the EU finds itself being bent over a barrel by Americans. And it’s frankly well deserved.

  • ☆ Yσɠƚԋσʂ ☆@lemmy.mlOPtoTechnology@lemmy.ml•TurboQuant compresses LLM key-value caches down to 3 bits per value. 6× memory reduction, up to 8× faster attention, and no 0 degradation.
    1·
    3 days ago

    Binary quantization and 1 bit vectors have definitely been floating around the space for years. The big difference here is not necessarily just better raw precision but how they completely eliminate the hidden memory tax that usually comes with extreme compression. Normally when you crush a 32 bit float down to a single bit you destroy a massive amount of scale and range information. To make the model actually usable after that traditional methods usually have to store extra full precision numbers alongside those compressed blocks to act as scaling factors or zero points. So your theoretical 1 bit compression actually ends up costing something like 2 or 3 bits per parameter in practice.

    TurboQuant gets around this by using the Quantized Johnson Lindenstrauss transform which is basically a mathematical guarantee that the relative distances between different data points will be preserved even when the data is aggressively shrunk. By doing this and dropping everything to just a positive or negative sign bit they completely remove the need to store any full precision scaling factors. It literally has zero memory overhead. To make sure the attention mechanism still works they use a special estimator that takes a high precision query and runs it against that low precision 1 bit cache in a way that mathematically eliminates bias.

    You also have to look at how they are actually applying it in the pipeline. They don’t just take the raw 32 bit vector and smash it down to 1 bit right out of the gate. They use that PolarQuant method first to map everything to polar coordinates and capture the main structure and strength of the vector. The 1 bit QJL algorithm is only deployed at the very end as a targeted cleanup to fix residual errors left over from the first step.