Understanding FLOPS within the Age of Massive Language Fashions
Earlier than we dive into numbers and {hardware} specs, let’s step again. Once we say we’re computing one thing — particularly within the realm of large-scale AI — what are we actually doing? We’re choreographing billions (typically trillions) of tiny mathematical operations to simulate understanding, generate textual content, or study patterns.
This isn’t simply technical engineering; it’s a philosophical frontier. We’re quantifying cognition by way of operations. However how will we measure that exercise — and why ought to we?
However what precisely are FLOPS ?
Consider a floating level quantity as what you may name a decimal quantity. A floating level operation is solely a mathematical calculation carried out with these numbers.
This contains operations like including, subtracting, multiplying, or dividing.
So, when you do 1.2 + 3.4, you’ve simply accomplished one floating level operation, or one “flop”
It’s vital to notice the excellence between “flops” (lowercase) and “FLOPS” (uppercase). Confusingly, they imply various things. “flops” (lowercase) merely means…
