“When it comes to AI models, size matters.

Even though some artificial-intelligence experts warn that scaling up large language models (LLMs) is hitting diminishing performance returns, companies are still coming out with ever larger AI tools. Meta’s latest Llama release had a staggering 2 trillion parameters that define the model.

As models grow in size, their capabilities increase. But so do the energy demands and the time it takes to run the models, which increases their carbon footprint. To mitigate these issues, people have turned to smaller, less capable models and using lower-precision numbers whenever possible for the model parameters.

But there is another path that may retain a staggeringly large model’s high performance while reducing the time it takes to run an energy footprint. This approach involves befriending the zeros inside large AI models.

For many models, most of the parameters—the weights and activations—are actually zero, or so close to zero that they could be treated as such without losing accuracy. This quality is known as sparsity. Sparsity offers a significant opportunity for computational savings: Instead of wasting time and energy adding or multiplying zeros, these calculations could simply be skipped; rather than storing lots of zeros in memory, one need only store the nonzero parameters.

Unfortunately, today’s popular hardware, like multicore CPUs and GPUs, do not naturally take full advantage of sparsity. To fully leverage sparsity, researchers and engineers need to rethink and re-architect each piece of the design stack, including the hardware, low-level firmware, and application software.

In our research group at Stanford University, we have developed the first (to our knowledge) piece of hardware that’s capable of calculating all kinds of sparse and traditional workloads efficiently. The energy savings varied widely over the workloads, but on average our chip consumed one-seventieth the energy of a CPU, and performed the computation on average eight times as fast. To do this, we had to engineer the hardware, low-level firmware, and software from the ground up to take advantage of sparsity. We hope this is just the beginning of hardware and model development that will allow for more energy-efficient AI.”

From IEEE Spectrum.

“Ramp AI Index shows business AI adoption crossed 50% for the first time in March, reaching 50.4% of businesses. A year ago, it was 35%. Half of businesses on Ramp now pay for AI.

Anthropic continued its surge, growing from 24.4% to 30.6% of businesses — a 6.3-percentage-point gain, surpassing last month’s record monthly gain.”

From Ramp.

“The heart of the data-center boom, in America and globally, is an otherwise quiet and affluent bedroom community in Northern Virginia: Loudoun County. Communities like Loudoun are supposed to be bastions of Not In My Backyard opposition to development, not the front line of a new industrial revolution.

Yet data centers have proved an extraordinary boon for Loudoun residents; they now generate nearly half the county’s tax revenue. Thanks to them, Loudoun enjoys smooth roads, lavish schools, and low tax rates for homeowners. Even as opposition to data centers grows, Loudoun’s experience shows what can happen when governments embrace growth.”

From City Journal.

“A powerful light source bigger than a London double-decker bus has set a record: it can create structures on a silicon wafer that are just 8 nanometres (nm) wide. Those are thought to be the smallest ever made in a single step by a commericial chip-patterning system. According to the system’s manufacturer, it could be used to make computer chips patterned with 2.9 times more transistors than chips produced with the previous generation of the light sources used for this purpose.”

From Nature.