Posts Tagged ‘semiconductor devices’

AI Hardware Accelerators and AI Models

August 3, 2020

At times it seems like IBM is just dallying with AI, but at a late July briefing IBM showed just how seriously it is pursuing AI and how difficult the challenge is. It starts with today’s computers and supercomputers, essentially bit processors. Can we say that’s the easy stuff; at least that’s the stuff we are most familiar with.

Neurons come next in the IBM scheme. Here biology and information are tapped for today’s rudimentary AI systems. Next you throw in qubits, which combine physics and information. Now we’re entering the realm of quantum machines.

DancingDinosaur is mesmerized by quantum computing but only understands it at the level of his 40-year old physics course. It starts with today’s computers and supercomputers, essentially bit processors. Can we say this is the easy stuff; at least that’s the stuff we are most familiar with.

Where all this is going is not toward some mesmerizing future of quantum systems dazzling us with nearly instant solutions to seemingly impossible problems. No, it seems, at one level, more prosaic than that, according to Jeffrey Burns.  IBM’s Director, AI Compute.

As Burns  puts it: IBM is building the future of computing. That future, he continues, is a pipeline of innovation for the future of Hybrid Cloud and AI. We should have known that except various IBMers have been saying it for several years at least and it just sounded too simple.

Burns breaks it down into four areas: Core Technology, Innovation for AI, Innovation for Hybrid Cloud, and Foundational Scientific Discovery. 

It is tempting to jump right to the Foundational Scientific Discovery stuff; that’s the sexy part. It includes new devices and materials, breakthrough data communications, computational, secured storage, and  persistent memory architectures.

At the other end is what Burns calls core technology. This encompasses semiconductor devices, processor architecture,  novel memory, MRAM, and advanced packaging.

Among the innovations for AI are AI hardware, real-time AI for transaction processing,  HW and SW for federated AI learning to enhance security and privacy.

Finally, there are innovations for hybrid cloud. These include Red Hat RHEL integration,  storage and data recovery,  high speed networking,  security, and heterogeneous system architecture for hybrid cloud.  

But, AI and Hybrid Cloud can advance only as far as hardware can take them, notes Burns. The processing demands at even the first two steps are significant. For example image recognition training with a dataset of 22K requires 4 GPUs, takes 16 days, and consumes 385 kWh. If you want it faster, you can throw 256 GPUs at it for 7 hours, which still consumes 450 kWh. Or think of it another way, he suggests: 1 model training run eats the equivalent of ~2 weeks of home energy consumption.

And we’ve just been talking about narrow AI. Broad AI, Burns continues, brings even more computational demands and greater functionality requirements at the edge.If you’re thinking of trying this with your data center, none of this is trivial. Last year IBM invested $2B  to create an  Artificial Intelligence Hardware Center. Twelve organizations have joined it and it continues to grow, Burns reports. You’re welcome to join.

IBM’s idea is to innovate and lead in AI accelerators for training and inferencing, leverage partnerships to drive AI leadership from materials and devices through software, and generate AI application demonstrators with an industry leading roadmap. 

Here is where Burns wants to take it: Extending performance by 2.5X/year through 2025.  Apply approximate computing principles to Digital AI cores with reduced precision, as well as Analog AI Cores (Remember analog? Burns sees it playing a big energy-saving role.), which could potentially offer another 100x in energy-efficiency.

If you want to try your hand at AI at this level, DancingDinosaur would love to know and throw some digital ink your way.

Alan Radding, a veteran information technology analyst, writer, and ghost-writer, is DancingDinosaur. Follow DancingDinosaur on Twitter, @mainframeblog, and see more of his work at

%d bloggers like this: