What to watch out for with AI

Before SpaceX goes public with a trillion dollar valuation we had to settle for OpenAI’s latest funding round of $122 billion which values the company at $852 billion. And we also learned from the CEO of Nvidia that Artificial General Intelligence (AGI) is already here.

Once SpaceX is public, before we know there will be 1 million AI satellites in orbit, Nvidia will provide all the required chips to further support AGI, perhaps transitioning to ASI (Artificial Super Intelligence), and OpenAI will launch the super app and will become hugely profitable. This ludicrous speed of innovation from these companies must leave you wondering. Is there still anything else we can do and invent? Or do we just sit back and give up

Sometimes it feels like being the Hunchback of Notre-Dame hearing the bells. Oh, the noise, the noise.

If you are wondering what’s coming up next, what to pay attention to, and what will be relevant in the next 2–3 years, here are a few examples for your reading pleasure.

In a typical technology cycle going from the mainframe/data center to the personal computer/mobile device, always grappling with the bandwidth constraint and computing power at each end, we will see the advent of Edge AI. While most likely you have heard the term GPU (Graphics Processing Unit) — it is the rainmaker tech for Nvidia — or various types of TPUs (Tensor Processing Units), custom designed chips by and for Google, AWS, Microsoft to name a few, there are other technologies which are being worked on.

Most notably, the Neuromorphic Chips (NPU). These are the chips that are designed to mimic the human brain neural structure and mimic synaptic transmission. What makes them different from the other chips? First, the power consumption is 100x — 1000x less than your traditional GPU. While we are talking about gigawatts of energy for data centers, these chips consume energy in low teens or even single digit watts. As you can imagine that’s a major must for any mobile application.

Secondly, these chips can learn without explicit reprogramming and adapt to real world scenarios. They can be used in real-time sensory data processing or pattern recognition. Anytime you hear the word robot, autonomous system or IoT device, the chances are that it will contain one of these chips.

What are the companies making them? Intel with its Loihi chip, BrainChipwith its Akida processor or IBM’s NorthPole. While you might not have the resources to build your own NPU chip, you might be interested in the software development and do some research around Neuromorphic computing. It is a space worth following.

What else?

One of the reasons why we need more and more GPUs is to train the next generation of AI models. That requires a never ending stream of data funneled in. But what if the data contains private or sensitive information? What if the amount of data is such that you don’t have the bandwidth to transmit the data to the central server?

That’s what Federated Learning is for. Imagine a central server with the initial model. That model gets distributed to all the end nodes or client devices. Each device trains the model on its own data and sends back only the updates to the model (no raw data) back to the central server. The server consolidates all the changes to a new version of the model and sends back that new version to all the end nodes. That continues either until the model reaches the desired performance or goes on forever every time new data comes in and requires retraining the model.

The benefits? Better privacy protection, less data to send back and forth and faster training. Sectors like healthcare, industrial IoT, telco or banking will benefit from this technology. Checkout Flower Labs with its Flower framework or Owkin with its Owkin Connect for better research in healthcare.

Another area you can direct your attention to are the highly specialized models which are trained on very small, focused subsets of information. The subject matter experts. When you interact with ChatGPT, Gemini or Claude, you are dealing with a generic system. The more generic it is the more it is prone to hallucination (nonsensical or factually wrong answers) — the math is clear on that. And you can correlate this with reality. While OpenAI — a jack of all trades, a master of none — is trying to identify what product they should build on top of its technology, startups are identifying specific use cases, business problems and applying the technology in very niche spaces.

These companies have something which the big companies can’t have or it would not be economical for them to do. They have a proprietary training data set, which usually comes with much higher quality. Also these data sets are much smaller, so the training of these specialized models is faster and cheaper. The outcome is higher accuracy — and very importantly — better explainability. If there is one area to pay attention to, it is the Explainable AI (XAI) which is trying to address the ‘black box’ problem of all the Large Language Models (LLMs) in use today.

These are a few examples of some of the technologies going unnoticed and unreported by the media. In a few years it will become the next hot trend but for you it will be old news. It is this recurrent pattern which gives you the confidence to see and recognize a thing coming up from miles away while filtering out the noise and hype.