AI for the 21st Century

Modular AI

Modular AI refers to the architecture. Corporations are actively moving away from a large language model (LLM) as a single, all knowing brain. The industry is moving modular, breaking up the architecture into individual or modular components. You can read what Digital Architecture Lab had to say in their article titled Modularity in AI: Understanding the Building Blocks of Intelligence. Let’s take a look at a common AI Modular stack.

Modular Stack

• Security – This component checks for prompt injection, leaks, malware and any other bad actors trying to break into the system.
• Router/Orchestration – Here is where intent is evaluated and decides which specialist or specialists to call.
• Specialists – These are the specialists within the system. There can be multiple specialists such as medical, legal, data analyst, etc.
• Ethics/Compliance – This last part of our stack evaluates the output and checks for bias, toxicity and hallucinations. The component will also check or cross-reference output to corporate policy.

Corporate Gains

• Swap-ability – The ability to swap out components within the stack is a huge gain for corporations as newer more advanced components become available. It’s also a much easier way to expand the model (add or change specialists).
• Auditing and Compliance – Modularity offers an audit trail. Should something go wrong, technical specialists can review the audit logs to determine where the problem occurred. Did the security component miss something, was it routed incorrectly, did the specialist make a mistake or did we skip something within the ethics or compliance component.
• Cost efficiency – To maintain and upgrade or expand the models is much more cost effective than tearing down and rebuilding a monolithic AI system. There are also savings within the processing itself. Instead of having expensive processing time allocated to a specific request, specialists are added to provide lightweight processing.

Greener Fields

Having light weight individual or modular components also aids in the reduction of environmental resources, making this approach greener.

1. Security Shield – Tiny, specialized classifier. Processing time is negligible, fractions of a watt of electricity.
2. Router – Linear text machine, like a semantic router. Here too the processing time is negligible.
3. Core workers/specialists – Highly specialized 8 billion parameter model which has very low processing time.
4. Ethics evaluation (compliance) – Rule based classifier or small validator. Once again, processing time is negligible.

Even with all components working together, the mathematical operations are often 90% lower when compared to a monolithic AI system.

Caution

There is a cautionary tale that must be told. It’s called the Jevons’ Paradox. Simply put, this economic theory says that as a product(s) become more efficient, the result is not less (through the efficiency itself) but rather more usage. What that means is that while there is efficiency gains to be had going modular, it would result in more, not less usage. Possibly wiping out the gains all together and even resulting with a negative impact to the environment.

• Water – Today large data-centers consume about 5 million gallons of water daily. This amounts to just over 1.8 billion gallons a year. As modular AI expands, we could see this usage jump 17 to 73 billions of gallons of water per year.
• Electricity – Today data-centers utilize about 415 terawatt hours of electricity annually. The IEA projects that global demand will hit 945 terawatt hours per year.
• E-Waste – Today, data-centers cycle through servers every 3 to 5 years. When we factor growth of modular systems, this e-waste is projected to grow to 2.5 million tonnes annually.

The above seems daunting, but it only tells one side of the story. The industry globally is aware of above and actively moving and innovating to prevent environmental crisis. Through modifications in the manufacturing process, to projects to reduce electrical and water requirements, to the circular economy. All these are underway now. To get more insight into how the technology sector is aiding the environment, check out AI for the 21st Century category on environment.

AI Future

• Photonics – The way of the future has already begun. It’s a hybrid approach to the chips manufacturing processing. By combining light with silicon, the energy and subsequently the water requirements are greatly reduced. You can read more about this in my article titled, Computer Chip Manufacturing.
• Timelines – The timeline to running full scale optical AI accelerator pipelines, end to end is in the late 2020’s.

Summary

In summary, modular AI architectures and infrastructure will greatly reduce the amount of environmental resources required. However, we also read about the Jevons’ Paradox where we learn that when something becomes more efficient, more, not less, will be used. This is the danger we face right now. While there are projects and processes in place, it will be up to people to make this work. You can read about the holy grail for industry in my article titled Circular Economy.