I have to admit I was very critical of the Promethean moment declared by “All you need is attention” (a premise that took ANNs back to the center of AI in a very expensive way) and OpenAI & et als’ star releases. And, as you are aware, I am not the only one. There is a heap of things to criticize about LLMs and the ANNs used to build them. But the fact is they are not going to go away or be thrown away. That’s just not how techology evolves, ever!
Never mind the data scraping. That wasn’t all that different from what Uber did in the shadows for years with their ghost software and willingness to bend the law. And sometimes even break the law. Data scraping was inevitable. It’s like the classic questions: 1. Do I let my landlord know I am going to knock walls down or put in bigger windows? 2. Do I go ahead with what I want to do and pay the consequences, if any, when the landlord finds out? In the case of data scraping the answer is obviously #2. All that data was just too valuable in the eyes of the major LLM builders. I mean, without all the data from the public internet (and private internet too likely) there was no way LLMs could have reach hundreds of billions of token or pseudo parameters to comprehend human language. So let’s look past the data scraping. The damage has been done and the major players affected have either sued or joined the party.
The really big thing in the room with LLMs is the expense of building, running and using these models. According to Stanford’s AI Index (estimated or estimates), the cost of training state-of-the-art AI models is unprecedented. Major evidence cited by Stanford include these numbers: OpenAI’s GPT-4 required $78 million of compute to train, and Google’s Gemini Ultra used $191 million for compute training.
Last year (2023), overall private investment in AI declined but funding for generative AI (i.e., LLMs and their ANNs) shot off the graph at an 8x growth rate from 2022 to $25.2 billion. In 2023, 61 notable AI models were released by U.S.-based institutions. For comparison, the EU released 21 and China 15.
And a new investigation by the Guardian, posted this week, revealed that an LLM internet search process burns 6x the electricity required for a standard non-AI search process. That’s very expensive from an energy perspective, especially given the flaws of LLMs. But that won’t really matter going forward.
The momentum has increased over the past few months and the new target to make money from LLMs is clear on Wall Street. Data centers! And data centers burn electricity 24/7, needs plenty of water or cold air. And we need many more data centers. But why?
The answer lies in a bigger truth or a bigger forest for the tree’s viewpoint. Basically, the latest leap forward (sic) given to humanity by some of the biggest corporations on the planet, is just a snowball rolling downhill in a land of powder. And this powder could also be seen as gun powder in the war for better AI. That’s right! The 2022 Gen AI moment has unleashed an appetite for new AIs of all kinds that just isn’t going away.
The founder of Infosys (India), Nandan Nilekani, recently pointed out that the next big AI snowball will be the application layer or layers that are built on top of the LLMs we have now. And he’s right! Nilekani is totally right about that! The applications built on top of LLMs will be the next wave of Gen AI value, along with related technologies. So, you could say that the Apple OS or Window OS models of the early 80s spawned the applications built to run on these new models (operating system models).
So, more data centers with more Nvidia and other top end GPUs is the future. And that stage, applications made from LLMs, will last a few years no doubt. So more data centers everywhere and damn the training costs or the electricity impact!
Therefore, Gen AI or LLMs are not going to go away. Just too much opportunity now and going forward. And let’s be fair. The new application layer built on top of LLM models will be much more useful that the models themselves. And many of these so-called AI apps will be monetized. Either as pay by the month or as fremium products sold with advertising on the same page. No so different from content. Expert content has few or no ads. General content is plastered with ads. The AI apps of the coming 3-5 years will be like that too.
My company, NexussPlus, is not developing generative AI apps or any kind of LLMs. And yet the AI models we are developing can not ignore the basic value offered by LLM apps that interact with the internet autonomously.
NexussPlus’s thinking is different. In fact, we are as different as you can get when it comes to AI. We are building models that require ultra-low compute costs, can be run on a desktop or smartphone and all trained much more by HI (human intelligence) than the dumb token ideas used by the “All you need is attention” transformer crowd. A big crowd to be sure!
The AI approach NexussPlus is using has a name: the Natural Intelligence (NI) approach. We build from the basics of how a child’s mind evolves from birth to age 14, say. NexussPlus is a company built on NI HI power. That’s right the natural intelligence of human intelligence or human individuals.
Our approach would use some ANNs but not really big ones. And anything we build will be compact and have very few parameters (or tokens) compared to LLMs.
But for now I have to find the money to build what I want to unlease. And for now LLMs and the applications that run on LLMs will continue to dominate the news . . .
That’s all for today and stay tuned for the next essay on AI, NexussPlus, and the similarities between HIs and AIs.
Leave a Reply