There’s a buzz around a recent test with a new powerful language model (LLM) that seemed to recognize it was being evaluated, even commenting on the relevance of what it was processing. Some folks are saying this might be a glimpse of metacognition in AI, where it understands its own thought processes. But let’s not get lost in the sci-fi stuff. The real story here is the sheer might of these models and the hefty costs they’re racking up.
As these models get bigger, they’re gaining new abilities but also becoming way more expensive. Think about it like the semiconductor industry, where only the big players can afford the latest and greatest technology. The same could soon go for AI, with only the tech giants and their buddies able to foot the bill for developing top-tier models like GPT-4 and Claude 3.
The price tag to train these beasts is reaching insane heights. We’re talking about up to $200 million for the latest ones. And according to Amodei, a big shot in the AI world, by 2025 or 2026, we could be looking at training costs in the range of $5 to 10 billion. That’s enough to scare off anyone but the biggest players.
It’s a bit like how the semiconductor industry evolved. Back in the day, everyone was making their own chips. But as costs soared, many companies outsourced production. Now, only a handful of big names like TSMC, Intel, and Samsung are leading the charge, while others just piggyback off their tech.
The same could happen with AI. Not every application needs the biggest, baddest model out there. There are smaller, more affordable options like Mistral and Llama3, or Microsoft’s Phi-3. Sure, they might not pack the same punch as GPT-4, but they get the job done for certain tasks.
In the end, these skyrocketing costs could lock out smaller players, stifling innovation. That’s why it’s crucial to support smaller, specialized models and promote collaboration in the AI community. We need to keep the playing field open so that everyone can contribute and benefit from the AI revolution.
