The world of technology moves rapidly, and it is hard to build a sustainable competitive advantage. Just ask Blackberry (business smartphones), Nokia (mobile phones), Yahoo (search), AOL (email and chat), MySpace (social network) and Kodak (film). Technology companies seek to build a “moat”, a durable competitive advantage that is hard to copy. Such a moat can take the form of patents (such as Google’s search algorithm), data and platform effects (like Facebook’s social graph), switching costs (such as SAP’s ERP application), scale (like AWS) or brand (like Apple). So, what is the moat in AI?
There is one company that has a plausible moat: Nvidia with its CUDA, its programming and hardware ecosystem that tightly integrates with its Graphics Processing Units (GPUs) that drive most AI systems today. To be sure, it has challengers, such as Tensor Processing Units (TPUs) from Google, AMD’s MI series and others. However, the deep integration of CUDA with major frameworks like TensorFlow and PyTorch , the high switching costs and the extensive developer base, mean that CUDA is a deep moat for competitors to cross. What about for large language models (LLMs)?
The situation for LLMs is different. All are essentially based on the same underlying concept. This was introduced in the landmark research paper from Google “Attention is all you need”, which introduced the transformer architecture. Google holds the patent on this, though it chose to open source the core paper and reference code, which complicate enforcing licences for its patent should it choose to do so in the future. OpenAI, despite its name, has recently filed for 37 patents (14 granted so far), and there are some other domain-specific patents held by companies, including IBM and Microsoft. However, at this point, every LLM is essentially based on the same core technological concept, which Google open-sourced. This means that the search for a technological moat for an LLM vendor needs to be somewhere else.
One potential moat is the sheer cost of training a foundation model LLM. Training an LLM requires vast amounts of data and processing capacity, with ChatGPT-4 (according to its CEO) costing $100 million to train compared to around $5 million for ChatGPT-3 in 2020. An LLM consists of a set of layered neural networks with parameters (weights) that the model learns in its training. These model weights (and even the number of neural net layers) are proprietary for LLMs like ChatGPT and Claude. Also confidential are their training data and curation methods, fine-tuning techniques and some specific features like the tool use of Claude. Many models have emerged that are truly open source, such as Llama 3 from Meta, Mistral, Qwen, Falcon and more. The closed models can continue to improve and do well at benchmarks, but some of the open-source models like DeepSeek-V3 and Mistral are more efficient than ChatGPT, and this efficiency and low cost (no license fee) may be more important to some customers than having the absolutely latest state-of-the-art model. It is noticeable that many US AI start-ups are now basing their products on Chinese open-source models like Qwen, DeepSeek, Kimi K2, and Wu Dao rather than on ChatGPT.
There is fierce competition amongst LLM vendors, with at least fifty foundation model LLM vendors and hundreds of different LLM models in all. This does not bode well for the long-term profitability of the LLM market. At present, all the leading vendors are almost certainly making a loss on their LLM products. OpenAI, the leading vendor by market share at 74% of the consumer market, made revenues of $3.7 billion in 2024 but lost $5 billion on that, with its own projections not anticipating profitability until at least 2029. Anthropic, with about 32% of the enterprise LLM market, lost $5.6 billion in 2024 on revenues of just under $1 billion. Its 2025 revenues are likely to be nearer $9 billion, but positive cash flow is not expected until 2028, with profitability to follow that. Google has around 20% market share in the enterprise space, but unlike OpenAI and Anthropic, it has vast cash reserves and a highly profitable core business. Other AI companies may be even more vulnerable than the foundation model vendors. Many are essentially wrappers built on top of other LLMs, often in specific industries or domains. Even companies closer to the hardware like Coreweave, which rents out clusters of GPUs, lost $863 million in 2024 (up from a $593 million loss in 2023) on revenues of about $5 billion for 2025 and has $9.7 billion in debt.
The problem for LLM vendors is that their fixed costs are huge, their operating costs are high, and it is a struggle to get enough customers to pay enough money to be sustainable. In late 2025, only 5% of ChatGPT’s 800 million customers actually paid anything. Enterprise adoption of LLMs has been problematic, with 95% of AI projects failing according to MIT, and AI adoption rates being sluggish, and even falling in some reports, such as a huge US Census Bureau survey of US businesses in mid-2025. For sure, there are plenty of industries being impacted by AI, such as graphic design, the music business and the film industry. But this is all based on an LLM industry essentially being subsidised by US venture capital investors. At some point, AI companies will have to charge a true market price. Will there be enough real income to go around? This is particularly an issue given the steady rise of open source LLMs, which are themselves steadily improving and snapping at the heels of ChatGPT and Claude. AI companies are going to have to start digging some deep technological moats soon if they are to be sustainable, and in the world of AI, outside of GPU hardware, moats seem few and far between. There is a danger that some very well-known AI companies at present risk becoming the next Blackberry or Nokia.
