The AA Omniscience benchmark of AI knowledge and hallucination was created by Artificial Analysis, an independent, venture-backed benchmarking company based in San Francisco. Originally launched in November 2025, this comprehensive benchmark covers 6,000 questions across six major domains (business, law, health, software, science and the humanities). Artificial Analysis published its quarterly update on October 1st…
“It was a bright cold day in April, and the clocks were striking thirteen.” This is my favourite opening sentence in any novel, which you may recognise as being from George Orwell’s 1984, which was published in 1949. This dystopian novel involves a society with mass surveillance, secret police and a political class that manipulates…
On 30th July 2026 the US State Department gave a presentation at an international development conference in Brazil. It included a map showing various African countries that it had the potential to invest in. This map quickly attracted worldwide attention as it was photographed by some delegates at the conference and sent to various media…
I have written previously about several aspects of AI and cybersecurity, including issues such as prompt injection attacks on LLMs, deepfakes, data poisoning and steganography, where secret messages are hidden inside images. What is the state of play today? There is no doubt that cybersecurity is one of the main concerns around AI, and LLMs…
If you access a large language model (LLM) at home in your web browser or app, then you appear to be directly interacting with the model. In actual fact there is extra software surrounding this conversation. Before your query is passed to the LLM, there are multiple software components that handle your prompt, as shown…
Chess is a game that has fascinated AI researchers. Alan Turing wrote a chess algorithm on paper in 1951, though the computers of the time were too slow to run it. In 1957 an IBM engineer called Alex Bernstein wrote a chess program for an IBM 704 computer, which ran on vacuum tubes. As silicon…
One of the biggest challenges facing large language models (LLMs) is hallucination: generating information that is false but presented as fact. Moreover, these fabrications tend to be plausible and convincing. For example, in a legal context, LLM-generated court documents regularly hallucinate precedent court cases. Many submissions by lawyers have been caught, and law firms and…
Although Large Language Models (LLMs) like ChatGPT are very much in vogue, there are many other flavours of AI. Some are mostly historical, like expert systems, but others are very relevant to the present. Machine learning, for example, is the technology behind most of the predictive maintenance use cases in industry that you will read…
How would you react if your AI costs went up an order of magnitude next month? AI promises productivity improvements in areas such as software code generation, but at what economic cost? Software budgets in large enterprises have traditionally been heavily skewed towards support and maintenance, with 60% or more of software budgets allocated to…
Corporations are investing a fortune in large language models (LLMs) like ChatGPT and Claude based on the promise that they can automate business processes, reduce costs and provide new insights. The reality has proved much trickier than the sales pitch, but just how reliable are these models at basic calculations and logic anyway? A research…