Log probabilities (logprobs) provide a window into how confident a language model is about its predictions. In this technical implementation, we demonstrate how to access and interpret logprobs via the OpenAI API, using a series of increasingly difficult multiplication tasks. Our experiment reveals that declining confidence scores can effectively signal
Claude Projects recently introduced a Retrieval-Augmented Generation (RAG) feature that extends its capabilities beyond the standard context window. Unlike traditional approaches that require custom vector databases and API integration, Claude's implementation offers a streamlined, no-code solution for handling large knowledge bases directly within the Projects interface. In this
Tool use (also called function calling) is one of the most powerful capabilities available in modern language models. It allows models to extend beyond text generation by interacting with external systems, databases, or custom logic—making AI agents capable of real-world tasks like checking weather, querying APIs, or running calculations.
To assess whether leading AI models can reliably distinguish between human-written and AI-generated text, we tested GPT-5.2, Claude Sonnet 4.5, and Gemini 3 on a set of cover letters. The models were asked to identify AI-generated content and then to rewrite letters to appear more human-written. The results
In this follow-up experiment, we tested how generative AI models handle handwritten text when provided as image input rather than PDF. Unlike in the earlier test, all models were able to process the images and return readable outputs. Performance Comparison 🏆 Champion as of October 2: Copilot, Gemini 2.5 Pro,
Optical character recognition (OCR) of handwritten text remains a demanding task, particularly once the focus shifts beyond English. In this experiment, we assessed a range of generative AI models on three handwritten text samples—one each in English, French, and Hungarian—to examine cross-linguistic performance. While accuracy was consistently high
In this experiment, we tested Manus AI’s ability to generate a full academic presentation from a research paper. The tool produced a slide deck that was functional and visually adequate, though not every slide was directly usable. The Manus AI interface supported adjustments before downloading—such as modifying layout,
We tested the new Sonnet 4.5 model on a demanding academic task: transforming a research paper into a structured 15-slide outline for a conference presentation. The prompt required not only conceptual rigour and narrative coherence but also attention to visual communication, with clear guidance on where charts, tables, and
In this post we present a comparative evaluation of several generative AI models, focusing on their ability to carry out a straightforward sentiment analysis task. The models were asked to classify customer reviews as either positive or negative, following a simple binary labelling instruction. While the task itself was unambiguous,
When using generative AI for structured data collection, the language of the prompt can make a real difference. In our test with Hungarian population statistics, both Claude Opus 4.1 and Sonnet 4 produced accurate outputs when prompted in Hungarian – but with an English prompt, Sonnet 4 generated rounded figures
In this post we examine how Claude Opus 4.1 can be applied to data collection tasks through a prompt-based approach. The model accessed and structured monthly U.S. unemployment rate data from the Federal Reserve Economic Data (FRED) platform for the period January 2000 to December 2024. The results
In this post we put Notebook LM’s new literature review feature to the test. The model was given six pre-selected sources on a single topic and asked to produce a structured review. The results were not bad at all: the output drew appropriately on the provided references and offered
