Having previously explored the FutureHouse Platform’s agents in tasks such as identifying tailor-made laws and generating a literature review on legislative backsliding, we now directly compare its Falcon agent and OpenAI’s o3. Our aim was to assess their performance on a focused literature search task: compiling a ranked
In recent years, Retrieval-Augmented Generation (RAG) has emerged as a powerful method for enhancing large language models with structured access to external document collections. By combining dense semantic search with contextual text generation, RAG systems have proven particularly useful for tasks such as answering questions based on extensive documentation, enabling
Navigating Horizon Europe’s 2025 Work Programme means dealing with twelve separate documents, each several hundred pages long. These include funding calls, eligibility conditions, strategic priorities, and legal annexes—making it difficult to locate critical information quickly. To address this challenge, poltextLAB developed a domain-specific Custom GPT (Horizon Navigator '
Can a large language model accurately solve a university-level microeconomics exercise on health insurance? We tested Manus AI on a multi-part problem involving demand curves, list prices, out-of-pocket prices, and the calculation of social loss under various insurance schemes – including full insurance, coinsurance, and copayment plans. Not only did the
In our earlier posts, we explored how useful existing Custom GPTs are for academic tasks and explained how to create your own GPT from scratch. This follow-up post puts those insights into practice by introducing QuantiCheck—a Custom GPT we developed specifically to assess the methodological rigour and reproducibility of
In our previous blog post, we examined how useful publicly available Custom GPTs are for academic tasks. While many showed promise, especially for brainstorming or outlining, they often fell short in areas where accuracy, source verification, and methodological transparency are crucial. In short: if your work demands high standards of
In this post, we present a straightforward yet revealing task: classifying 25 philosophical and scientific arguments as either deductive or inductive. We tested Mistral Le Chat on this argument classification task to evaluate its ability to distinguish between these fundamental forms of reasoning. Le Chat demonstrated a solid grasp of
Thematic content analysis is a key method in political text interpretation, but it typically requires human judgement to define categories and trace meaning. In this study, we explored whether Grok 3, a state-of-the-art large language model, can carry out this task autonomously—without predefined themes or external guidance. Using Martin
Can a language model accurately classify argumentative claims without any prior examples or fine-tuning? We put DeepSeek-V3 to the test on a real-world stance classification task involving 200 claims from a structured dataset. The model was asked to determine, for each claim, whether it supported (PRO) or opposed (CON) a
Argument mapping is a useful method for visualising the logical structure of reasoning, particularly in complex or multi-step arguments. In this post, we examine how Claude 3.7 Sonnet performs when prompted to identify the structure of arguments and represent them visually. The model was given five tasks, each involving
Can a generative AI model perform critical discourse analysis at a level comparable to a trained scholar? This case study puts OpenAI’s o3 model to the test by asking it to analyse Tony Blair’s 1996 “stakeholder economy” speech—a foundational text in New Labour rhetoric. The model'
Can large language models return accurate results when asked to find real academic texts written by specific philosophers on a specific topic? In this case, the topic was Marxist theory — and the instruction was to list peer-reviewed publications in which Gilles Deleuze or Michel Foucault offer a critique of it.
