Abstract
With the ongoing advances in generative artificial intelligence (GenAI), the initial skepticism surrounding its tools is gradually diminishing. In fact, tools such as ChatGPT, Copilot and similar, are often used in everyday tasks, both in our personal lives and in educational contexts. Educators may use them for content creation, grading exams, or automating repetitive tasks. Students resort to them to better understand a topic, get feedback on an assignment and brainstorm ideas. Research has shown that, if used correctly, these tools can spur and support both teaching and learning. However, these continuous advancements and the increasing number of available tools also require more research to benchmark all these models and, if possible, provide quantifiable indications of which tool is better to use for which specific subtopic. As such, we created FermBench, a dataset of fermentation knowledge, which can be used to benchmark various large language models (LLMs). The models selected for the experiment are: GPT-5 mini, Claude Sonnet 4.5, Gemini 3 Flash, Mistral Small 3.2, and DeepSeek-V3.2. The performance of the models is scored using the LLM-as-a-Judge approach, with GPT 5.2 and Claude Opus 4.5 as judges. The LLMs agree that the model used in the free tier of ChatGPT is the most factually accurate. DeepSeek and Gemini outperform the other models in readability and helpfulness, while Claude and Gemini provide the most concise answers. Overall, we conclude that the quality of the responses generated is satisfactory for educational contexts. We also provide students with practical guidelines on how to be more critical of responses generated by LLMs and to improve their prompts. All code and curated data are open-source.