We need a new Turing test to assess AI’s real-world knowledge

Assessing AI's Real-World Knowledge

A new Turing test is needed to evaluate artificial intelligence's real-world knowledge, which could help specialists better understand AI.

According to a study, AI models can perform as well as humans on law exams, answering multiple-choice, short-answer, and essay questions https://doi.org/p89q. However, they struggle with real-world legal tasks.

Some lawyers have learnt that the hard way, and have been fined for filing AI-generated court briefs that misrepresented principles of law and cited non-existent cases.

Author: Chaudhri, principal scientist at Knowledge Systems Research in Sunnyvale, California. Author summary: New benchmarks for AI.

more

Nature Nature — 2025-10-30

More News