A new Turing test is needed to evaluate artificial intelligence's real-world knowledge, which could help specialists better understand AI.
According to a study, AI models can perform as well as humans on law exams, answering multiple-choice, short-answer, and essay questions https://doi.org/p89q. However, they struggle with real-world legal tasks.
Some lawyers have learnt that the hard way, and have been fined for filing AI-generated court briefs that misrepresented principles of law and cited non-existent cases.
Author: Chaudhri, principal scientist at Knowledge Systems Research in Sunnyvale, California. Author summary: New benchmarks for AI.