Colombian Court Rejects Appeal for AI Writing, Then Gets Flagged By Its Own AI Detector

Colombia's Supreme Court Faces Scrutiny Over AI Detection Tool Reliability

Colombia's Supreme Court recently rejected a cassation appeal, citing that it was machine-generated. However, the same AI detection tool used by the court to flag the appeal also indicated that the court's own ruling contained a high percentage of AI-generated text, raising questions about the reliability of such tools.

Court's Decision and Subsequent Findings

The court utilized the Winston AI tool, which reported that the submitted appeal contained only 7% human content, suggesting significant AI influence. Based on this assessment and similar results from other tools, the court dismissed the appeal as inadmissible. Subsequently, attorney Emmanuel Alessio Velasquez tested the court's ruling with the same Winston AI software, which returned a result of 93% AI-generated text. This finding highlighted the methodological fragility of using AI detectors as sole evidence.

Further tests conducted by various individuals yielded similar inconsistencies. GPTZero initially flagged the opening words of the court's verdict as 100% AI-generated. However, when processing a longer version of the text, the same tool reversed its assessment, classifying it as 100% human-generated. This demonstrates the unreliability of these tools for definitive judgments.

Wider Concerns and Expert Opinions

Legal professionals and tech experts have voiced significant concerns regarding the accuracy and consistency of AI detection software. Attorney Andres F. Arango G. reported that a 2019 court filing, predating current large language models, was flagged as 95% AI-generated. Nicolas Buelvas found his 2020 undergraduate thesis also flagged as 100% AI.

Dario Cabrera Montealegre pointed out the inherent contradiction in using AI to detect AI. Carlos Alejandro Torres Pinedo argued that no publicly accessible tool can accurately define the percentage of AI use in text, and the lack of transparency in their source code prevents verification. He questioned how these tools could be used to undermine access to justice.

Technical Limitations and Academic Studies

The technical basis for AI detector failures lies in their measurement of statistical patterns like sentence length, vocabulary predictability, and 'burstiness'—natural variations in human writing. These patterns are also common in formal legal prose, academic writing, and texts by non-native speakers.

Studies corroborate these issues. A 2023 study in Patterns found that over 61% of TOEFL essays by non-native English speakers were misidentified as AI-generated. A systematic review by Weber-Wulff concluded that no available tool is precise or reliable. Turnitin acknowledged higher false positive rates for content below 20% AI. OpenAI itself removed its AI detection tool due to inaccuracies.

Universities have also encountered problems. Vanderbilt disabled Turnitin's AI detector due to an estimated 3,000 false positives annually. The University of Arizona removed AI-detection features after a student faced a grade reduction. UC Davis flagged 17 linguistics students, mostly non-native speakers, in a 2024 case. These tools tend to penalize formal, repetitive, or careful writing styles, characteristic of lawyers, academics, and second-language speakers.

Regulatory and Judicial Context in Colombia

Colombia's Supreme Court's rulings in AC739-2026 and AP760-2026 are among the first judicial decisions in the region to directly address the misuse of generative AI in legal filings. In December 2024, Colombia's judicial branch adopted formal guidelines for AI use. These guidelines permit AI for administrative tasks like drafting emails and organizing agendas, but require careful human review for legal research and drafting procedural documents.

The rules explicitly prohibit using AI for evaluating evidence, interpreting law, or making judicial decisions, emphasizing human judges' responsibility and the need to disclose AI tool usage. These guidelines, documented in the "PCSJA24-12243" agreement, could potentially be used to contest decisions based on unreliable AI detection.