ARGOS

A Human-in-the-Loop ML System for Legal Aid Classification in Espírito Santo

Abstract

We present the modeling of an AI system named ARGOS to classify free legal aid requests in the Court of Justice of Espírito Santo (TJES). The model replicates the decision-making pattern of a judge from the 6th Civil Court of Vila Velha, developed within Challenge 16 of PitchGovES by Atlas.IA, promoted by the State Management Laboratory of Espírito Santo (Lab.ges). The challenge proposed an automated solution to assess applicants' eligibility by cross-referencing multiple data sources to support judicial decisions. The approach consists of three key components: (i) a machine learning model for classifying requests, (ii) an information retrieval engine to support the model, and (iii) an explainability mechanism to justify decisions. A global model for TJES was initially considered but discarded due to legal and technical constraints. Instead, a judge-specific model was developed to align with the principle of Judicial Independence in Decision-Making. Model development began with analysing past judicial
decisions, classifying requests as granted or denied. A significant challenge was data imbalance, as requests were not evenly distributed across classes. To address this, balancing techniques and feature selection methods were applied. Various machine learning algorithms, including decision trees and deep learning models, were tested. The final model balanced accuracy and
interpretability, ensuring magistrates could understand the factors influencing decisions. The second component involved creating an information retrieval engine to supplement applicant data. Given that free legal aid assessment relies on financial capacity, the tool gathered external data. However, access to government databases was restricted, so alternative sources were used, including public income records, social assistance histories, and judicial metadata. The solution comprises four modules: applicant data capture, a dashboard, a decision interface with model training, and a recommendation system. The initial implementation achieved 81.3% balanced accuracy, surpassing the 70% target, demonstrating potential for broader judicial adoption.