Kleyton V. S. da Costa

The important thing is not to stop questioning. Curiosity has its own reason for existing. (Einstein, A.)

I’m an MSc student in Computer Science at PUC-Rio, advised by Prof. Hélio Côrtes Vieira Lopes. Currently, I work as a AI Researcher at Holistic AI (London, UK) and a Research Engineer at Exacta (PUC-Rio). I am also a contributor to Understanding Artificial Intelligence (UAI) (UAI, IEA-USP).

My research interests include (and is not limited to):

Learning on Graphs
Responsible AI
Algorithms for Optimization

The best way to contact me is by email: kleyton.vsc@gmail.com. I’m always open to interesting discussions and collaboration opportunities. It’s possible check my research on Google Scholar.

news

Jan 30, 2024	We release an updated version of the paper Evaluating explainability for machine learning predictions using model-agnostic metrics
Sep 03, 2023	Our paper Deep learning model fragility and implications for financial stability and regulation was published as Staff Working Paper of Bank of England.
Mar 10, 2023	I started my MSc in Computer Science at Pontifical Catholic University of Rio de Janeiro (PUC-Rio).

latest posts

Mar 02, 2024	Many explanations are wrong, but some are useful
Jun 24, 2023	The Great Step of Art and History in AI Society
Jun 21, 2023	Build your own Bias Measuring and Mitigation dashboard in 5 steps

selected publications

Arxiv

Evaluating explainability for machine learning predictions using model-agnostic metrics

Cristian Munoz , Kleyton Costa , Bernardo Modenesi , and 1 more author

arXiv preprint arXiv:2302.12094, 2024

Abs PDF

Rapid advancements in artificial intelligence (AI) technology have brought about a plethora of new challenges in terms of governance and regulation. AI systems are being integrated into various industries and sectors, creating a demand from decision-makers to possess a comprehensive and nuanced understanding of the capabilities and limitations of these systems. One critical aspect of this demand is the ability to explain the results of machine learning models, which is crucial to promoting transparency and trust in AI systems, as well as fundamental in helping machine learning models to be trained ethically. In this paper, we present novel metrics to quantify the degree of which AI model predictions can be easily explainable by its features. Our metrics summarize different aspects of explainability into scalars, providing a more comprehensive understanding of model predictions and facilitating communication between decision-makers and stakeholders, thereby increasing the overall transparency and accountability of AI systems.
Bank of England

Deep learning model fragility and implications for financial stability and regulation

Rishabh Kumar , Adriano Koshiyama , Kleyton Costa , and 6 more authors

2023

Abs PDF

Deep learning models are being utilised increasingly within finance. Given the models are opaque in nature and are now being deployed for internal and consumer facing decisions, there are increasing concerns around the trustworthiness of their results. We test the stability of predictions and explanations of different deep learning models, which differ between each other only via subtle changes to model settings, with each model trained over the same data. Our results show that the models produce similar predictions but different explanations, even when the differences in model architecture are due to arbitrary factors like random seeds. We compare this behaviour with traditional, interpretable, ’glass-box models’, which show similar accuracies while maintaining stable explanations and predictions. Finally, we show a methodology based on network analysis to compare deep learning models. Our analysis has implications for the adoption and risk management of future deep learning models by regulated institutions.