EXPECTATION

Project Abstract

2021-07-01T00:00:00+01:00

Explainable AI (XAI) has recently emerged proposing a set of techniques attempting to explain machine learning (ML) models. The recipients (explainee) are intended to be humans or other intelligent virtual entities. Transparency, trust, and debuging are the underlying features calling for XAI. However, in real-world settings, systems are distributed, data are heterogeneous, the “system” knowledge is bounded, and privacy concerns are subject to variable constraints. Current XAI approaches cannot cope with such requirements. Therefore, there is a need for personalized explainable artificial intelligence. We plan to develop models and mechanisms to reconcile sub-symbolic, symbolic, and semantic representations leveraging on the agent-based paradigm. In particular, the proposed approach combines inter-agent, intra-agent, and human-agent interactions to benefit from both the specialization of ML agents and the establishment of agent collaboration mechanisms, which will integrate heterogeneous knowledge/explanations extracted from efficient black-box AI agents. The project includes the validation of the personalization and heterogeneous knowledge integration approach through a prototype application in the domain of food and nutrition monitoring and recommendation, including the evaluation of agent-human explainability, and the performance of the employed techniques in a collaborative AI environment.

Funded by CHIST-ERA

The Consortium

2021-07-01T00:00:00+01:00

HES-SO

University of Applied Sciences and Arts Western Switzerland

Find out more!

UNIBO

Alma Mater Studiorum Università di Bologna

Find out more!

UNILU

University of Luxembourg

Find out more!

OZU

Özyeğin University

Find out more!

LIST

Luxembourg Institute of Science and Technology

Find out more!

Promo

2021-07-04T00:00:00+01:00

A proud team!

2021-07-04T00:00:00+01:00

The team has multidisciplinary competences sharing the Multi-Agent Systems as common thread.

HES-SO People (from Switzerland)

Prof. Michael I. Schumacher

Full Professor at HES-SO

Personal Homepage

Dr. Davide Calvaresi

Senior researcher at HES-SO

Personal Homepage

Dr. Jean-Paul Calbimonte

Senior researcher at HES-SO

Personal Homepage

Victor Hugo Contreras Ordonez

PhD Student at HES-SO

Personal Homepage

UNIBO People (from Italy)

Prof. Andrea Omicini

Full Professor at UNIBO

Personal Homepage

Giovanni Ciatto

Post-doctoral Research Fellow at UNIBO

Personal Homepage

Andrea Agiollo

PhD Student at UNIBO

Personal Homepage

Federico Sabbatini

Research Fellow at UNIBO (up to Oct. 2021)

Personal Homepage

Matteo Magnini

Research Fellow at UNIBO

Personal Homepage

UNILU People (from Luxemburg)

Prof. Leon Van der Torre

Full Professor at UNILU

Personal Homepage

Joris Hulstijn

Researcher Responsible AI

Personal Homepage

Igor Tchappi

Research Associate

Personal Homepage

LIST People (from Luxemburg)

Dr. Amro Najjar

Full time researcher at LIST

Personal Homepage

OZU People

Prof. Reyhan Aydoğan

Professor at OZU

Personal Homepage

Anil Dogru

PhD Student at OZU

Personal Homepage

Zehra Kesemen

Master’s Student at OZU

Personal Homepage

Berkecan Kocyigit

Master’s Student at OZU

Personal Homepage

Emre Kuru

Bachelor’s Student at OZU

Personal Homepage

(Formerly)

Furkan Canturk

Master’s Student at OZU

Personal Homepage

(Formerly)

Berk Buzcu

Master’s Student at OZU

Personal Homepage

Libraries and Datasets

2022-04-11T00:00:00+01:00

Software Tools

PSyKE: A Python library for the extraction of symbolic knowledge from ML predictors.
PSyKI: A Python library for the injection of symbolic knowledge into ML predictors.
DEXiRE: A Python library for rule extraction from Deep Learning models.
SBG: Synthetic Behavioral Generator (SBG) tool simulates user context, and recipe interactions.
Pro-DEXiRE: A Python library that complements DEXiRE’s rule based explanations with probabilistic reasoning.

Datasets

Recipes dataset: Dataset collected querying GPT API with 7000 recipes.

Yearly Reports

2024-04-08T00:00:00+01:00

Yearly Reports

[1st year] 1st year report
[2nd year] 2nd year report

Deliverables

2022-04-08T00:00:00+01:00

Deliverables

[D1.4] Data Management Plan (DMP)
[D2.1] Tech report on symbolic knowledge extraction and injection
[D2.2] Scientific paper on symbolic knowledge extraction and injection
[D2.3] Software libraries supporting extraction and injection
[D3.1] Technical report detailing the developed models and data integration
[D3.2a] Scientific paper focusing on heterogeneous data integration
[D3.2b] Scientific papers focusing on conflict resolution
[D4.1] Technical report detailing the developed user model and agent-based profiling
[D4.2] Scientific paper about model and mechanisms for conflict detection, impact of different communication modalities employed in the explanation personalization process.
[D5.1] Technical report detailing architectural, ethical, and societal requirements
[D5.2 and D5.3] Technical report detailing the architecture, integration, and development of the overall systems and scientific paper detailing the system’s implementation, evaluation, and results.

Expectation: Personalized Explainable Artificial Intelligence for Decentralized Agents with Heterogeneous Knowledge

2021-07-04T00:00:00+01:00

by Davide Calvaresi, Giovanni Ciatto, Amro Najjar, Reyhan Aydoğan, Leon Van der Torre, Andrea Omicini, and Michael I. Schumacher

Abstract

Explainable AI (XAI) has emerged in recent years as a set of techniques and methodologies to interpret and explain machine learning (ML) predictors. To date, many initiatives have been proposed. Nevertheless, current research efforts mainly focus on methods tailored to specific ML tasks and algorithms, such as image classification and sentiment analysis. However, explanation techniques are still embryotic, and they target principally ML experts rather than heterogeneous end-users. Furthermore, existing solutions assume data to be centralized, homogeneous, and fully/continuously accessible, a situation which rarely holds in practice. Arguably, a system-wide perspective is currently missing. The project named “Personalized Explainable Artificial Intelligence for Decentralized Agents with Heterogeneous Knowledge” (Expectation) aims at overcoming such limitations. This manuscript presents the overall objectives and approach of the Expectation project, focusing on advancing both theoretically and practically the state of the art of XAI towards the construction of personalized explanations in spite of decentralization and heterogeneity of knowledge, agents, and explainee (both humans or virtual). To tackle the challenges posed by personalization, decentralization, and heterogeneity, the project fruitfully combines abstractions, methods, and approaches from the multi-agent systems, knowledge extraction and injection, negotiation, argumentation, and symbolic reasoning communities.

How to access

How to cite

Bibtex

@incollection{expectation-extraamas2021,
	address = {Basel, Switzerland},
	author = {Calvaresi, Davide and Ciatto, Giovanni and Najjar, Amro and Aydo{\u g}an, Reyhan and Van der Torre, Leon and Omicini, Andrea and Schumacher, Michael},
	booktitle = {Explainable and Transparent AI and Multi-Agent Systems. Third International Workshop, EXTRAAMAS 2021, Virtual Event, May 3--7, 2021, Revised Selected Papers},
	doi = {10.1007/978-3-030-82017-6_20},
	editor = {Calvaresi, Davide and Najjar, Amro and Winikoff, Michael and Fr{\"a}mling, Kary},
	isbn = {978-3-030-82016-9},
	isbn-online = {978-3-030-82017-6},
	issn = {0302-9743},
	keywords = {Multi-agent systems; eXplanable AI; Chist-Era IV; Personalisation; Decentralisation; Expectation},
	pages = {331--343},
	publisher = {Springer Nature},
	series = {Lecture Notes in Computer Science},
	subseries = {Lecture Notes in Artificial Intelligence},
	title = {{{\sc Expectation}}: Personalized Explainable Artificial Intelligence for Decentralized Agents with Heterogeneous Knowledge},
	url = {http://link.springer.com/10.1007/978-3-030-82017-6_20},
	volume = 12688,
	year = 2021
}

Towards Explainable Visionary Agents: License to Dare and Imagine

2021-07-04T00:00:00+01:00

by Giovanni Ciatto, Amro Najjar, Jean-Paul Calbimonte, and Davide Calvaresi

Abstract

Since their appearance, computer programs have embodied discipline and structured approaches and methodologies. Yet, to this day, equipping machines with imaginative and creative capabilities remains one of the most challenging and fascinating goals we pursue. Intelligent software agents can behave intelligently in well-defined scenarios, relying on Machine Learning (ML), symbolic reasoning, and their developers' capability of tailoring smart behaviors on the particular application domain(s). However, to forecast all possible scenarios’ evolutions is unfeasible. Thus, intelligent agents should autonomously/creatively adapt to the world’s mutability. This paper investigates the meaning of imagination in the context of cognitive agents, addresses techniques and approaches to let agents autonomously imagine/simulate their course of action and generate explanations supporting it, and formalizes thematic challenges. In particular, we investigated research areas such as: i) reasoning and automatic theorem proving (which is exploited to synthesize novel knowledge via inference), ii) automatic planning and simulation (which is exploited to speculate over alternative courses of action), iii) machine learning and data mining (which are exploited to induce new knowledge from experience), and (iv) biochemical coordination (which keeps imagination dynamic by continuously reorganizing it).

How to access

How to cite

Bibtex

@incollection{imagination-extraamas2021,
	address = {Basel, Switzerland},
	author = {Ciatto, Giovanni and Najjar, Amro and Calbimonte, Jean-Paul and Calvaresi, Davide},
	booktitle = {Explainable and Transparent AI and Multi-Agent Systems. Third International Workshop, EXTRAAMAS 2021, Virtual Event, May 3--7, 2021, Revised Selected Papers},
	doi = {10.1007/978-3-030-82017-6_9},
	editor = {Calvaresi, Davide and Najjar, Amro and Winikoff, Michael and Fr{\"a}mling, Kary},
	isbn = {978-3-030-82016-9},
	isbn-online = {978-3-030-82017-6},
	issn = {0302-9743},
	keywords = {Multi-agent systems; Imagination; BDI; Cognitive agents; XAI},
	pages = {139--157},
	publisher = {Springer Nature},
	series = {Lecture Notes in Computer Science},
	subseries = {Lecture Notes in Artificial Intelligence},
	title = {Towards Explainable Visionary Agents: License to Dare and Imagine},
	url = {http://link.springer.com/10.1007/978-3-030-82017-6_9},
	volume = 12688,
	year = 2021}

Shallow2Deep: Restraining Neural Networks Opacity through Neural Architecture Search

2021-07-04T00:00:00+01:00

by Andrea Agiollo, Giovanni Ciatto, and Andrea Omicini

Abstract

Recently, the Deep Learning (DL) research community has focused on developing efficient and highly performing Neural Networks (NN). Meanwhile, the eXplainable AI (XAI) research community has focused on making Machine Learning (ML) and Deep Learning methods interpretable and transparent, seeking explainability. This work is a preliminary study on the applicability of Neural Architecture Search (NAS) (a sub-field of DL looking for automatic design of NN structures) in XAI. We propose Shallow2Deep, an evolutionary NAS algorithm that exploits local variability to restrain opacity of DL-systems through NN architectures simplification. Shallow2Deep effectively reduces NN complexity – therefore their opacity – while reaching state-of-the-art performances. Unlike its competitors, Shallow2Deep promotes variability of localised structures in NN, helping to reduce NN opacity. The proposed work analyses the role of local variability in NN architectures design, presenting experimental results that show how this feature is actually desirable.

How to access

How to cite

Bibtex

@incollection{shallow2deep-extraamas2021,
	address = {Basel, Switzerland},
	author = {Agiollo, Ansdrea and Ciatto, Giovanni and Omicini, Andrea},
	booktitle = {Explainable and Transparent AI and Multi-Agent Systems. Third International Workshop, EXTRAAMAS 2021, Virtual Event, May 3--7, 2021, Revised Selected Papers},
	doi = {10.1007/978-3-030-82017-6_5},
	editor = {Calvaresi, Davide and Najjar, Amro and Winikoff, Michael and Fr{\"a}mling, Kary},
	isbn = {978-3-030-82016-9},
	isbn-online = {978-3-030-82017-6},
	issn = {0302-9743},
	keywords = {Neural Architecture Search; Evolutionary Algorithm; Opacity; Interpretability},
	pages = {63--82},
	publisher = {Springer Nature},
	series = {Lecture Notes in Computer Science},
	subseries = {Lecture Notes in Artificial Intelligence},
	title = {{\it Shallow2Deep}: Restraining Neural Networks Opacity through Neural Architecture Search},
	url = {http://link.springer.com/10.1007/978-3-030-82017-6_5},
	volume = 12688,
	year = 2021
}

Logic Programming library for Machine Learning: API design and prototype

2023-04-29T00:00:00+01:00

by Giovanni Ciatto, Matteo Castigliò, and Roberta Calegari

Abstract

In this paper we address the problem of hybridising symbolic and sub-symbolic approaches in artificial intelligence, following the purpose of creating flexible and data-driven systems, which are simultaneously comprehensible and capable of automated learning. In particular, we propose a logic API for supervised machine learning, enabling logic programmers to exploit neural networks – among the others – in their programs. Accordingly, we discuss the design and architecture of a library reifying APIs for the Prolog language in the 2P-Kt logic ecosystem. Finally, we discuss a number of snippets aimed at exemplifying the major benefits of our approach when designing hybrid systems.

How to access

URL: http://ceur-ws.org/Vol-3204/paper_12.pdf

How to cite

Bibtex

@inproceedings{logicapiml-cilc2022,
    author = {Ciatto, Giovanni and Castigliò, Matteo and Calegari, Roberta},
    booktitle = {CILC 2022 -- Italian Conference on Computational Logic},
    editor = {Calegari, Roberta and Ciatto, Giovanni and Omicini, Andrea},
    issn = {1613-0073},
    keywords = {logic programming, machine learning, API, 2P-Kt},
    location = {Bologna, Italy},
    numpages = 15,
    pages = {104--118},
    publisher = {CEUR-WS},
    series = {ceurws},
    subseries = {AI*IA Series},
    title = {Logic Programming library for Machine Learning: {API} design and prototype},
    url = {http://ceur-ws.org/Vol-3204/paper_12.pdf},
    urlopenaccess = {http://ceur-ws.org/Vol-3204/paper_12.pdf},
    urlpdf = {http://ceur-ws.org/Vol-3204/paper_12.pdf},
    volume = 3204,
    year = 2022
}

GridEx: An Algorithm for Knowledge Extraction from Black-Box Regressors

2021-07-04T00:00:00+01:00

by Federico Sabbatini, Giovanni Ciatto, and Andrea Omicini

Abstract

Knowledge extraction methods are applied to ML-based predictors to attain explainable representations of their functioning when the lack of interpretable results constitutes a problem. Several algorithms have been proposed for knowledge extraction, mostly focusing on the extraction of either lists or trees of rules. Yet, most of them only support supervised learning – and, in particular, classification – tasks. ITER is among the few rule extraction methods capable of extracting symbolic rules out of sub-symbolic regressors. However, its performance – here intended as the interpretability of the rules it extracts – easily degrades as the complexity of the regression task at hand increases. In this paper we propose GridEx, an extension of the ITER algorithm, aimed at extracting symbolic knowledge – in the form of lists of if-then-else rules – from any sort of sub-symbolic regressor—there including neural networks of arbitrary depth. With respect to ITER, GridEx produces shorter rule lists retaining higher fidelity w.r.t. the original regressor. Furthermore, to demonstrate the feasibility and effectiveness of GridEx, we report several experiments assessing its performance in comparison to both ITER and decision tree regressors, used as benchmarks.

How to access

How to cite

Bibtex

@incollection{gridex-extraamas2021,
	address = {Basel, Switzerland},
	author = {Sabbatini, Federico and Ciatto, Giovanni and Omicini, Andrea},
	booktitle = {Explainable and Transparent AI and Multi-Agent Systems. Third International Workshop, EXTRAAMAS 2021, Virtual Event, May 3--7, 2021, Revised Selected Papers},
	doi = {10.1007/978-3-030-82017-6_2},
	editor = {Calvaresi, Davide and Najjar, Amro and Winikoff, Michael and Fr{\"a}mling, Kary},
	isbn = {978-3-030-82016-9},
	isbn-online = {978-3-030-82017-6},
	issn = {0302-9743},
	issn-online = {1611-3349},
	pages = {18--38},
	publisher = {Springer Nature},
	series = {Lecture Notes in Computer Science},
	subseries = {Lecture Notes in Artificial Intelligence},
	title = {{GridEx}: An Algorithm for Knowledge Extraction from Black-Box Regressors},
	url = {http://link.springer.com/10.1007/978-3-030-82017-6_2},
	volume = 12688,
	year = 2021
}

Graph Neural Networks as the Copula Mundi between Logic and Machine Learning: A Roadmap

2021-10-07T00:00:00+01:00

by Andrea Agiollo, Giovanni Ciatto, and Andrea Omicini

Abstract

Combining machine learning (ML) and computational logic (CL) is hard, mostly because of the inherently-different ways they use to represent knowledge. In fact, while ML relies on fixed-size numeric representations leveraging on vectors, matrices, or tensors of real numbers, CL relies on logic terms and clauses—which are unlimited in size and structure. Graph neural networks (GNN) are a novelty in the ML world introduced for dealing with graph-structured data in a sub-symbolic way. In other words, GNN pave the way towards the application of ML to logic clauses and knowledge bases. However, there are several ways to encode logic knowledge into graphs: which is the best one heavily depends on the specific task at hand. Accordingly, in this paper, we (i) elicit a number of problems from the field of CL that may benefit from many graph-related problems where GNN has been proved effective; (ii) exemplify the application of GNN to logic theories via an end-to-end toy example, to demonstrate the many intricacies hidden behind the technique; (iii) discuss the possible future directions of the application of GNN to CL in general, pointing out opportunities and open issues.

How to access

URL: http://ceur-ws.org/Vol-2963/paper18.pdf

How to cite

Bibtex

@inproceedings{gnn-woa2021,
	author = {Agiollo, Andrea and Ciatto, Giovanni and Omicini, Andrea},
	booktitle = {WOA 2021 -- 22nd Workshop ``From Objects to Agents''},
	editor = {Calegari, Roberta and Ciatto, Giovanni and Denti, Enrico and Omicini, Andrea and Sartor, Giovanni},
	issn = {1613-0073},
	keywords = {Graph Neural Networks, Machine Learning, Embedding, Computational Logic},
	location = {Bologna, Italy},
	month = oct,
	note = {22nd Workshop ``From Objects to Agents'' (WOA 2021), Bologna, Italy, 1--3~} # sep # {~2021. Proceedings},
	numpages = 18,
	pages = {98--115},
	publisher = {Sun SITE Central Europe, RWTH Aachen University},
	series = {CEUR Workshop Proceedings},
	subseries = {AI*IA Series},
	title = {Graph Neural Networks as the Copula Mundi between Logic and Machine Learning: A Roadmap},
	url = {http://ceur-ws.org/Vol-2963/paper18.pdf},
	volume = 2963,
	year = 2021
}

On the Design of PSyKE: A Platform for Symbolic Knowledge Extraction

2021-10-07T00:00:00+01:00

by Federico Sabbatini, Giovanni Ciatto, Roberta Calegari, and Andrea Omicini

How to access

URL: http://ceur-ws.org/Vol-2963/paper14.pdf

Abstract

A common practice in modern explainable AI is to post-hoc explain black-box machine learning (ML) predictors – such as neural networks – by extracting symbolic knowledge out of them, in the form of either rule lists or decision trees. By acting as a surrogate model, the extracted knowledge aims at revealing the inner working of the black box, thus enabling its inspection, representation, and explanation. Various knowledge-extraction algorithms have been presented in the literature so far. Unfortunately, running implementations of most of them are currently either proof of concepts or unavailable. In any case, a unified, coherent software framework supporting them all – as well as their interchange, comparison, and exploitation in arbitrary ML workflows – is currently missing. Accordingly, in this paper we present the design of PSyKE, a platform providing general-purpose support to symbolic knowledge extraction from different sorts of black-box predictors via many extraction algorithms. Notably, PSyKE targets the extraction of symbolic knowledge in logic form, making it possible to extract first-order logic clauses as output. The extracted knowledge is thus both machine- and human- interpretable, and it can be used as a starting point for further symbolic processing—e.g. automated reasoning.

How to cite

Bibtex

@inproceedings{psyke-woa2021,
	articleno = 3,
	author = {Sabbatini, Federico and Ciatto, Giovanni and Calegari, Roberta and Omicini, Andrea},
	booktitle = {WOA 2021 -- 22nd Workshop ``From Objects to Agents''},
	editor = {Calegari, Roberta and Ciatto, Giovanni and Denti, Enrico and Omicini, Andrea and Sartor, Giovanni},
	issn = {1613-0073},
	keywords = {explainable AI, knowledge extraction, interpretable prediction, PSyKE},
	location = {Bologna, Italy},
	month = oct,
	note = {22nd Workshop ``From Objects to Agents'' (WOA 2021), Bologna, Italy, 1--3~} # sep # {~2021. Proceedings},
	numpages = 20,
	pages = {29--48},
	publisher = {Sun SITE Central Europe, RWTH Aachen University},
	series = {CEUR Workshop Proceedings},
	subseries = {AI*IA Series},
	title = {On the Design of {PSyKE}: A Platform for Symbolic Knowledge Extraction},
	url = {http://ceur-ws.org/Vol-2963/paper14.pdf},
	volume = 2963,
	year = 2021
}

2P-Kt: A Logic-Based Ecosystem for Symbolic AI

2021-10-07T00:00:00+01:00

by Giovanni Ciatto, Roberta Calegari, and Andrea Omicini

Abstract

To date, logic-based technologies are either built on top or as extensions of the Prolog language, mostly working as monolithic solutions tailored upon specific inference procedures, unification mechanisms, or knowledge representation techniques. Instead, to maximise their impact, logic-based technologies should support and enable the general-purpose exploitation of all the manifold contributions from logic programming. Accordingly, we present 2P-Kt, a reboot of the tuProlog project offering a general, extensible, and interoperable ecosystem for logic programming and symbolic AI.

How to access

DOI: https://doi.org/10.1016/j.softx.2021.100817

How to cite

Bibtex

@article{2pkt-swx16,
	author = {Ciatto, Giovanni and Calegari, Roberta and Omicini, Andrea},
	doi = {10.1016/j.softx.2021.100817},
	issn = {2352-7110},
	journal = {SoftwareX},
	keywords = {logic programming, artificial intelligence, Prolog, Kotlin, tuProlog},
	month = dec,
	pages = {100817:1--7},
	title = {{\textsf{2}P-Kt}: A Logic-Based Ecosystem for Symbolic {AI}},
	url = {https://www.sciencedirect.com/science/article/pii/S2352711021001126},
	volume = 16,
	year = 2021
}

Towards cooperative argumentation for MAS: an Actor-based approach

2021-10-07T00:00:00+01:00

by Giuseppe Pisano, Roberta Calegari, and Andrea Omicini

Abstract

We discuss the problem of cooperative argumentation in multi-agent systems, focusing on the computational model. An actor-based model is proposed as a first step towards cooperative argumentation in multi-agent systems to tackle distribution issues—illustrating a preliminary fully-distributed version of the argumentation process completely based on message passing.

How to access

URL: http://ceur-ws.org/Vol-2963/paper17.pdf

How to cite

Bibtex

@inproceedings{distributedarg-woa2021,
	author = {Pisano, Giuseppe and Calegari, Roberta and Omicini, Andrea},
	booktitle = {WOA 2021 -- 22nd Workshop ``From Objects to Agents''},
	editor = {Calegari, Roberta and Ciatto, Giovanni and Denti, Enrico and Omicini, Andrea and Sartor, Giovanni},
	issn = {1613-0073},
	keywords = {Argumentation, MAS, cooperative argumentation, distributed argumentation process},
	location = {Bologna, Italy},
	month = oct,
	note = {22nd Workshop ``From Objects to Agents'' (WOA 2021), Bologna, Italy, 1--3~} # sep # {~2021. Proceedings},
	numpages = 16,
	pages = {162--177},
	publisher = {Sun SITE Central Europe, RWTH Aachen University},
	series = {CEUR Workshop Proceedings},
	subseries = {AI*IA Series},
	title = {Towards cooperative argumentation for {MAS}: An actor-based approach},
	url = {http://ceur-ws.org/Vol-2963/paper17.pdf},
	volume = 2963,
	year = 2021
}

On Explainable Negotiations via Argumentation

2022-09-16T00:00:00+01:00

by Contreras, Victor and Aydoğan, Reyhan and Najjar, Amro and Calvaresi, Davide

Abstract

TBD

How to access

URL: http://publications.hevs.ch/index.php/publications/show/2883

How to cite

Bibtex

@incollection{canc-bnaic-2021-explanable-negotiations,
    address = {},
    author = {Contreras, Victor and Aydoğan, Reyhan and Najjar, Amro and Calvaresi, Davide},
    booktitle = {Proceedings of BNAIC 2021},
    doi = {},
    editor = {},
    isbn = {},
    isbn-online = {},
    issn = {},
    keywords = {explainable negotiation},
    pages = {},
    publisher = {ACM},
    series = {},
    subseries = {},
    title = {On Explainable Negotiations via Argumentation},
    url = {http://publications.hevs.ch/index.php/publications/show/2883},
    volume = 00,
    year = 2021
}

A global taxonomy of interpretable AI: unifying the terminology for the technical and social sciences

2022-09-16T00:00:00+01:00

by Graziani, Mara and Dutkiewicz, Lidia and Calvaresi, Davide and Amorim, Jos{'e} Pereira and Yordanova, Katerina and Vered, Mor and Nair, Rahul and Abreu, Pedro Henriques and Blanke, Tobias and Pulignano, Valeria and Prior, John O. and Lauwaert, Lode and Reijers, Wessel and Depeursinge, Adrien and Andrearczyk, Vincent and Müller, Henning

Abstract

Since its emergence in the 1960s, Artificial Intelligence (AI) has grown to conquer many technology products and their fields of application. Machine learning, as a major part of the current AI solutions, can learn from the data and through experience to reach high performance on various tasks. This growing success of AI algorithms has led to a need for interpretability to understand opaque models such as deep neural networks. Various requirements have been raised from different domains, together with numerous tools to debug, justify outcomes, and establish the safety, fairness and reliability of the models. This variety of tasks has led to inconsistencies in the terminology with, for instance, terms such as interpretable, explainable and transparent being often used interchangeably in methodology papers. These words, however, convey different meanings and are “weighted" differently across domains, for example in the technical and social sciences. In this paper, we propose an overarching terminology of interpretability of AI systems that can be referred to by the technical developers as much as by the social sciences community to pursue clarity and efficiency in the definition of regulations for ethical and reliable AI development. We show how our taxonomy and definition of interpretable AI differ from the ones in previous research and how they apply with high versatility to several domains and use cases, proposing a—highly needed—standard for the communication among interdisciplinary areas of AI.

How to access

URL: https://link.springer.com/article/10.1007/s10462-022-10256-8

How to cite

Bibtex

@Article{cdca+air-2022-global-taxonomy-xai,
  author    = {Graziani, Mara and Dutkiewicz, Lidia and Calvaresi, Davide and Amorim, Jos{\'e} Pereira and Yordanova, Katerina and Vered, Mor and Nair, Rahul and Abreu, Pedro Henriques and Blanke, Tobias and Pulignano, Valeria and Prior, John O. and Lauwaert, Lode and Reijers, Wessel and Depeursinge, Adrien and Andrearczyk, Vincent and Müller, Henning},
  journal   = {Artificial Intelligence Review},
  title     = {A global taxonomy of interpretable AI: unifying the terminology for the technical and social sciences},
  year      = {2022},
  pages     = {1--32},
  doi       = {10.1007/s10462-022-10256-8},
  publisher = {Springer},
  url       = {https://link.springer.com/article/10.1007/s10462-022-10256-8},
}

Human-Social Robots Interaction: the blurred line between necessary anthropomorphization and manipulation

2022-09-16T00:00:00+01:00

by Rachele Carli and Amro Najjar and Davide Calvaresi

Abstract

TBD

How to access

How to cite

Bibtex

@inproceedings{CarliNC22,
  author       = {Rachele Carli and
                  Amro Najjar and
                  Davide Calvaresi},
  editor       = {Christoph Bartneck and
                  Takayuki Kanda and
                  Mohammad Obaid and
                  Wafa Johal},
  title        = {Human-Social Robots Interaction: The Blurred Line between Necessary
                  Anthropomorphization and Manipulation},
  booktitle    = {International Conference on Human-Agent Interaction, {HAI} 2022, Christchurch,
                  New Zealand, December 5-8, 2022},
  pages        = {321--323},
  publisher    = {{ACM}},
  year         = {2022},
  url          = {https://doi.org/10.1145/3527188.3563941},
  doi          = {10.1145/3527188.3563941},
  timestamp    = {Mon, 05 Dec 2022 13:35:52 +0100},
  biburl       = {https://dblp.org/rec/conf/hai/CarliNC22.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

The quest of parsimonious XAI: A human-agent architecture for explanation formulation

2022-09-16T00:00:00+01:00

by Yazan Mualla and Igor Tchappi and Timotheus Kampik and Amro Najjar and Davide Calvaresi and Abdeljalil Abbas-Turki and Stéphane Galland and Christophe Nicolle

Abstract

With the widespread use of Artificial Intelligence (AI), understanding the behavior of intelligent agents and robots is crucial to guarantee successful human-agent collaboration since it is not straightforward for humans to understand an agent’s state of mind. Recent empirical studies have confirmed that explaining a system’s behavior to human users fosters the latter’s acceptance of the system. However, providing overwhelming or unnecessary information may also confuse the users and cause failure. For these reasons, parsimony has been outlined as one of the key features allowing successful human-agent interaction with parsimonious explanation defined as the simplest explanation (i.e. least complex) that describes the situation adequately (i.e. descriptive adequacy). While parsimony is receiving growing attention in the literature, most of the works are carried out on the conceptual front. This paper proposes a mechanism for parsimonious eXplainable AI (XAI). In particular, it introduces the process of explanation formulation and proposes HAExA, a human-agent explainability architecture allowing to make it operational for remote robots. To provide parsimonious explanations, HAExA relies on both contrastive explanations and explanation filtering. To evaluate the proposed architecture, several research hypotheses are investigated in an empirical user study that relies on well-established XAI metrics to estimate how trustworthy and satisfactory the explanations provided by HAExA are. The results are analyzed using parametric and non-parametric statistical testing.

How to access

URL: https://www.sciencedirect.com/science/article/pii/S0004370221001247

How to cite

Bibtex

@Article{mtkn+ai-2022-quest-parsimonious-xai,
  author    = {Mualla, Yazan and Tchappi, Igor and Kampik, Timotheus and Najjar, Amro and Calvaresi, Davide and Abbas-Turki, Abdeljalil and Galland, St{\'e}phane and Nicolle, Christophe},
  journal   = {Artificial Intelligence},
  title     = {The quest of parsimonious XAI: A human-agent architecture for explanation formulation},
  year      = {2022},
  month     = jan,
  pages     = {103573},
  volume    = {302},
  doi       = {10.1016/j.artint.2021.103573},
  publisher = {Elsevier},
  url       = {https://www.sciencedirect.com/science/article/pii/S0004370221001247},
}

Explanation-Based Negotiation Protocol for Nutrition Virtual Coaching

2023-04-29T00:00:00+01:00

by Berk Buzcu, Vanitha Varadhajaran, Igor Tchappi, Amro Najjar, Davide Calvaresi and Reyhan Aydoğan

Abstract

People’s awareness about the importance of healthy lifestyles is rising. This opens new possibilities for personalized intelligent health and coaching applications. In particular, there is a need for more than simple recommendations and mechanistic interactions. Recent studies have identified nutrition virtual coaching systems (NVC) as a technological solution, possibly bridging technologies such as recommender, informative, persuasive, and argumentation systems. Enabling NVC to explain recommendations and discuss (argument) dietary solutions and alternative items or behaviors is crucial to improve the transparency of these applications and enhance user acceptability and retain their engagement. This study primarily focuses on virtual agents personalizing the generation of food recipes recommendation according to users’ allergies, eating habits, lifestyles, nutritional values, etc. Although the agent would nudge the user to consume healthier food, users may tend to object in favor of tastier food. To resolve this divergence, we propose a user-agent negotiation interacting over the revision of the recommendation (via feedback and explanations) or convincing (via explainable arguments) the user of its benefits and importance. Finally, the paper presents our initial findings on the acceptability and usability of such a system obtained via tests with real users. Our preliminary experimental results show that the majority of the participants appreciate the ability to express their feedback as well as receive explanations of the recommendations, while there is still room for improvement in the persuasiveness of the explanations.

How to access

DOI: https://doi.org/10.1007/978-3-031-21203-1_2

How to cite

Bibtex

@inproceedings{BuzcuVTNCA22,
  author       = {Berk Buzcu and
                  Vanitha Varadhajaran and
                  Igor Tchappi and
                  Amro Najjar and
                  Davide Calvaresi and
                  Reyhan Aydogan},
  editor       = {Reyhan Aydogan and
                  Natalia Criado and
                  J{\'{e}}r{\^{o}}me Lang and
                  V{\'{\i}}ctor S{\'{a}}nchez{-}Anguix and
                  Marc Serramia},
  title        = {Explanation-Based Negotiation Protocol for Nutrition Virtual Coaching},
  booktitle    = {{PRIMA} 2022: Principles and Practice of Multi-Agent Systems - 24th
                  International Conference, Valencia, Spain, November 16-18, 2022, Proceedings},
  series       = {Lecture Notes in Computer Science},
  volume       = {13753},
  pages        = {20--36},
  publisher    = {Springer},
  year         = {2022},
  url          = {https://doi.org/10.1007/978-3-031-21203-1\_2},
  doi          = {10.1007/978-3-031-21203-1\_2},
  timestamp    = {Mon, 05 Dec 2022 13:35:34 +0100},
  biburl       = {https://dblp.org/rec/conf/prima/BuzcuVTNCA22.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

Integration of Local and Global Features Explanation with Global Rules Extraction and Generation Tools

2023-04-29T00:00:00+01:00

by Contreras Ordoñez Victor Hugo, Davide Calvaresi, and Michael I. Schumacher

Abstract

Widely used in a growing number of domains, Deep Learning predictors are achieving remarkable results. However, the lack of transparency (i.e., opacity) of their inner mechanisms has raised trust and employability concerns. Nevertheless, several approaches fostering models of interpretability and explainability have been developed in the last decade. This paper combines approaches for local feature explanation (i.e., Contextual Importance and Utility – CIU) and global feature explanation (i.e., Explainable Layers) with a rule extraction system, namely ECLAIRE. The proposed pipeline has been tested in four scenarios employing a breast cancer diagnosis dataset. The results show improvements such as the production of more human-interpretable rules and adherence of the produced rules with the original model.

How to access

DOI: https://doi.org/10.1007/978-3-031-15565-9_2

How to cite

Bibtex

@inproceedings{ContrerasSC22,
  author       = {Victor Contreras and
                  Michael Schumacher and
                  Davide Calvaresi},
  editor       = {Davide Calvaresi and
                  Amro Najjar and
                  Michael Winikoff and
                  Kary Fr{\"{a}}mling},
  title        = {Integration of Local and Global Features Explanation with Global Rules
                  Extraction and Generation Tools},
  booktitle    = {Explainable and Transparent {AI} and Multi-Agent Systems - 4th International
                  Workshop, {EXTRAAMAS} 2022, Virtual Event, May 9-10, 2022, Revised
                  Selected Papers},
  series       = {Lecture Notes in Computer Science},
  volume       = {13283},
  pages        = {19--37},
  publisher    = {Springer},
  year         = {2022},
  url          = {https://doi.org/10.1007/978-3-031-15565-9\_2},
  doi          = {10.1007/978-3-031-15565-9\_2},
  timestamp    = {Tue, 18 Oct 2022 22:16:54 +0200},
  biburl       = {https://dblp.org/rec/conf/atal/ContrerasSC22.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

Bidding Support by the Pocket Negotiator Improves Negotiation Outcomes

2023-04-29T00:00:00+01:00

by Reyhan Aydoğan, and Catholijn M. Jonker

Abstract

This paper presents the negotiation support mechanisms provided by the Pocket Negotiator (PN) and an elaborate empirical evaluation of the economic decision support (EDS) mechanisms during the bidding phase of negotiations as provided by the PN. Some of these support mechanisms are offered actively, some passively. With passive support we mean that the user only gets that support by clicking a button, whereas active support is provided without prompting. Our results show, that PN improves negotiation outcomes, counters cognitive depletion, and encourages exploration of potential outcomes. We found that the active mechanisms were used more effectively than the passive ones and, overall, the various mechanisms were not used optimally, which opens up new avenues for research. As expected, the participants with higher negotiation skills outperformed the other groups, but still they benefited from PN support. Our experimental results show that people with enough technical skills and with some basic negotiation knowledge will benefit most from PN support. Our results also show that the cognitive depletion effect is reduced by Pocket Negotiator support. The questionnaire taken after the experiment shows that overall the participants found Pocket Negotiator easy to interact with, that it made them negotiate more quickly and that it improves their outcome. Based on our findings, we recommend to 1) provide active support mechanisms (push) to nudge users to be more effective, and 2) provide support mechanisms that shield the user from mathematical complexities.

How to access

DOI: https://doi.org/10.1007/978-981-99-0561-4_4

How to cite

Bibtex

@inproceedings{AydoganJ22a,
  author       = {Reyhan Aydogan and
                  Catholijn M. Jonker},
  editor       = {Rafik Hadfi and
                  Reyhan Aydogan and
                  Takayuki Ito and
                  Ryuta Arisaka},
  title        = {Bidding Support by the Pocket Negotiator Improves Negotiation Outcomes},
  booktitle    = {Recent Advances in Agent-Based Negotiation: Applications and Competition
                  Challenges, ACAN@IJCAI 2022, Vienna, Austria, July 24, 2022},
  series       = {Studies in Computational Intelligence},
  volume       = {1092},
  pages        = {52--83},
  publisher    = {Springer},
  year         = {2022},
  url          = {https://doi.org/10.1007/978-981-99-0561-4\_4},
  doi          = {10.1007/978-981-99-0561-4\_4},
  timestamp    = {Tue, 28 Mar 2023 19:49:35 +0200},
  biburl       = {https://dblp.org/rec/conf/acan/AydoganJ22a.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

A Survey of Decision Support Mechanisms for Negotiation

2023-04-29T00:00:00+01:00

by Reyhan Aydoğan, and Catholijn M. Jonker

Abstract

This paper introduces a dependency analysis and a categorization of conceptualized and existing economic decision support mechanisms for negotiation. The focus of our survey is on economic decision support mechanisms, although some behavioural support mechanisms were included, to recognize the important work in that area. We categorize support mechanisms from four different aspects: (i) economic versus behavioral decision support, (ii) analytical versus strategical support, (iii) active versus passive support and (iv) implicit versus explicit support. Our survey suggests that active mechanisms would be more effective than passive ones, and that implicit mechanisms can shield the user from mathematical complexities. Furthermore, we provide a list of existing economic support mechanisms.

How to access

DOI: https://doi.org/10.1007/978-981-99-0561-4_3

How to cite

Bibtex

@inproceedings{AydoganJ22b,
  author       = {Reyhan Aydogan and
                  Catholijn M. Jonker},
  editor       = {Rafik Hadfi and
                  Reyhan Aydogan and
                  Takayuki Ito and
                  Ryuta Arisaka},
  title        = {A Survey of Decision Support Mechanisms for Negotiation},
  booktitle    = {Recent Advances in Agent-Based Negotiation: Applications and Competition
                  Challenges, ACAN@IJCAI 2022, Vienna, Austria, July 24, 2022},
  series       = {Studies in Computational Intelligence},
  volume       = {1092},
  pages        = {30--51},
  publisher    = {Springer},
  year         = {2022},
  url          = {https://doi.org/10.1007/978-981-99-0561-4\_3},
  doi          = {10.1007/978-981-99-0561-4\_3},
  timestamp    = {Tue, 28 Mar 2023 19:49:35 +0200},
  biburl       = {https://dblp.org/rec/conf/acan/AydoganJ22.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

A DEXiRE for Extracting Propositional Rules from Neural Networks via Binarization

2023-04-29T00:00:00+01:00

by Contreras Ordoñez Victor Hugo, Niccolo Marini, Lora Fanda, Gaetano Manzo, Yazan Mualla, Jean-Paul Calbimonte, Michael I. Schumacher, and Davide Calvaresi

Abstract

Background: Despite the advancement in eXplainable Artificial Intelligence, the explanations provided by model-agnostic predictors still call for improvements (i.e., lack of accurate descriptions of predictors’ behaviors). Contribution: We present a tool for Deep Explanations and Rule Extraction (DEXiRE) to approximate rules for Deep Learning models with any number of hidden layers. Methodology: DEXiRE proposes the binarization of neural networks to induce Boolean functions in the hidden layers, generating as many intermediate rule sets. A rule set is inducted between the first hidden layer and the input layer. Finally, the complete rule set is obtained using inverse substitution on intermediate rule sets and first-layer rules. Statistical tests and satisfiability algorithms reduce the final rule set’s size and complexity (filtering redundant, inconsistent, and non-frequent rules). DEXiRE has been tested in binary and multiclass classifications with six datasets having different structures and models. Results: The performance is consistent (in terms of accuracy, fidelity, and rule length) with respect to the state-of-the-art rule extractors (i.e., ECLAIRE). Moreover, compared with ECLAIRE, DEXiRE has generated shorter rules (i.e., up to 74% fewer terms) and has shortened the execution time (improving up to 197% in the best-case scenario). Conclusions: DEXiRE can be applied for binary and multiclass classification of deep learning predictors with any number of hidden layers. Moreover, DEXiRE can identify the activation pattern per class and use it to reduce the search space for rule extractors (pruning irrelevant/redundant neurons)—shorter rules and execution times with respect to ECLAIRE.

How to access

How to cite

Bibtex


@Article{electronics11244171,
  AUTHOR = {Contreras, Victor and Marini, Niccolo and Fanda, Lora and Manzo, Gaetano and Mualla, Yazan and Calbimonte, Jean-Paul and Schumacher, Michael and Calvaresi, Davide},
  TITLE = {A DEXiRE for Extracting Propositional Rules from Neural Networks via Binarization},
  JOURNAL = {Electronics},
  VOLUME = {11},
  YEAR = {2022},
  NUMBER = {24},
  ARTICLE-NUMBER = {4171},
  URL = {https://www.mdpi.com/2079-9292/11/24/4171},
  ISSN = {2079-9292},
  DOI = {10.3390/electronics11244171}
}

Ethical and legal considerations for nutrition virtual coaches

2023-04-29T00:00:00+01:00

by Davide Calvaresi, Rachele Carli, Jean-Gabriel Piguet, Contreras Ordoñez Victor Hugo, Gloria Luzzani, Amro Najjar, Jean-Paul Calbimonte, and Michael I. Schumacher

Abstract

Choices and preferences of individuals are nowadays increasingly influenced by countless inputs and recommendations provided by artificial intelligence-based systems. The accuracy of recommender systems (RS) has achieved remarkable results in several domains, from infotainment to marketing and lifestyle. However, in sensitive use-cases, such as nutrition, there is a need for more complex dynamics and responsibilities beyond conventional RS frameworks. On one hand, virtual coaching systems (VCS) are intended to support and educate the users about food, integrating additional dimensions w.r.t. the conventional RS (i.e., leveraging persuasion techniques, argumentation, informative systems, and recommendation paradigms) and show promising results. On the other hand, as of today, VCS raise unexplored ethical and legal concerns. This paper discusses the need for a clear understanding of the ethical/legal-technological entanglements, formalizing 21 ethical and ten legal challenges and the related mitigation strategies. Moreover, it elaborates on nutrition sustainability as a further nutrition virtual coaches dimension for a better society.

How to access

DOI: https://doi.org/10.1007/s43681-022-00237-6

How to cite

Bibtex

@article{CalvaresiCPCL+2022,
  doi = {10.1007/s43681-022-00237-6},
  url = {https://doi.org/10.1007/s43681-022-00237-6},
  year = {2022},
  month = nov,
  publisher = {Springer Science and Business Media {LLC}},
  author = {Davide Calvaresi and Rachele Carli and Jean-Gabriel Piguet and Victor H. Contreras and Gloria Luzzani and Amro Najjar and Jean-Paul Calbimonte and Michael Schumacher},
  title = {Ethical and legal considerations for nutrition virtual coaches},
  journal = {{AI} and Ethics}
}

Risk and Exposure of XAI in Persuasion and Argumentation: The case of Manipulation

2023-04-29T00:00:00+01:00

by Rachele Carli, Amro Najjar, and Davide Calvaresi

Abstract

In the last decades, Artificial intelligence (AI) systems have been increasingly adopted in assistive (possibly collaborative) decision-making tools. In particular, AI-based persuasive technologies are designed to steer/influence users’ behaviour, habits, and choices to facilitate the achievement of their own - predetermined - goals. Nowadays, the inputs received by the assistive systems leverage heavily AI data-driven approaches. Thus, it is imperative to have transparent and understandable (to the user) both the process leading to the recommendations and the recommendations. The Explainable AI (XAI) community has progressively contributed to “opening the black box”, ensuring the interaction’s effectiveness, and pursuing the safety of the individuals involved. However, principles and methods ensuring the efficacy and information retain on the human have not been introduced yet. The risk is to underestimate the context dependency and subjectivity of the explanations’ understanding, interpretation, and relevance. Moreover, even a plausible (and possibly expected) explanation can lead to an imprecise or incorrect outcome or its understanding. This can lead to unbalanced and unfair circumstances, such as giving a financial advantage to the system owner/provider and the detriment of the user.

This paper highlights that the sole explanations - especially in the context of persuasive technologies - are not self-sufficient to protect users’ psychological and physical integrity. Conversely, explanations could be misused, becoming themselves a tool of manipulation. Therefore, we suggest characteristics safeguarding the explanation from being manipulative and legal principles to be used as criteria for evaluating the operation of XAI systems, both from an ex-ante and ex-post perspective.

How to access

DOI: https://doi.org/10.1007/978-3-031-15565-9_13

How to cite

Bibtex

@inproceedings{CarliNC22,
  author       = {Rachele Carli and
                  Amro Najjar and
                  Davide Calvaresi},
  editor       = {Davide Calvaresi and
                  Amro Najjar and
                  Michael Winikoff and
                  Kary Fr{\"{a}}mling},
  title        = {Risk and Exposure of {XAI} in Persuasion and Argumentation: The case
                  of Manipulation},
  booktitle    = {Explainable and Transparent {AI} and Multi-Agent Systems - 4th International
                  Workshop, {EXTRAAMAS} 2022, Virtual Event, May 9-10, 2022, Revised
                  Selected Papers},
  series       = {Lecture Notes in Computer Science},
  volume       = {13283},
  pages        = {204--220},
  publisher    = {Springer},
  year         = {2022},
  url          = {https://doi.org/10.1007/978-3-031-15565-9\_13},
  doi          = {10.1007/978-3-031-15565-9\_13},
  timestamp    = {Tue, 18 Oct 2022 22:16:54 +0200},
  biburl       = {https://dblp.org/rec/conf/atal/CarliNC22.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

Hypercube-Based Methods for Symbolic Knowledge Extraction: Towards a Unified Model

2023-04-29T00:00:00+01:00

by Federico Sabbatini, Giovanni Ciatto, Roberta Calegari, and Andrea Omicini

Abstract

Symbolic knowledge-extraction (SKE) algorithms proposed by the XAI community to obtain human-intelligible explanations for opaque machine learning predictors are currently being studied and developed with growing interest, also in order to achieve believability in interactions. However, choosing the most adequate extraction procedure amongst the many existing in the literature is becoming more and more challenging, as the amount of available methods increases. In fact, most of the proposed algorithms come with constraints over their applicability. In this paper we focus upon a quite general class of SKE techniques, namely hypercube-based methods. Despite being commonly considered regression-specific, we discuss why hypercube-based SKE methods are flexible enough to deal with classification problems as well. More generally, we propose a common generalised model for hypercube-based methods, and we show how they can be exploited to perform SKE on datasets, predictors, or learning tasks of any sort.

How to access

URL: http://ceur-ws.org/Vol-3261/paper4.pdf

How to cite

Bibtex

@incollection{hypercube-woa2022,
    author = {Sabbatini, Federico and Ciatto, Giovanni and Calegari, Roberta and Omicini, Andrea},
    booktitle = {WOA 2022 -- 23rd Workshop ``From Objects to Agents''},
    dblp = {conf/woa/SabbatiniCCO22},
    editor = {Ferrando, Angelo and Mascardi, Viviana},
    iris = {11585/899358},
    issn = {1613-0073},
    keywords = {Explainable AI; Knowledge extraction; Interpretable prediction; PSyKE},
    month = nov,
    numpages = 13,
    pages = {48--60},
    publisher = {Sun SITE Central Europe, RWTH Aachen University},
    scholar = {8614662013642803891},
    series = {CEUR Workshop Proceedings},
    subseries = {AIxIA Series},
    title = {Hypercube-Based Methods for Symbolic Knowledge Extraction: Towards a Unified Model},
    url = {http://ceur-ws.org/Vol-3261/paper4.pdf},
    urlopenaccess = {http://ceur-ws.org/Vol-3261/paper4.pdf},
    urlpdf = {http://ceur-ws.org/Vol-3261/paper4.pdf},
    volume = 3261,
    year = 2022
}

On the Design of PSyKI: a Platform for Symbolic Knowledge Injection into Sub-Symbolic Predictors

2023-04-29T00:00:00+01:00

by Matteo Magnini, Giovanni Ciatto, and Andrea Omicini

Abstract

A long-standing ambition in artificial intelligence is to integrate predictors' inductive features (i.e., learning from examples) with deductive capabilities (i.e., drawing inferences from prior symbolic knowledge). Many algorithms methods in the literature support injection of prior symbolic knowledge into predictors, generally following the purpose of attaining better (i.e., more effective or efficient w.r.t. predictive performance) predictors. However, to the best of our knowledge, running implementations of these algorithms are currently either proof of concepts or unavailable in most cases. Moreover, a unified, coherent software framework supporting them as well as their interchange, comparison and exploitation in arbitrary ML workflows is currently missing. Accordingly, in this paper we present PSyKI, a platform providing general-purpose support to symbolic knowledge injection into predictors via different algorithms.

How to access

DOI: https://doi.org/10.1007/978-3-031-15565-9_6

How to cite

Bibtex

@incollection{psyki-extraamas2022,
    author = {Magnini, Matteo and Ciatto, Giovanni and Omicini, Andrea},
    booktitle = {Explainable and Transparent AI and Multi-Agent Systems},
    chapter = 6,
    dblp = {conf/atal/MagniniCO22},
    doi = {10.1007/978-3-031-15565-9_6},
    editor = {Calvaresi, Davide and Najjar, Amro and Winikoff, Michael and Främling, Kary},
    eisbn = {978-3-031-15565-9},
    eissn = {1611-3349},
    iris = {11585/899511},
    isbn = {978-3-031-15564-2},
    issn = {0302-9743},
    keywords = {Symbolic Knowledge Injection, Explainable AI, XAI, Neural Networks, PSyKI},
    note = {4th International Workshop, EXTRAAMAS 2022, Virtual Event, May 9--10, 2022, Revised Selected Papers},
    pages = {90--108},
    publisher = {Springer},
    scholar = {7587528289517313138},
    scopus = {2-s2.0-85138317005},
    series = {Lecture Notes in Computer Science},
    title = {On the Design of {PSyKI}: a Platform for Symbolic Knowledge Injection into Sub-Symbolic Predictors},
    url = {https://link.springer.com/chapter/10.1007/978-3-031-15565-9_6},
    urlpdf = {https://link.springer.com/content/pdf/10.1007/978-3-031-15565-9_6.pdf},
    volume = 13283,
    wos = {000870042100006},
    year = 2022
}

KINS: Knowledge Injection via Network Structuring

2023-04-29T00:00:00+01:00

by Matteo Magnini, Giovanni Ciatto, and Andrea Omicini

Abstract

We propose a novel method to inject symbolic knowledge in form of Datalog formulæ into neural networks (NN), called KINS (Knowledge Injection via Network Structuring). The idea behind our method is to extend NN internal structure with ad-hoc layers built out the injected symbolic knowledge. KINS does not constrain NN to any specific architecture, neither requires logic formulæ to be ground. Moreover, it is robust w.r.t. both lack of data and imperfect/incomplete knowledge. Experiments are reported to demonstrate the potential of KINS.

How to access

URL: http://ceur-ws.org/Vol-3204/paper_25.pdf

How to cite

Bibtex

@inproceedings{kins-cilc2022,
    author = {Magnini, Matteo and Ciatto, Giovanni and Omicini, Andrea},
    booktitle = {CILC 2022 -- Italian Conference on Computational Logic},
    dblp = {conf/cilc/MagniniCO22},
    editor = {Calegari, Roberta and Ciatto, Giovanni and Omicini, Andrea},
    iris = {11585/899494},
    issn = {1613-0073},
    keywords = {neural network; explainable AI; symbolic knowledge injection; KINS; PSyKI},
    location = {Bologna, Italy},
    numpages = 14,
    pages = {254--267},
    publisher = {CEUR-WS},
    scholar = {10469078385425944401},
    scopus = {2-s2.0-85138240764},
    series = {CEUR Workshop Proceedings},
    subseries = {AI*IA Series},
    title = {{KINS}: Knowledge Injection via Network Structuring},
    url = {http://ceur-ws.org/Vol-3204/paper_25.pdf},
    urlopenaccess = {http://ceur-ws.org/Vol-3204/paper_25.pdf},
    urlpdf = {http://ceur-ws.org/Vol-3204/paper_25.pdf},
    volume = 3204,
    year = 2022
}

Semantic Web-Based Interoperability for Intelligent Agents with PSyKE

2023-04-29T00:00:00+01:00

by Federico Sabbatini, Giovanni Ciatto, and Andrea Omicini

Abstract

Modern distributed systems require communicating agents to agree on a shared, formal semantics for the data they exchange and operate upon. The Semantic Web offers tools to encode semantics in the form of ontologies, where data is represented in the form knowledge graphs (KG). Applying such tools to intelligent agents equipped with machine learning (ML) capabilities is of particular interest, as it may enable a higher degree of interoperability among heterogeneous agents. Indeed, inputs and outputs of ML models can be formalised through ontologies, while the data they operate upon can be represented as KG.

In this paper we explore the combination of Semantic Web tools with knowledge extraction-that is, a research line aimed at extracting intelligible rules mimicking the behaviour of ML predictors, with the purpose of explaining their behaviour. Along this line, we study whether and to what extent ontologies and KG can be exploited as both the source and the outcome of a rule extraction procedure. In other words, we investigate the extraction of semantic rules out of sub-symbolic predictors trained upon data as KG-possibly adhering to some ontology. In doing so, we extend our PSyKE framework for rule extraction with Semantic Web support. In practice, we make PSyKE able to (i) train ML predictors out of OWL ontologies and RDF knowledge graphs, and (ii) extract semantic knowledge out of them, in the form of SWRL rules. A discussion among the major benefits and issues of our approach is provided, along with a description of the overall workflow.

How to access

DOI: https://doi.org/10.1007/978-3-031-15565-9_8

How to cite

Bibtex

@incollection{swpsyke-extraamas2022,
    author = {Sabbatini, Federico and Ciatto, Giovanni and Omicini, Andrea},
    booktitle = {Explainable and Transparent AI and Multi-Agent Systems},
    chapter = 8,
    dblp = {conf/atal/SabbatiniCO22},
    doi = {10.1007/978-3-031-15565-9_8},
    editor = {Calvaresi, Davide and Najjar, Amro and Winikoff, Michael and Främling, Kary},
    iris = {11585/899474},
    isbn = {978-3-031-15564-2},
    keywords = {Symbolic Knowledge Injection, Explainable AI, XAI, Neural Networks, PSyKI},
    numpages = 19,
    pages = {124--142},
    publisher = {Springer},
    scholar = {11339767386934277898},
    scopus = {2-s2.0-85140488560},
    series = {Lecture Notes in Computer Science},
    subtitle = {4th International Workshop, EXTRAAMAS 2022, Virtual Event, May 9–10, 2022, Revised Selected Papers},
    title = {Semantic Web-Based Interoperability for Intelligent Agents with {PSyKE}},
    url = {http://link.springer.com/10.1007/978-3-031-15565-9_8},
    urlpdf = {https://link.springer.com/content/pdf/10.1007/978-3-031-15565-9_8.pdf},
    volume = 13283,
    wos = {000870042100008},
    year = 2022
}

A view to a KILL: Knowledge Injection via Lambda Layer

2023-04-29T00:00:00+01:00

by Matteo Magnini, Giovanni Ciatto, and Andrea Omicini

Abstract

We propose KILL (Knowledge Injection via Lambda Layer) as a novel method for the injection of symbolic knowledge into neural networks (NN) allowing data scientists to control what the network should (not) learn. Unlike other similar approaches, our method does not (i) require ground input formulae, (ii) impose any constraint on the NN undergoing injection, (iii) affect the loss function of the NN. Instead, it acts directly at the backpropagation level, by increasing penalty whenever the NN output is violating the injected knowledge. Experiments are reported to demonstrate the potential (and limits) of our approach.

How to access

DOI: http://ceur-ws.org/Vol-3261/paper5.pdf

How to cite

Bibtex

@incollection{kill-woa2022,
    author = {Magnini, Matteo and Ciatto, Giovanni and Omicini, Andrea},
    booktitle = {WOA 2022 -- 23rd Workshop ``From Objects to Agents''},
    dblp = {conf/woa/MagniniCO22},
    editor = {Ferrando, Angelo and Mascardi, Viviana},
    iris = {11585/899373},
    issn = {1613-0073},
    keywords = {symbolic knowledge injection; AI; ML; neural networks; KILL; PSyKI},
    month = nov,
    numpages = 16,
    pages = {61--76},
    publisher = {Sun SITE Central Europe, RWTH Aachen University},
    scholar = {514437633827732665},
    scopus = {2-s2.0-85142481549},
    series = {CEUR Workshop Proceedings},
    subseries = {AIxIA Series},
    title = {A view to a {KILL}: Knowledge Injection via Lambda Layer},
    url = {http://ceur-ws.org/Vol-3261/paper5.pdf},
    urlopenaccess = {http://ceur-ws.org/Vol-3261/paper5.pdf},
    urlpdf = {http://ceur-ws.org/Vol-3261/paper5.pdf},
    volume = 3261,
    year = 2022
}

Symbolic Knowledge Extraction for Explainable Nutritional Recommenders

2023-04-29T00:00:00+01:00

by Matteo Magnini, Giovanni Ciatto, Furkan Canturk, Reyhan Aydoğan, and Andrea Omicini

Abstract

Background and objective This paper focuses on nutritional recommendation systems (RS), i.e. AI-powered automatic systems providing users with suggestions about what to eat to pursue their weight/body shape goals. A trade-off among (potentially) conflictual requirements must be taken into account when designing these kinds of systems, there including: (i) adherence to experts’ prescriptions, (ii) adherence to users’ tastes and preferences, (iii) explainability of the whole recommendation process. Accordingly, in this paper we propose a novel approach to the engineering of nutritional RS, combining machine learning and symbolic knowledge extraction to profile users—hence harmonising the aforementioned requirements.

Methods Our contribution focuses on the data processing workflow. Stemming from neural networks (NN) trained to predict user preferences, we use CART Breiman et al.(1984) to extract symbolic rules in Prolog Körner et al.(2022) form, and we combine them with expert prescriptions brought in similar form. We can then query the resulting symbolic knowledge base via logic solvers, to draw explainable recommendations.

Results Experiments are performed involving a publicly available dataset of 45,723 recipes, plus 12 synthetic datasets about as many imaginary users, and 6 experts’ prescriptions. Fully-connected 4-layered NN are trained on those datasets, reaching ∼86% test-set accuracy, on average. Extracted rules, in turn, have ∼80% fidelity w.r.t. those NN. The resulting recommendation system has a test-set precision of ∼74%. The symbolic approach makes it possible to devise how the system draws recommendations.

Conclusions Thanks to our approach, intelligent agents may learn users’ preferences from data, convert them into symbolic form, and extend them with experts’ goal-directed prescriptions. The resulting recommendations are then simultaneously acceptable for the end user and adequate under a nutritional perspective, while the whole process of recommendation generation is made explainable.

How to access

How to cite

Bibtex

@article{skerecommender-cmbp2023,
    articleno = 107536,
    author = {Magnini, Matteo and Ciatto, Giovanni and Cantürk, Furkan and Aydoǧan, Reyhan and Omicini, Andrea},
    doi = {10.1016/j.cmpb.2023.107536},
    iris = {11585/923772},
    issn = {0169-2607},
    journal = {Computer Methods and Programs in Biomedicine},
    keywords = {explainable artificial intelligence, symbolic knowledge extraction, recommendation systems, nutrition, neural networks},
    month = jun,
    numpages = 32,
    pubmed = {37060685},
    scholar = {14455392383017605572},
    scopus = {2-s2.0-85152230884},
    title = {Symbolic Knowledge Extraction for Explainable Nutritional Recommenders},
    url = {https://www.sciencedirect.com/science/article/pii/S0169260723002018},
    volume = 235,
    year = 2023
}

Reinterpreting Vulnerability to Tackle Deception in Principles-Based XAI for Human-Computer Interaction

2023-04-29T00:00:00+01:00

by Rachele Carli, and Davide Calvaresi

Abstract

TBD

How to access

TBD

How to cite

Bibtex

TBD

A General-Purpose Protocol for Multi-Agent based Explanations

2024-03-07T00:00:00+01:00

by Giovanni Ciatto, Matteo Magnini, Berk Buzcu, Reyhan Aydoğan, and Andrea Omicini

Abstract

Building on prior works on explanation negotiation protocols, this paper proposes a general-purpose protocol for multi-agent systems where recommender agents may need to provide explanations for their recommendations. The protocol specifies the roles and responsibilities of the explainee and the explainer agent and the types of information that should be exchanged between them to ensure a clear and effective explanation. However, it does not prescribe any particular sort of recommendation or explanation, hence remaining agnostic w.r.t. such notions. Novelty lays in the extended support for both ordinary and contrastive explanations, as well as for the situation where no explanation is needed as none is requested by the explainee. Accordingly, we formally present and analyse the protocol, motivating its design and discussing its generality. We also discuss the reification of the protocol into a re-usable software library, namely PyXMas, which is meant to support developers willing to build explainable MAS leveraging our protocol. Finally, we discuss how custom notions of recommendation and explanation can be easily plugged into PyXMas.

How to access

DOI: http://dx.doi.org/10.1007/978-3-031-40878-6_3

How to cite

Bibtex

@incollection{explanationprotocol-extraamas2023,
    author = {Ciatto, Giovanni and Magnini, Matteo and Bezcu, Berk and Aydoǧan, Reyhan and Omicini, Andrea},
    booktitle = {Explainable and Transparent {AI} and Multi-Agent Systems},
    chapter = 3,
    dblp = {conf/extraamas/CiattoMBAO23},
    doi = {10.1007/978-3-031-40878-6_3},
    editor = {Calvaresi, Davide and Najjar, Amro and Omicini, Andrea and Aydoǧan, Reyhan and Carli, Rachele and Ciatto, Giovanni and Mualla, Yazan and Främling, Kary},
    iris = {11585/940656},
    isbn = {978-3-031-40878-6},
    issn = {0302-9743},
    keywords = {XAI, Recommender Systems, Multi-agent systems, Explanation protocols},
    month = sep,
    numpages = 22,
    pages = {38--58},
    publisher = {Springer},
    scholar = {5371175139312621961},
    scopus = {2-s2.0-85172214167},
    series = {Lecture Notes in Computer Science},
    subseries = {Lecture Notes in Artificial Intelligence},
    title = {A General-Purpose Protocol for Multi-Agent based Explanations},
    url = {http://link.springer.com/10.1007/978-3-031-40878-6_3},
    volume = 14127,
    year = 2023
}

Explanation Generation via Decompositional Rules Extraction for Head and Neck Cancer Classification

2023-04-29T00:00:00+01:00

by Contreras Ordoñez Victor Hugo, Andrea Bagante, Niccolò Marini, Michael I. Schumacher, Vincent Andrearczyk and Davide Calvaresi

Abstract

TBD

How to access

TBD

How to cite

Bibtex

TBD

Metrics for Evaluating Explainable Recommender Systems

2023-04-29T00:00:00+01:00

by Joris Hulstijn, Igor Tchappi, Amro Najjar, and Reyhan Aydoğan

Abstract

Recommender systems aim to support their users by reducing information overload so that they can make better decisions. Recommender systems must be transparent, so users can form mental models about the system’s goals, internal state, and capabilities, that are in line with their actual design. Explanations and transparent behaviour of the system should inspire trust and, ultimately, lead to more persuasive recommendations. Here, explanations convey reasons why a recommendation is given or how the system forms its recommendations. This paper focuses on the question how such claims about effectiveness of explanations can be evaluated. Accordingly, we investigate various models that are used to assess the effects of explanations and recommendations. We discuss objective and subjective measurement and argue that both are needed. We define a set of metrics for measuring the effectiveness of explanations and recommendations. The feasibility of using these metrics is discussed in the context of a specific explainable recommender system in the food and health domain.

How to access

URL: https://link.springer.com/chapter/10.1007/978-3-031-40878-6_12

How to cite

Bibtex

@inproceedings{HulstijnTNA23,
  author       = {Joris Hulstijn and
                  Igor Tchappi and
                  Amro Najjar and
                  Reyhan Aydogan},
  editor       = {Davide Calvaresi et al},
  title        = {Metrics for Evaluating Explainable Recommender Systems},
  booktitle    = {Explainable and Transparent {AI} and Multi-Agent Systems - 5th International
                  Workshop, (EXTRAAMAS 2023), London, UK. 
                  Papers},
  series       = {Lecture Notes in Computer Science},
  volume       = {14127},
  pages        = {212--230},
  publisher    = {Springer},
  year         = {2023},
  url          = {https://link.springer.com/chapter/10.1007/978-3-031-40878-6_12},
  doi          = {10.1007/978-3-031-40878-6_12}
}

Computational Accountability

2023-11-06T00:00:00+01:00

by Joris Hulstijn

Abstract

Automated decision making systems take decisions that matter. Some human or legal person remains responsible. Looking back, that person is accountable for the decisions made by the system, and may even be liable in case of damages. That puts constraints on the way in which decision making systems are designed, and how they are deployed in organizations. In this paper, we analyze computational accountability in three steps. First, being accountable is analyzed as a relationship between an actor deploying the system and a critical forum of subjects, users, experts and developers. Second, we discuss system design. In principle, evidence must be collected about the decision rule and the case data that were applied. However, many AI algorithms are not interpretable for humans. Alternatively, internal controls must ensure that a system uses valid algorithms and reliable data sets for training, which are appropriate for the application domain. Third, we discuss the governance model: roles, responsibilities, procedures and infrastructure, to ensure effective operation of these controls. The paper ends with a case study in the IT audit domain, to illustrate practical feasibility.

How to access

URL: https://dl.acm.org/doi/10.1145/3594536.3595122

How to cite

Bibtex

@inproceedings{Hulstijn23,
  author       = {Joris Hulstijn},
  editor       = {Matthias Grabmair and
                  Francisco Andrade and
                  Paulo Novais},
  title        = {Computational Accountability},
  booktitle    = {Proceedings of the Nineteenth International Conference on Artificial
                  Intelligence and Law, {ICAIL} 2023, Braga, Portugal, June 19-23, 2023},
  pages        = {121--130},
  publisher    = {{ACM}},
  year         = {2023},
  url          = {https://dl.acm.org/doi/10.1145/3594536.3595122},
  doi          = {10.1145/3594536.3595122}
}

Symbolic Knowledge Extraction and Injection with Sub-symbolic Predictors: A Systematic Literature Review

2024-03-07T00:00:00+01:00

by Giovanni Ciatto, Federico Sabbatini, Andrea Agiollo, Matteo Magnini, and Andrea Omicini

Abstract

In this paper we focus on the issue of opacity of sub-symbolic machine-learning predictors by promoting two complementary activities—namely, symbolic knowledge extraction (SKE) and injection (SKI) from and into sub-symbolic predictors. We consider as symbolic any language being intelligible and interpretable for both humans and computers. Accordingly, we propose general meta-models for both SKE and SKI, along with two taxonomies for the classification of SKE/SKI methods. By adopting an eXplainable AI (XAI) perspective, we highlight how such methods can be exploited to either mitigate the aforementioned opacity issue. Our taxonomies are attained by surveying and classifying existing methods from the literature, following a systematic approach, and by generalising the results of previous surveys targeting specific sub-topics of either SKE or SKI alone. More precisely, we analyse 129 methods for SKE and 117 methods for SKI, and we categorise them according to their purpose, operation, expected input/output data and predictor types. For each method, we also indicate the presence/lack of runnable software implementations. Our work may be of interest for data scientists aiming at selecting the most adequate SKE/SKI method for their needs, and also work as suggestions for researchers interested in filling the gaps of the current state of the art, as well as for developers willing to implement SKE/SKI-based technologies.

How to access

DOI: http://dx.doi.org/10.1145/3645103

How to cite

Bibtex

@article{skeislr-acmcs,
    acm = {3645103},
    author = {Ciatto, Giovanni and Sabbatini, Federico and Agiollo, Andrea and Magnini, Matteo and Omicini, Andrea},
    doi = {10.1145/3645103},
    journal = {ACM Computing Surveys},
    keywords = {Logic; Machine learning theory; Hybrid symbolic-numeric methods; Knowledge representation and reasoning},
    numpages = 34,
    publisher = {ACM},
    scholar = {13701373869146776438},
    semanticscholar = {267611660},
    title = {Symbolic Knowledge Extraction and Injection with Sub-symbolic Predictors: A Systematic Literature Review},
    url = {https://dl.acm.org/doi/10.1145/3645103},
    year = 2024
}

Conflict-based negotiation strategy for human-agent negotiation

2024-03-07T00:00:00+01:00

by Mehmet Onur Keskin, Berk Buzcu, and Reyhan Aydoğan

Abstract

Day by day, human-agent negotiation becomes more and more vital to reach a socially beneficial agreement when stakeholders need to make a joint decision together. Developing agents who understand not only human preferences but also attitudes is a significant prerequisite for this kind of interaction. Studies on opponent modeling are predominantly based on automated negotiation and may yield good predictions after exchanging hundreds of offers. However, this is not the case in human-agent negotiation in which the total number of rounds does not usually exceed tens. For this reason, an opponent model technique is needed to extract the maximum information gained with limited interaction. This study presents a conflict-based opponent modeling technique and compares its prediction performance with the well-known approaches in human-agent and automated negotiation experimental settings. According to the results of human-agent studies, the proposed model outpr erforms them despite the diversity of participants’ negotiation behaviors. Besides, the conflict-based opponent model estimates the entire bid space much more successfully than its competitors in automated negotiation sessions when a small portion of the outcome space was explored. This study may contribute to developing agents that can perceive their human counterparts’ preferences and behaviors more accurately, acting cooperatively and reaching an admissible settlement for joint interests.

How to access

DOI: https://doi.org/10.1007/s10489-023-05001-9

How to cite

Bibtex

@article{KeskinBA23,
  author       = {Mehmet Onur Keskin and
                  Berk Buzcu and
                  Reyhan Aydogan},
  title        = {Conflict-based negotiation strategy for human-agent negotiation},
  journal      = {Appl. Intell.},
  volume       = {53},
  number       = {24},
  pages        = {29741--29757},
  year         = {2023},
  url          = {https://doi.org/10.1007/s10489-023-05001-9},
  doi          = {10.1007/S10489-023-05001-9},
  timestamp    = {Sat, 13 Jan 2024 17:36:05 +0100},
  biburl       = {https://dblp.org/rec/journals/apin/KeskinBA23.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

Towards interactive explanation-based nutrition virtual coaching systems

2024-04-11T00:00:00+01:00

by Berk Buzcu, Melissa Tessa, Igor Tchappi, Amro Najjar, Joris Hulstijn, Davide Calvaresi and Reyhan Aydoğan

Abstract

The awareness about healthy lifestyles is increasing, opening to personalized intelligent health coaching applications. A demand for more than mere suggestions and mechanistic interactions has driven attention to nutrition virtual coaching systems (NVC) as a bridge between human–machine interaction and recommender, informative, persuasive, and argumentation systems. NVC can rely on data-driven opaque mechanisms. Therefore, it is crucial to enable NVC to explain their doing (i.e., engaging the user in discussions (via arguments) about dietary solutions/alternatives). By doing so, transparency, user acceptance, and engagement are expected to be boosted. This study focuses on NVC agents generating personalized food recommendations based on user-specific factors such as allergies, eating habits, lifestyles, and ingredient preferences. In particular, we propose a user-agent negotiation process entailing run-time feedback mechanisms to react to both recommendations and related explanations. Lastly, the study presents the findings obtained by the experiments conducted with multi-background participants to evaluate the acceptability and effectiveness of the proposed system. The results indicate that most participants value the opportunity to provide feedback and receive explanations for recommendations. Additionally, the users are fond of receiving information tailored to their needs. Furthermore, our interactive recommendation system performed better than the corresponding traditional recommendation system in terms of effectiveness regarding the number of agreements and rounds.

How to access

DOI: https://doi.org/10.1007/s10458-023-09634-5

How to cite

Bibtex

@article{BuzcuTTNHCA24,
  author       = {Berk Buzcu and
                  Melissa Tessa and
                  Igor Tchappi and
                  Amro Najjar and
                  Joris Hulstijn and
                  Davide Calvaresi and
                  Reyhan Aydogan},
  title        = {Towards interactive explanation-based nutrition virtual coaching systems},
  journal      = {Auton. Agents Multi Agent Syst.},
  volume       = {38},
  number       = {1},
  pages        = {5},
  year         = {2024},
  url          = {https://doi.org/10.1007/s10458-023-09634-5},
  doi          = {10.1007/S10458-023-09634-5},
  timestamp    = {Fri, 08 Mar 2024 13:21:53 +0100},
  biburl       = {https://dblp.org/rec/journals/aamas/BuzcuTTNHCA24.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

User-centric Explanation Strategies for Interactive Recommenders

2024-04-12T00:00:00+01:00

by Berk Buzcu, Kuru Emre, and Reyhan Aydoğan

Abstract

With the pervasive usage of recommendation systems across various domains, there is a growing need for transparent and convincing interactions to build a rapport with the system users. Incorporating explainability into recommendation systems has become a promising strategy to bolster user trust and sociability. This study centers on recommendation systems that leverage varying explainability techniques to cultivate trust by delivering comprehensible customized explanations for the given recommendations. Accordingly, we propose two explanation methods aligning with a cluster-based recommendation strategy.

How to access

How to cite

Bibtex

@inproceedings{buzcu2024user,
  title={User-centric Explanation Strategies for Interactive Recommenders},
  author={Buzcu, Berk and Kuru, Emre and Aydogan, Reyhan},
  booktitle={AAMAS 2024, Extended Abstract},
  address={New Zealand, Auckland},
  year={2024}
}

You Look Nice, but I Am Here to Negotiate: The Influence of Robot Appearance on Negotiation Dynamics

2024-04-12T00:00:00+01:00

by Mehmet Onur Keskin, Selen Akay, Ayse Dogan, Berkecan Koçyigit, Junko Kanero and Reyhan Aydoğan

Abstract

This report presents two experimental studies examining whether relatively subtle differences in the appearances of humanoid robots impact (1) the outcomes of human-robot negotiation (i.e., utility scores) and (2) the participant’s attitudes toward their robot negotiation partner. Study I compared Nao and Pepper, and Study II compared Nao and QT in identical negotiation settings. While the appearance of robots influenced the participant’s attitudes toward the robot before and after the negotiation, such differences were not manifested in the utility scores. The consistent utility scores across different robots reassure that minor variations in the visual characteristics of robots do not alter how users negotiate with a robot. Yet, as participants felt differently about the three robots, there remains the possibility that the differences in their appearances may influence the user’s initial inclination to approach each robot. As among the first to systematically investigate the influence of robot appearance on human-robot negotiations, this study emphasizes the importance of assessing both objective outcome scores and the subjective experience of the user in human-robot interaction (HRI) research and offers valuable insights for designing and implementing social robots in real-world settings including customer service and other AI-based interactions.

How to access

DOI: https://doi.org/10.1145/3610978.3640759

How to cite

Bibtex

@inproceedings{KeskinADKKA24,
  author       = {Mehmet Onur Keskin and
                  Selen Akay and
                  Ayse Dogan and
                  Berkecan Ko{\c{c}}yigit and
                  Junko Kanero and
                  Reyhan Aydogan},
  editor       = {Dan Grollman and
                  Elizabeth Broadbent and
                  Wendy Ju and
                  Harold Soh and
                  Tom Williams},
  title        = {You Look Nice, but {I} Am Here to Negotiate: The Influence of Robot
                  Appearance on Negotiation Dynamics},
  booktitle    = {Companion of the 2024 {ACM/IEEE} International Conference on Human-Robot
                  Interaction, {HRI} 2024, Boulder, CO, USA, March 11-15, 2024},
  pages        = {598--602},
  publisher    = {{ACM}},
  year         = {2024},
  url          = {https://doi.org/10.1145/3610978.3640759},
  doi          = {10.1145/3610978.3640759},
  timestamp    = {Mon, 01 Apr 2024 11:14:14 +0200},
  biburl       = {https://dblp.org/rec/conf/hri/KeskinADKKA24.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}

What should I do and why?

2024-04-12T00:00:00+01:00

by Joris Hulstijn and Leon Van der Torre

Abstract

There is a lot of interest in explainable AI [2, 11]. When a system takes decisions that affect people, they can demand an explanation of how the decision was derived, or a justification of why the decision is justified. Note that explanation and justification are related, but not the same [1]. The need for explanation or justification is more pressing, when the system makes legal decisions [3], or when the decision is based on social or ethical norms [5]. Requests for explanation and justification are typically made in a dialogue setting [5].

How to access

URL: https://www.collegepublications.co.uk/downloads/DEON00004.pdf#page=348

How to cite

Bibtex

@inbook{Hulstijn_vdTorre2023,
   author = {Hulstijn, J.  and van der Torre, L.},
   title = {What should I do and why?},
   booktitle = {Proceedings of The 16th International Conference on Deontic Logic and Normative Systems (DEON 2023)},
   editor = {Peterson, C.},
   publisher = {College Publications},
   address = {Trois Rivieres, Canada},
   pages = {335-336},
   year = {2023}
}

Towards interactive and social explainable artificial intelligence for digital history

2024-04-12T00:00:00+01:00

by Albrecht Richard, Amro Najjar, Igor Tchappi and Joris Hulstijn

Abstract

Due to recent development and improvements in the field of artificial intelligence (AI), methods of that field are increasingly adopted in various domains, including historical research. However, modern state-of-the-art machine learning (ML) models are black-boxes that lack transparency and interpretability. Therefore, explainable AI (XAI) methods are used to make black-box models more transparent and inspire user trust. Despite numerous opportunities of applying XAI in this field, they have only recently been adopted. Even though calls for incorporating insights from social sciences into the application of XAI exist, applied XAI methods to generate historical insights are static and not user-centric. Furthermore, there exist theoretical frameworks to use XAI methods interactively in order to generate user understanding incrementally, like the social XAI framework.

How to access

URL: TBA

How to cite

Bibtex

@inproceedings{Albrecht_etal2024,

   author = {Albrecht, Richard and Najjar, Amro and Tchappi, Igor and Hulstijn, Joris},

   title = {Towards interactive and social explainable artificial intelligence for digital history},

  booktitle={Explainable and Transparent AI and Multi-Agent Systems (EXTRAAMAS 2024)},

   year = {2024}

}

The Wildcard XAI: from a Necessity, to a Resource, to a Dangerous Decoy

2024-10-24T00:00:00+01:00

by Rachele Carli, and Davide Calvaresi

Abstract

There has been a growing interest in Explainable Artificial Intelligence (henceforth XAI) models among researchers and AI programmers in recent years. Indeed, the development of highly interactive technologies that can collaborate closely with users has made explainability a necessity. This intends to reduce mistrust and the sense of unpredictability that AI can create, especially among non-experts. Moreover, the potential of XAI as a valuable resource has been recognized, considering that it can make intelligent systems more user-friendly and reduce the negative impact of black box systems. Building on such considerations, the paper discusses the potential dangers of large language models (LLMs) that generate explanations to support the outcomes produced. While these models may give users the illusion of control over the system’s responses, they actually have persuasive and non-explanatory effects. Therefore, it is argued here that XAI, appropriately regulated, should be a resource to empower users of AI systems. Any other apparent explanations should be reported to avoid misleading and circumventing effects.

How to access

URL: https://publications.hevs.ch/index.php/publications/show/3146

How to cite

Bibtex

@ARTICLE{carli2024wildcard,
  author = {Carli, Rachele and Calvaresi, Davide},
  keywords = {Anthropomorphism, Dependency, XAI},
  month = aug,
  title = {The Wildcard XAI: from a Necessity, to a Resource, to a Dangerous Decoy},
  pages={224--241},
  journal = {In post-proceedings of the 6th International Workshop on EXplainable and TRAnsparent AI and Multi-Agent Systems},
  year = {2024},
  organization={Springer}
}

Rethinking Health Recommender Systems for Active Aging: An Autonomy-Based Ethical Analysis

2024-10-24T00:00:00+01:00

by Simona Tiribelli, and Davide Calvaresi

Abstract

Health Recommender Systems (HRS) are promising Articial-Intelligence (AI)-based tools endowing healthy lifestyles and therapy adherence in healthcare and medicine. Among the most supported areas, it is worth mentioning active aging (AA). However, current HRS supporting AA raise ethical challenges that still need to be properly formalized and explored. This study proposes to rethink HRS for AA through an autonomy-based ethical analysis. In particular, a brief overview of the HRS' technical aspects allows us to shed light on the ethical risks and challenges they might raise on individuals' well-being as they age. Moreover, the study proposes a categorization, understanding, and possible preventive/mitigation actions for the elicited risks and challenges through rethinking the AI ethics core principle of autonomy. Finally, elaborating on autonomy-related ethical theories, the paper proposes an autonomy-based ethical framework and how it can foster the development of autonomy-enabling HRS for AA.

How to access

How to cite

Bibtex

@ARTICLE{tiribelli2024rethinking,
  author = {Tiribelli, Simona and Calvaresi, Davide},
  keywords = {active aging, AI ethics, autonomy, health recommender systems, persuasion technology},
  month = may,
  title = {Rethinking Health Recommender Systems for Active Aging: An Autonomy-Based Ethical Analysis},
  journal = {Science and Engineering Ethics},
  year = {2024},
  doi = {https://doi.org/10.1017/s1358246117000042},
  volume={30},
  number={3},
  pages={22},
  publisher={Springer}
}

Explanation of Deep Learning Models via Logic Rules Enhanced by Embeddings Analysis, and Probabilistic Models

2024-10-24T00:00:00+01:00

by Contreras Ordoñez Victor Hugo, Michael I. Schumacher and Davide Calvaresi

Abstract

Deep Learning (DL) models are increasingly dealing with heterogeneous data (i.e., a mix of structured and unstructured data), calling for adequate eXplainable Artificial Intelligence (XAI) methods. Nevertheless, only some of the existing techniques consider the uncertainty inherent to the data. To this end, this study proposes a pipeline to explain heterogeneous data-based DL models by combining embed- ding analysis, rule extraction methods, and probabilistic models. The proposed pipeline has been tested using synthetic data (multi-individual food items tracking). This study has achieved (i) inference enhancement through probabilistic and evidential reasoning, (ii) generation of logical explanations based on extracted rules and predictions, and (iii) integration of textual data into the explanation pipeline through embedding analysis.

How to access

How to cite

Bibtex

@INPROCEEDINGS{contreras2024explanation,
     author = {Contreras, Victor H. and Schumacher, Michael and Calvaresi, Davide},
     keywords = {Deep Learning, Heterogeneous data processing, Preference modeling, rule extraction, Uncertainty reasoning, XAI},
     month = aug,
     title = {Explanation of Deep Learning Models via Logic Rules Enhanced by Embeddings Analysis, and Probabilistic Models},booktitle = {Post-proceedings of the 6th International Workshop on EXplainable and TRAnsparent AI and Multi-Agent Systems},
     year = {2024},
     pages={155--183},s
     organization={Springer},
     doi={10.1007/978-3-031-70074-3_9},
     isbn={978-3-031-70074-3}
}

Evaluation of the User-centric Explanation Strategies for Interactive Recommenders

2024-10-24T00:00:00+01:00

by Berk Buzcu, Emre Kuru, Davide Calvaresi and Reyhan Aydoğan

Abstract

As recommendation systems become increasingly prevalent in numerous fields, the need for clear and persuasive interactions with users is rising. Integrating explainability into these systems is emerging as an effective approach to enhance user trust and sociability. This research focuses on recommendation systems that utilize a range of explainability techniques to foster trust by providing understandable personalized explanations for the recommendations made. In line with this, we study three distinct explanation methods that correspond with three basic recommendation strategies and assess their efficacy through user experiments. The findings from the experiments indicate that the majority of participants value the suggested explanation styles and favor straightforward, concise explanations over comparative ones.

How to access

How to cite

Bibtex

@INPROCEEDINGS{buzcu2024evaluation,
     author = {Buzcu, Berk and Kuru, Emre and Calvaresi, Davide and Aydoğan, Reyhan},
     keywords = {Explainable Recommendations, Explanation Strategies, User Studies},
     month = aug,
     title = {Evaluation of the User-centric Explanation Strategies for Interactive Recommenders},
     booktitle = {Post-proceedings in the 6th International Workshop on EXplainable and TRAnsparent AI and Multi-Agent Systems},
     year = {2024},
     pages={21--38},
     organization={Springer}
}

A Framework for Explainable Multi-purpose Virtual Assistants: A Nutrition-Focused Case Study

2024-10-24T00:00:00+01:00

by Berk Buzcu, Yvan Pannatier, Reyhan Aydoğan, Michael I. Schumacher, Jean-Paul Calbimonte, and Davide Calvaresi

Abstract

Existing agent-based chatbot frameworks need seamless mechanisms to include explainable dialogic engines within the contextual flow. To this end, this paper presents a set of novel modules within the EREBOTS agent-based framework for chatbot development, including dialog-based plug-and-play custom algorithms, agnostic back/front ends, and embedded interactive explainable engines that can manage human feedback at run time. The framework has been employed to implement an explainable agent-based interactive food recommender system. The latter has been tested with 44 participants, who followed a nutrition recommendation interaction series, generating explained recommendations and suggestions, which were, in general, well received. Additionally, the participants provided important insights to be included in future work.

How to access

How to cite

Bibtex

@INPROCEEDINGS{buzcu2024framework,
     author = {Buzcu, Berk and Pannatier, Yvan and Aydoğan, Reyhan and Schumacher, Michael and Calbimonte, Jean-Paul and Calvaresi, Davide},
   keywords = {Chatbot Framework, Explainable AI, User Study},
      month = aug,
      title = {A Framework for Explainable Multi-purpose Virtual Assistants: A Nutrition-Focused Case Study},
  booktitle = {In post-proceedings of the 6th International Workshop on EXplainable and TRAnsparent AI and Multi-Agent Systems},
       year = {2024},
      pages = {58-78},
  publisher = {Springer Nature Switzerland},
       issn = {1611-3349},
       isbn = {9783031700743},
        doi = {10.1007/978-3-031-70074-3_4}
}

GNN2GNN: Graph Neural Networks to Generate Neural Networks

2024-10-28T00:00:00+01:00

by Andrea Agiollo and Andrea Omicini

Abstract

The success of neural networks (NNs) is tightly linked with their architectural design—a complex problem by itself. We here introduce a novel framework leveraging Graph Neural Networks to Generate Neural Networks (GNN2GNN) where powerful NN architectures can be learned out of a set of available architecture-performance pairs. GNN2GNN relies on a three-way adversarial training of GNN, to optimise a generator model capable of producing predictions about powerful NN architectures. Unlike Neural Architecture Search (NAS) techniques proposing efficient searching algorithms over a set of NN architec- tures, GNN2GNN relies on learning NN architectural design criteria. GNN2GNN learns to propose NN architectures in a single step – i.e., training of the generator –, overcoming the recursive approach characterising NAS. Therefore, GNN2GNN avoids the expensive and inflexible search of efficient structures typical of NAS approaches. Extensive experiments over two state-of-the-art datasets prove the strength of our framework, showing that it can generate powerful architectures with high probability. Moreover, GNN2GNN outperforms possible counterparts for generating NN architectures, and shows flexibility against dataset quality degradation. Finally, GNN2GNN paves the way towards generalisation between datasets.

How to access

URL: https://proceedings.mlr.press/v180/agiollo22a.html
URL open access: https://proceedings.mlr.press/v180/agiollo22a/agiollo22a.pdf

How to cite

Bibtex

@inproceedings{gnn2gnn-uai2022,
    author = {Agiollo, Andrea and Omicini, Andrea},
    booktitle = {Uncertainty in Artificial Intelligence},
    dblp = {conf/uai/AgiolloO22},
    editor = {Cussens, James and Zhang, Kun},
    iris = {11585/899465},
    issn = {2640-3498},
    month = aug,
    note = {Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, {UAI} 2022, 1-5 August 2022, Eindhoven, The Netherlands},
    pages = {32--42},
    publisher = {ML Research Press},
    scholar = {3643983533846865361},
    series = {Proceedings of Machine Learning Research},
    title = {{GNN2GNN}: Graph Neural Networks to Generate Neural Networks},
    url = {https://proceedings.mlr.press/v180/agiollo22a.html},
    urlopenaccess = {https://proceedings.mlr.press/v180/agiollo22a/agiollo22a.pdf},
    volume = 180,
    year = 2022
}

Position Paper: On the Role of Abductive Reasoning in Semantic Image Segmentation

2024-10-28T00:00:00+01:00

by Andrea Rafanelli, Stefania Costantini and Andrea Omicini

Abstract

This position paper provides insights aiming at resolving the most pressing needs and issues of computer vision algorithms. Specifically, these problems relate to the scarcity of data, the inability of such algorithms to adapt to never-seen-before conditions, and the challenge of developing explainable and trustworthy algorithms. This work proposes the incorporation of reasoning systems, and in particular of abductive reasoning, into image segmentation algorithms as a potential solution to the aforementioned issues.

How to access

URL: https://ceur-ws.org/Vol-3419/paper9.pdf

How to cite

Bibtex

@inproceedings{abdsemimgsegm-aixia2022,
    address = {Aachen, Germany},
    author = {Rafanelli, Andrea and Stefania Costantini and Omicini, Andrea},
    booktitle = {AIxIA 2022 Discussion Papers},
    chapter = 9,
    dblp = {conf/aiia/RafanelliCO22},
    editor = {Dovier, Agostino and Montanari, Angelo and Orlandini, Andrea},
    iris = {11585/933658},
    issn = {1613-0073},
    keywords = {neuro-symbolic, abductive reasoning, semantic segmentation, XAI},
    location = {Udine, Italy},
    month = jun,
    note = {Proceedings of the Discussion Papers -- 22nd International Conference of the Italian Association for Artificial Intelligence (AIxIA 2022 DP), co-located with 22nd International Conference of the Italian Association for Artificial Intelligence (AIxIA 2022) -- Udine, Italy, November 28--December 2, 2022},
    numpages = 10,
    pages = {75--84},
    publisher = {Sun SITE Central Europe, RWTH Aachen University},
    scholar = {4710138681206481431},
    scopus = {2-s2.0-85164192487},
    series = {CEUR Workshop Proceedings},
    subseries = {AIxIA Series},
    title = {Position Paper: On the Role of Abductive Reasoning in Semantic Image Segmentation},
    url = {https://ceur-ws.org/Vol-3419/paper9.pdf},
    urlopenaccess = {https://ceur-ws.org/Vol-3419/paper9.pdf},
    urlpdf = {https://ceur-ws.org/Vol-3419/paper9.pdf},
    volume = 3419,
    year = 2023
}

Measuring Trustworthiness in Neuro-Symbolic Integration

2024-10-28T00:00:00+01:00

by Andrea Agiollo and Andrea Omicini

Abstract

Neuro-symbolic integration of symbolic and subsymbolic techniques represents a fast-growing AI trend aimed at mitigating the issues of neural networks in terms of decision processes, reasoning, and interpretability. Several state-of-the-art neuro-symbolic approaches aim at improving performance, most of them focusing on proving their effectiveness in terms of raw predictive performance and/or reasoning capabilities. Meanwhile, few efforts have been devoted to increasing model trustworthiness, interpretability, and efficiency – mostly due to the complexity of measuring effectively improvements in terms of trustworthiness and interpretability. This is why here we analyse and discuss the need for ad-hoc trustworthiness metrics for neuro-symbolic techniques. We focus on two popular paradigms mixing subsymbolic computation and symbolic knowledge, namely: (i) symbolic knowledge extraction (SKE), aimed at mapping subsymbolic models into human-interpretable knowledge bases; and (ii) symbolic knowledge injection (SKI), aimed at forcing subsymbolic models to adhere to a given symbolic knowledge. We first emphasise the need for assessing neuro-symbolic approaches from a trustworthiness perspective, highlighting the research challenges linked with this evaluation and the need for ad-hoc trust definitions. Then we summarise recent developments in SKE and SKI metrics focusing specifically on several trustworthiness pillars such as interpretability, efficiency, and robustness of neuro-symbolic methods. Finally, we highlight open research opportunities towards reliable and flexible trustworthiness metrics for neuro-symbolic integration.

How to access

DOI: https://doi.org/10.15439/2023F6019
URL: https://annals-csis.org/Volume_35/drp/6019.html
URL open access: https://annals-csis.org/Volume_35/drp/pdf/6019.pdf

How to cite

Bibtex

@inproceedings{keynote-fedcsis2023,
    author = {Agiollo, Andrea and Omicini, Andrea},
    booktitle = {Proceedings of the 18th Conference on Computer Science and Intelligence Systems},
    dblp = {conf/fedcsis/AgiolloO23},
    doi = {10.15439/2023F6019},
    editor = {Maria Ganzha and Leszek Maciaszek and Marcin Paprzycki and Dominik Ślęzak},
    eisbn = {978-83-967447-8-4},
    ieee = {10305942},
    iris = {11585/951750},
    isbn = {978-83-969601-0-8},
    isbn13 = {978-83-967447-9-1},
    issn = {2300-5963},
    month = sep,
    numpages = 10,
    pages = {1--10},
    scholar = {4358274351024166022},
    scopus = {2-s2.0-85179180821},
    series = {Annals of Computer Sciences and Information Systems},
    title = {Measuring Trustworthiness in Neuro-Symbolic Integration},
    url = {https://annals-csis.org/Volume_35/drp/6019.html},
    urlopenaccess = {https://annals-csis.org/Volume_35/drp/pdf/6019.pdf},
    urlpdf = {https://annals-csis.org/Volume_35/drp/pdf/6019.pdf},
    volume = 35,
    year = 2023
}

The Quarrel of Local Post-hoc Explainers for Moral Values Classification in Natural Language Processing

2024-10-28T00:00:00+01:00

by Andrea Agiollo, Luciano C. Siebert, Pradeep K. Murukannaiah and Andrea Omicini

Abstract

Although popular and effective, large language models (LLM) are characterised by a performance vs. transparency trade-off that hinders their applicability to sensitive scenarios. This is the main reason behind many approaches focusing on local post-hoc explanations recently proposed by the XAI community. However, to the best of our knowledge, a thorough comparison among available explainability techniques is currently missing, mainly for the lack of a general metric to measure their benefits. We compare state-of-the-art local post-hoc explanation mechanisms for models trained over moral value classification tasks based on a measure of correlation. By relying on a novel framework for comparing global impact scores, our experiments show how most local post-hoc explainers are loosely correlated, and highlight huge discrepancies in their results—their “quarrel” about explanations. Finally, we compare the impact scores distribution obtained from each local post-hoc explainer with human-made dictionaries, and point out that there is no correlation between explanation outputs and the concepts humans consider as salient.

How to access

How to cite

Bibtex

@incollection{quarrel-extraamas2023,
    apice = {QuarrelExtraamas2023},
    author = {Agiollo, Andrea and Siebert, Luciano C. and Murukannaiah, Pradeep K. and Omicini, Andrea},
    booktitle = {Explainable and Transparent {AI} and Multi-Agent Systems},
    chapter = 6,
    dblp = {conf/extraamas/AgiolloSMO23},
    doi = {10.1007/978-3-031-40878-6_6},
    editor = {Calvaresi, Davide and Najjar, Amro and Omicini, Andrea and Aydoǧan, Reyhan and Carli, Rachele and Ciatto, Giovanni and Mualla, Yazan and Främling, Kary},
    iris = {11585/940734},
    isbn = {978-3-031-40878-6},
    issn = {0302-9743},
    keywords = {Natural Language Processing, Moral Values Classification, eXplainable Artificial Intelligence, Local Post-hoc Explanations.},
    month = sep,
    numpages = 19,
    pages = {97--115},
    publisher = {Springer},
    scholar = {16243619767560534499},
    scopus = {2-s2.0-85172189313},
    series = {Lecture Notes in Computer Science},
    subseries = {Lecture Notes in Artificial Intelligence},
    title = {The Quarrel of Local Post-hoc Explainers for Moral Values Classification in Natural Language Processing},
    url = {http://link.springer.com/10.1007/978-3-031-40878-6_6},
    urlpdf = {https://link.springer.com/content/pdf/10.1007/978-3-031-40878-6_6.pdf},
    volume = 14127,
    year = 2023
}

Peer-Reviewed Federated Learning

2024-10-28T00:00:00+01:00

by Mattia Passeri, Andrea Agiollo, and Andrea Omicini

Abstract

While representing the de-facto framework for enabling distributed training of Machine Learning models, Federated Learning (FL) still suffers convergence issues when non-Independent and Identically Distributed (non-IID) data are considered. In this context, the local model optimisation on different data distributions generate dissimilar updates, which are difficult to aggregate and translate into sub-optimal convergence. To tackle this issues, we propose Peer-Reviewed Federated Learning (PRFL), an extension of the traditional FL training process inspired by the peer-review procedure common in the academic field, where model updates are reviewed by several other clients in the federation before being aggregated at the server-side. PRFL aims at enabling the identification of relevant updates, while disregarding the ineffective ones. We implement PRFL on top of the Flower FL library, and make Peer-Reviewed Flower a publicly-available library for the modular implementation of any review-based FL algorithm. A preliminary case study on both regression and classification tasks highlights the potential of PRFL, showcasing how the distributed solution can achieve performance similar to that obtained by the corresponding centralised algorithm, even when non-IID data are considered.

How to access

URL: https://ceur-ws.org/Vol-3579/paper4.pdf

How to cite

Bibtex



@inproceedings{prfl-woa2023,
    author = {Passeri, Mattia and Agiollo, Andrea and Omicini, Andrea},
    booktitle = {WOA 2023 -- 24th Workshop ``From Objects to Agents''},
    dblp = {conf/woa/PasseriAO23},
    editor = {Falcone, Rino and Castelfranchi, Cristiano and Sapienza, Alessandro and Cantucci, Filippo},
    iris = {11585/950373},
    issn = {1613-0073},
    keywords = {Federated Learning; non-IID data; Peer-review},
    month = nov,
    numpages = 17,
    pages = {49--65},
    publisher = {Sun SITE Central Europe, RWTH Aachen University},
    scholar = {7461839519241304430},
    scopus = {2-s2.0-85179627285},
    series = {CEUR Workshop Proceedings},
    subseries = {AIxIA Series},
    title = {Peer-Reviewed Federated Learning},
    url = {https://ceur-ws.org/Vol-3579/paper4.pdf},
    urlopenaccess = {https://ceur-ws.org/Vol-3579/paper4.pdf},
    urlpdf = {https://ceur-ws.org/Vol-3579/paper4.pdf},
    volume = 3579,
    year = 2023
}

Towards a Unified Model for Symbolic Knowledge Extraction with Hypercube-Based Methods

2024-10-28T00:00:00+01:00

by Federico Sabbatini, Giovanni Ciatto, Roberta Calegari, and Andrea Omicini

Abstract

The XAI community is currently studying and developing symbolic knowledge-extraction (SKE) algorithms as a means to produce human-intelligible explanations for black-box machine learning predictors, so as to achieve believability in human-machine interaction. However, many extraction procedures exist in the literature, and choosing the most adequate one is increasingly cumbersome, as novel methods keep on emerging. Challenges arise from the fact that SKE algorithms are commonly defined based on theoretical assumptions that typically hinder practical applicability. This paper focuses on hypercube-based SKE methods, a quite general class of extraction techniques mostly devoted to regression-specific tasks. We first show that hypercube-based methods are flexible enough to support classification problems as well, then we propose a general model for them, and discuss how they support SKE on datasets, predictors, or learning tasks of any sort. Empirical examples are reported as well –based upon the PSyKE framework –, showing the applicability of hypercube-based methods to actual classification tasks.

How to access

How to cite

Bibtex

@article{hypercubeske-ia17,
    author = {Sabbatini, Federico and Ciatto, Giovanni and Calegari, Roberta and Omicini, Andrea},
    dblp = {journals/ia/SabbatiniCCO23},
    doi = {10.3233/IA-230001},
    editor = {Ferrando, Angelo and Mascardi, Viviana},
    iris = {11585/941033},
    journal = {Intelligenza Artificiale},
    keywords = {explainable AI, knowledge extraction, interpretable prediction, PSyKE},
    month = jun,
    note = {Special issue: Selected papers from the 23rd Workshop ``From Objects to Agents'' (WOA 2022)},
    number = 1,
    numpages = 13,
    pages = {63--75},
    publisher = {IOS Press},
    scholar = {14669296704428238758},
    scopus = {2-s2.0-85168332389},
    semanticscholar = {259324728},
    title = {Towards a Unified Model for Symbolic Knowledge Extraction with Hypercube-Based Methods},
    url = {https://content.iospress.com/articles/intelligenza-artificiale/ia230001},
    urlopenaccess = {https://cris.unibo.it/retrieve/3a2a510e-9dc5-4b07-b3f7-398cf2f21419/ia-2023-psyke.pdf},
    volume = 17,
    year = 2023
}

Knowledge injection of Datalog rules via Neural Network Structuring with KINS

2024-10-28T00:00:00+01:00

by Matteo Magnini, Giovanni Ciatto, and Andrea Omicini

Abstract

We propose a novel method to inject symbolic knowledge in form of Datalog formulæ into neural networks (NN), called Knowledge Injection via Network Structuring (KINS). The idea behind our method is to extend NN internal structure with ad-hoc layers built out of the injected symbolic knowledge. KINS does not constrain NN to any specific architecture, neither requires logic formulæ to be ground. Moreover, it is robust w.r.t. both lack of data and imperfect/incomplete knowledge. Experiments are reported, involving multiple datasets and predictor types, to demonstrate how KINS can significantly improve the predictive performance of the neural networks it is applied to.

How to access

How to cite

Bibtex

@article{kins-jlc33,
    author = {Magnini, Matteo and Ciatto, Giovanni and Omicini, Andrea},
    dblp = {journals/logcom/MagniniCO23},
    doi = {10.1093/logcom/exad037},
    eissn = {1465-363X},
    iris = {11585/950567},
    issn = {0955-792X},
    journal = {Journal of Logic and Computation},
    keywords = {neural network, expalinable AI, symbolic knowledge injection, KINS, PSyKI},
    month = dec,
    number = 8,
    numpages = 19,
    pages = {1832--1850},
    publisher = {Oxford University Press},
    scholar = {2353304508513748358},
    scopus = {2-s2.0-85179896166},
    semanticscholar = {266726315},
    title = {Knowledge injection of {D}atalog rules via Neural Network Structuring with {KINS}},
    url = {https://academic.oup.com/logcom/article/33/8/1832/7190990},
    volume = 33,
    wos = {WOS:001003002200001},
    year = 2023
}

EneA-FL: Energy-aware Orchestration for Serverless Federated Learning

2024-10-28T00:00:00+01:00

by Andrea Agiollo, Paolo Bellavista, Matteo Mendula and Andrea Omicini

Abstract

Federated Learning (FL) represents the de-facto standard paradigm for enabling distributed learning over multiple clients in real-world scenarios. Despite the great strides reached in terms of accuracy and privacy awareness, the real adoption of FL in real-world scenarios, in particular in industrial deployment environments, is still an open thread. This is mainly due to privacy constraints and to the additional complexity stemming from the set of hyperparameters to tune when employing AI techniques on bandwidth-, computing-, and energy-constrained nodes. Motivated by these issues, we focus on scenarios where participating clients are characterised by highly heterogeneous computing capabilities and energy budgets proposing EneA-FL, an innovative scheme for serverless smart energy management. This novel approach dynamically adapts to optimise the training process while fostering seamless interaction between Internet of Things (IoT) devices and edge nodes. In particular, the proposed middleware provides a containerised software module that efficiently manages the interaction of each worker node with the central aggregator. By monitoring local energy budget, computational capabilities, and target accuracy, EneA-FL intelligently takes informed decisions about the inclusion of specific nodes in the subsequent training rounds, effectively balancing the tripartite trade-off between energy consumption, training time, and final accuracy. Finally, in a series of extensive experiments across diverse scenarios, our solution demonstrates impressive results, achieving between 30% and 60% lower energy consumption against popular client selection approaches available in the literature while being up to 3.5 times more efficient than standard FL solutions.

How to access

How to cite

Bibtex

@article{eneafl-fgcs154,
    author = {Agiollo, Andrea and Bellavista, Paolo and Mendula, Matteo and Omicini, Andrea},
    dblp = {journals/fgcs/AgiolloBMO24},
    doi = {10.1016/j.future.2024.01.007},
    editor = {Hao Wu and Carlo Puliafito and Omer F. Rana and Luiz F. Bittencourt},
    iris = {11585/953081},
    issn = {0167-739X},
    journal = {Future Generation Computer Systems},
    keywords = {Serverless, Federated Learning, Energy Management, Internet of Things, Resource-constrained Learning},
    month = may,
    note = {Special Issue ``Serverless Computing in the Cloud-to-Edge Continuum''},
    numpages = 16,
    pages = {219--234},
    publisher = {Elsevier Science B.V.},
    scholar = {15164122000920541506},
    scopus = {2-s2.0-85182399653},
    title = {{EneA-FL}: Energy-aware Orchestration for Serverless Federated Learning},
    url = {https://www.sciencedirect.com/science/article/pii/S0167739X24000074},
    volume = 154,
    wos = {WOS:001164533000001},
    year = 2024
}

From Large Language Models to Small Logic Programs: Building Global Explanations from Disagreeing Local Post-hoc Explainers

2024-10-28T00:00:00+01:00

by Andrea Agiollo, Luciano Siebert Cavalcante, Pradeep Kumar Murukannaiah and Andrea Omicini

Abstract

The expressive power and efectiveness of large language models (LLMs) is going to increasingly push intelligent agents towards sub-symbolic models for natural language processing (NLP) tasks in human–agent interaction. However, LLMs are characterised by a performance vs. transparency trade-of that hinders their applicability to such sensitive scenarios. This is the main reason behind many approaches focusing on local post-hoc explanations, recently proposed by the XAI community in the NLP realm. However, to the best of our knowledge, a thorough comparison among available explainability techniques is currently missing, as well as approaches for constructing global post-hoc explanations leveraging the local information. This is why we propose a novel framework for comparing state-of-the-art local post-hoc explanation mechanisms and for extracting logic programs surrogating LLMs. Our experiments—over a wide variety of text classifcation tasks— show how most local post-hoc explainers are loosely correlated, highlighting substantial discrepancies in their results. By relying on the proposed novel framework, we also show how it is possible to extract faithful and efficient global explanations for the original LLM over multiple tasks, enabling explainable and resource-friendly AI techniques.

How to access

DOI: https://doi.org/10.1007/s10458-024-09663-8
URL open access: https://link.springer.com/content/pdf/10.1007/s10458-024-09663-8.pdf
URL: http://link.springer.com/10.1007/s10458-024-09663-8

How to cite

Bibtex

@article{skenlp-jaamas38,
    articleno = 32,
    author = {Agiollo, Andrea and Siebert, Luciano Cavalcante and Murukannaiah, Pradeep Kumar and Omicini, Andrea},
    dblp = {journals/aamas/AgiolloSMO24},
    doi = {10.1007/s10458-024-09663-8},
    iris = {11585/973894},
    journal = {Autonomous Agents and Multi-Agent Systems},
    keywords = {Natural language processing, post-hoc explanations, symbolic knowledge extraction, eXplainable AI, resource-friendly AI},
    month = jul,
    note = {Special Issue on Multi-Agent Systems and Explainable AI},
    numpages = 33,
    pages = {1--33},
    publisher = {Springer},
    scholar = {13114388339509722903},
    scopus = {2-s2.0-85197706028},
    title = {From Large Language Models to Small Logic Programs: Building Global Explanations from Disagreeing Local Post-hoc Explainers},
    url = {http://link.springer.com/10.1007/s10458-024-09663-8},
    urlopenaccess = {https://link.springer.com/content/pdf/10.1007/s10458-024-09663-8.pdf},
    urlpdf = {https://link.springer.com/content/pdf/10.1007/s10458-024-09663-8.pdf},
    volume = 38,
    wos = {WOS:001264787900001},
    year = 2024
}