2023Activity reportProject-TeamCOMETE

3.1.1 Three way optimization between privacy and utility

One of the main problems in the design of privacy mechanisms is the preservation of the utility. In the case of local privacy, namely when the data are sanitized by the user before they are collected, the notion of utility is twofold:

• Utility as quality of service (QoS):
  The user usually gives his data in exchange of some service, and in general the quality of the service depends on the precision of such data. For instance, consider a scenario in which Alice wants to use a LBS (Location-Based Service) to find some restaurant near her location $x$. The LBS needs of course to know Alice's location, at least approximately, in order to provide the service. If Alice is worried about her privacy, she may send to the LBS an approximate location $y$ instead of $x$. Clearly, the LBS will send a list of restaurants near $x$, so if $y$ is too far from $x$ the service will degrade, while if it is too close Alice's privacy would be at stake.
• Utility as statistical quality of the data (Stat):
  Bob, the service provider, is motivated to offer his service because in this way he can collect Alice's data, and quality data are very valuable for the big-data industry. We will consider in particular the use of the data collections for statistical purposes, namely for extracting general information about the population (and not about Alice as an individual). Of course, the more Alice's data are obfuscated, the less statistical value they have.

We intend to consider both kinds of utility, and study the “three way” optimization problem in the context of $d$-privacy, our approach to local differential privacy  45. Namely, we want to develop methods for producing mechanisms that offer the best trade-off between $d$-privacy, QoS and Stat, at the same time. In order to achieve this goal, we will need to investigate various issues. In particular:

• how to best estimate the original distribution from a collection of noisy data, in order to perform the intended statistical analysis,
• what metrics to use for assessing the statistical value of a distributions (for a given application), in order to reason about Stat, and
• how to compute in an efficient way the best noise from the point of view of the trade-off between $d$-privacy, QoS and Stat.

Estimation of the original distribution

The only methods for the estimation of the original distribution from perturbed data that have been proposed so far in the literature are the iterative Bayesian update (IBU) and the matrix inversion (INV). The IBU is more general and based on solid statistical principles, but it is not ye well known in the in the privacy community, and it has not been studied much in this context. We are motivated to investigate this method because from preliminary experiments it seems more efficient on date obfuscated by geo-indistinguishability mechanisms (cfr. next section). Furthermore, we believe that the IBU is compositional, namely it can deal naturally and efficiently with the combination of data generated by different noisy functions, which is important since in the local model of privacy every user can, in principle, use a different mechanisms or a different level of noise. We intend to establish the foundations of the IBU in the context of privacy, and study its properties like the compositionality mentioned above, and investigate its performance in the state-of-the-art locally differentially private mechanisms.

The central and the local models of differential privacy

Hybrid model

An interesting line of research will be to consider an intermediate model between the local and the central models of differential privacy (cfr. Figure 1). The idea is to define a privacy mechanism based on perturbing the data locally, and then collecting them into a dataset organized as an histogram. We call this model “hibrid” because the collector is trusted like in central differential privacy, but the data are sanitized according to the local model. The resulting dataset would satisfy differential privacy from the point of view of an external observer, while the statistical utility would be as high as in the local model. One further advantage is that the IBU is compositional, hence the datasets sanitized in this way could be combined without any loss of precision in the application of the IBU. In other words, the statistical utility of the union of sanitized datasets is the same as the statistical utility of the sanitized union of datasets, which is of course an improvement (for the law of large numbers) wrt each separate dataset. One important application would be the cooperative sharing of sanitized data owned by different different companies or institution, to the purpose of improving statistical utility while preserving the privacy of their respective datasets.

Geo-indistinguishability: a framework to protect the privacy of the user when dealing with location-based services (a). The framework guarrantees $d$-privacy, a distance-based variant of differential privacy (b). The typical implementation uses (extended) Laplace noise (c).

3.1.2 Geo-indistinguishability

We plan to further develop our line of research on location privacy, and in particular, enhance our framework of geo-indistinguishability 2 (cfr. Figure 2) with mechanisms that allow to take into account sanitize high-dimensional traces without destroying utility (or privacy). One problem with the geo-indistinguishable mechanisms developed so far (the planar Laplace an the planar geometric) is that they add the same noise function uniformly on the map. This is sometimes undesirable: for instance, a user located in a small island in the middle of a lake should generate much more noise to conceal his location, so to report also other locations on the ground, because the adversary knows that it is unlikely that the user is in the water. Furthermore, for the same reason, it does not offer a good protection with respect to re-identification attacks: a user who lives in an isolated place, for instance, can be easily singled out because he reports locations far away from all others. Finally, and this is a common problem with all methods based on DP, the repeated use of the mechanism degrades the privacy, and even when the degradation is linear, as in the case of all DP-based methods, it becomes quickly unacceptable when dealing with highly structured data such as spatio-temporal traces.

Privacy breach in machine learning as a service.

3.1.3 Threats for privacy in machine learning

In recent years several researchers have observed that machine learning models leak information about the training data. In particular, in certain cases an attacker can infer with relatively high probability whether a certain individual participated in the dataset (membership inference attack) od the value of his data (model inversion attack). This can happen even if the attacker has nop access to the internals of the model, i.e., under the black box assumption, which is the typical scenario when machine learning is used as a service (cfr. Figure 3). We plan to develop methods to reason about the information-leakage of training data from deep learning systems, by identifying appropriate measures of leakage and their properties, and use this theoretical framework as a basis for the analysis of attacks and for the development of robust mitigation techniques. More specifically, we aim at:

• Developing compelling case studies based on state-of-the-art algorithms to perform attacks, showcasing the feasibility of uncovering specified sensitive information from a trained software (model) on real data.
• Quantifying information leakage. Based on the uncovered attacks, the amount of sensitive information present in trained software will be quantified and measured. We will study suitable notions of leakage, possibly based on information-theoretical concepts, and establish firm foundations for these.
• Mitigating information leakage. Strategies will be explored to avoid the uncovered attacks and minimize the potential information leakage of a trained model.

3.1.4 Relation between privacy and robustness in machine learning

The relation between privacy and robustness, namely resilience to adversarial attacks, is rather complicated. Indeed the literature on the topic seems contradictory: on the one hand, there are works that show that differential privacy can help to mitigate both the risk of inference attacks and of misclassification (cfr. 52). On the other hand, there are studies that show that there is a trade-off between protection from inference attacks and robustness 54. We intend to shed light on this confusing situation. We believe that the different variations of differential privacy play a role in this apparent contradiction. In particular, preprocessing the training data with $d$-privacy seems to go along with the concept of robustness, because it guarantees that small variations in the input cannot result in large variations in the output, which is exactly the principle of robustness. On the other hand, the addition of random noise on the output result (postprocessing), which is the typical method in central DP, should reduce the precision and therefore increase the possibility of misclassification. We intend to make a taxonomy of the differential privacy variants, in relation to their effect on robustness, and develop a principled approach to protect both privacy and security in an optimal way.

One promising research direction for the deployment of $d$-privacy in this context is to consider Bayesian neural networks (BNNs). These are neural networks with distributions over their weights, which can capture the uncertainty within the learning model, and which provide a natural notion of distance (between distributions) on which we can define a meaningful notion of $d$-privacy. Such neural networks allow to compute an uncertainty estimate along with the output, which is important for safety-critical applications.

3.1.5 Relation between privacy and fairness

Both fairness and privacy are multi-faces notions, assuming different meaning depending on the application domain, on the situation, and on what exactly we want to protect. Fairness, in particular, has received many different definitions, some even in contrast with each other. One of the definitions of fairness is the property that similar “similar” input data produce "similar" outputs. Such notion corresponds closely to $d$-privacy. Other notions of fairness, however, are in opposition to standard differential privacy. This is the case, notably, of Equalized Odds47 and of Equality of False Positives and Equality of False Negatives46. We intend to study a tassonomy of the relation between the main notions of fairness an the various variants of differential privacy. In particular, we intend to study the relation between the recently-introduced notions of causal fairness and causal differential privacy  55.

Another line of research related to privacy and fairness, that we intend to explore, is the design of to pre-process the training set so to obtain machine learning models that are both privacy-friendly and fair.

3.2.1 Non-0-sum games

The framework of $g$-leakage does not take into account two important factors: (a) the loss of the user, and (b) the cost of the attack for the adversary. Regarding (a), we observe that in general the goal of the adversary may not necessarily coincide with causing maximal damage to the user, i.e., there may be a mismatch between the aims of the attacker and what the user tries to protect the most. To model this more general scenario, we had started investigating the interplay between defender and attacker in a game-theoretic setting, starting with the simple case of 0-sum games which corresponds to $g$-leakage. The idea was that, once the simple 0-sum case would be well understood, we would extend the study to the non-0-sum case, that is needed to represent (a) and (b) above. However, we had first to invent and lay the foundations of a new kind of games, the information leakage games42 because the notion of leakage cannot be expressed in terms of payoff in standard game theory. Now that the theory of these new games is well established, we intend to go ahead with our plan, namely study costs and damages of attacks in terms of non-0-sum information leakage games.

3.2.2 Black-box estimation of leakage via machine learning

Most of the works in QIF rely on the so-called white-box assumption, namely, they assume that it is possible to compute exactly the (probabilistic) input-output relation of the system, seen as an information-theoretic channel. This is necessary in order to apply the formula that expresses the leakage. In practical situations, however, it may not be possible to compute the input-output relation, either because the system is too complicated, or simply because it is not accessible. Such scenario is called black-box. The only assumption we make is that the adversary can interact with the system, by feeding to it inputs of his choice and observing the corresponding outputs.

Given the practical interest of the black-box model, we intend to study methods to estimate its leakage. Clearly the standard QIF methods are not applicable. We plan to use, instead, a machine learning approach, continuing the work we started in 10. In particular, we plan to investigate whether we can improve the efficiency of the method proposed by leveraging on the experience that we have acquired with the GANs 53. The idea is to construct a training set and a testing set from the input-output samples collected by interacting with the system, and then build a classifier that learns from the training set to classify the input from the output so to maximize its gain. The measure of its performance on the testing set should then give an estimation of the posterior $g$-vulnerability.

3.3.1 Privacy protection

In 50 we have investigated potential leakage in social networks, namely, the unintended propagation and collection of confidential information. We intend to enrich this model with epistemic aspects, in order to take into account the belief of the users and how it influences the behavior of agents with respect the transmission of information.

Furthermore, we plan to investigate attack models used to reveal a user’s private information, and explore the framework of $g$-leakage to formalize the privacy threats. This will provide the basis to study suitable protection mechanisms.

3.3.2 Polarization and Belief in influence graphs

In social scenarios, a group may shape their beliefs by attributing more value to the opinions of influential figures. This cognitive bias is known as authority bias. Furthermore, in a group with uniform views, users may become extreme by reinforcing one another’s opinions, giving more value to opinions that confirm their own beliefs; another common cognitive bias known as confirmation bias. As a result, social networks can cause their users to become radical and isolated in their own ideological circle causing dangerous splits in society (polarization). We intend to study these dynamics in a model called influence graph, which is a weighted directed graph describing connectivity and influence of each agent over the others. We will consider two kinds of belief updates: the authority belief update, which gives more value to the opinion of agents with higher influence, and the confirmation bias update, which gives more value to the opinion of agents with similar views.

We plan to study the evolution of polarization in these graphs. In particular, we aim at defining a suitable measure of polarization, characterizing graph structures and conditions under which polarization eventually converges to 0 (vanishes), and methods to compute the change in the polarization value over time.

Another purpose of this line of research is how the bias of the agents whose data are being collected impacts the fairness of learning algorithms based on these data.

3.3.3 Concurrency models for the propagation of information

Due to their popularity and computational nature, social networks have exacerbated group polarization. Existing models of group polarization from economics and social psychology state its basic principles and measures 48. Nevertheless, unlike our computational ccp models, they are not suitable for describing the dynamics of agents in distributed systems. Our challenge is to coherently combine our ccp models for epistemic behavior with principles and techniques from economics and social psychology for GP. We plan to develop a ccp-based process calculus which incorporates structures from social networks, such as communication, influence, individual opinions and beliefs, and privacy policies. The expected outcome is a computational model that will allow us to specify the interaction of groups of agents exchanging epistemic information among them and to predict and measure the leakage of private information, as well as the degree of polarization that such group may reach.

7.1.1 Multi-Freq-LDPy

• Name:
  Multiple Frequency Estimation Under Local Differential Privacy in Python
• Keywords:
  Privacy, Python, Benchmarking
• Scientific Description:
  The purpose of Multi-Freq-LDPy is to allow the scientific community to benchmark and experiment with Locally Differentially Private (LDP) frequency (or histogram) estimation mechanisms. Indeed, estimating histograms is a fundamental task in data analysis and data mining that requires collecting and processing data in a continuous manner. In addition to the standard single frequency estimation task, Multi-Freq-LDPy features separate and combined multidimensional and longitudinal data collections, i.e., the frequency estimation of multiple attributes, of a single attribute throughout time, and of multiple attributes throughout time.
• Functional Description:

  Local Differential Privacy (LDP) is a gold standard for achieving local privacy with several real-world implementations by big tech companies such as Google, Apple, and Microsoft. The primary application of LDP is frequency (or histogram) estimation, in which the aggregator estimates the number of times each value has been reported.

  Multi-Freq-LDPy provides an easy-to-use and fast implementation of state-of-the-art LDP mechanisms for frequency estimation of: single attribute (i.e., the building blocks), multiple attributes (i.e., multidimensional data), multiple collections (i.e., longitudinal data), and both multiple attributes/collections.

  Multi-Freq-LDPy is now a stable package, which is built on the well-established Numpy package - a de facto standard for scientific computing in Python - and the Numba package for fast execution.

• URL:
  https://­github.­com/­hharcolezi/­multi-freq-ldpy
• Publication:
  hal-03816212
• Contact:
  Heber Hwang Arcolezi
• Participants:
  Heber Hwang Arcolezi, Jean-François Couchot, Sebastien Gambs, Catuscia Palamidessi, Majid Zolfaghari

7.1.2 LOLOHA

• Name:
  LOngitudinal LOcal HAshing For Locally Private Frequency Monitoring
• Keyword:
  Privacy
• Functional Description:
  This is a Python implementation of our locally differentially private mechanism named LOLOHA. We implemented a private-oriented version named BiLOLOHA and a utility-oriented version named OLOLOHA. We benchmarked our mechanisms in comparison with Google's RAPPOR mechanism and Microsoft's dBitFlipPM mechanism.
• URL:
  https://­github.­com/­hharcolezi/­LOLOHA
• Publication:
  hal-03911550
• Contact:
  Heber Hwang Arcolezi
• Participants:
  Heber Hwang Arcolezi, Sebastien Gambs, Catuscia Palamidessi, Carlos Pinzon Henao

7.1.3 PRiLDP

• Name:
  Privacy Risks of Local Differential Privacy
• Keyword:
  Privacy
• Functional Description:
  This is a Python implementation of two privacy threats we identified against locally differentially private (LDP) mechanisms. We implemented attribute inference attacks as well as re-identification attacks, benchmarking the robustness of five state-of-the-art LDP mechanisms.
• URL:
  https://­github.­com/­hharcolezi/­risks-ldp
• Publication:
  hal-04082592
• Contact:
  Heber Hwang Arcolezi
• Participants:
  Heber Hwang Arcolezi, Sebastien Gambs, Jean-François Couchot, Catuscia Palamidessi

7.1.4 PRIVIC

• Name:
  A privacy-preserving method for incremental collection of location data
• Keyword:
  Privacy
• Functional Description:
  This library contains various tools for the PRIVIC project: the implementation of the Blahut-Arimoto mechanism for metric privacy, the Iterative Bayesian Update, and the implementation of an algorithm performing an incremental collection of data under metric differential privacy protection, and gradual improvement of the mechanism from the point of view of utility.
• URL:
  https://­github.­com/­blitzwas/­PRIVIC
• Publication:
  hal-03968692
• Contact:
  Sayan Biswas
• Participants:
  Sayan Biswas, Catuscia Palamidessi

7.1.5 LDP-FAIRNESS

• Name:
  Impact of Local Differential Privacy on Fairness
• Keywords:
  Privacy, Fairness
• Functional Description:
  This library contains various tools for the study of the impact of Local Differential Privacy on fairness.
• URL:
  https://­github.­com/­hharcolezi/­ldp-fairness-impact
• Publication:
  hal-04175027
• Contact:
  Heber Hwang Arcolezi
• Participants:
  Heber Hwang Arcolezi, Karima Makhlouf, Catuscia Palamidessi

7.1.6 Causal-based Fairness

• Name:
  Causal-based Machine Learning Discrimination Estimation
• Keywords:
  Fairness, Causal discovery
• Functional Description:
  Addressing the problem of fairness is crucial to safely use machine learning algorithms to support decisions with a critical impact on people's lives such as job hiring, child maltreatment, disease diagnosis, loan granting, etc. Several notions of fairness have been defined and examined in the past decade, such as statistical parity and equalized odds. The most recent fairness notions, however, are causal-based and reflect the now widely accepted idea that using causality is necessary to appropriately address the problem of fairness. The big impediment to the use of causality to address fairness, however, is the unavailability of the causal model (typically represented as a causal graph). This library contains the software tools that implement all required steps to estimate discrimination using a causal approach, including, the causal discovery, the adjustment of the causal model, and the estimation of discrimination. The software is to be deployed as a web application which makes it accessible online without any required setup on the user side.
• Publication:
  hal-04355882
• Contact:
  Sami Zhioua
• Participants:
  Raluca Panainte, Yassine Turki, Sami Zhioua

7.1.7 Polarization

• Name:
  A model for polarization
• Keyword:
  Social network
• Functional Description:
  This is a Python implementation of our polarization model. The implementation is parametric in the social influence graph and belief update representing the social network and it allows for the simulation of belief evolution and measuring the polarization of the network.
• URL:
  https://­github.­com/­Sirquini/­Polarization
• Publication:
  hal-03872692
• Contact:
  Frank Valencia
• Participants:
  Frank Valencia, Mario Ferreira Alvim Junior, Sophia Knight, Santiago Quintero

7.1.8 GMeet

• Name:
  GMeet Algorithms
• Keyword:
  Distributed computing
• Functional Description:
  This is a Python library containing the implementation of our methods to compute distributed knowledge in multi-agent systems. The implementation allows for experimental comparison between the different methods on randomly generated inputs.
• URL:
  https://­caph1993.­github.­io/­GMeetMono/
• Publication:
  hal-02422624
• Contact:
  Frank Valencia

7.1.9 Fairness-Accuracy

• Name:
  On the trade-off between Fairness and Accuracy
• Keywords:
  Fairness, Machine learning
• Functional Description:

  This software is composed by two main modules that serve the following purposes:

  (1) To visualize the perimeter of all possible machine learning models in the Equal Opportunity - Accuracy space, and to show that, for certain distributions, Equal Opportunity implies that the best Accuracy achievable is that of a trivial model.

  (2) To compute the Pareto optimality between Equal Opportunity Difference and Accuracy.

• Publication:
  hal-04308195
• Contact:
  Catuscia Palamidessi
• Participants:
  Carlos Pinzon Henao, Catuscia Palamidessi, Pablo Piantanida, Frank Valencia

FACTS

Participants: Frank Valencia, Mario Sergio Ferreira Alvim Junior.

• Title:
  Foundational Approach to Cognition in Today's Society.
• Program:
  ECOS NORD.
• Duration:
  2019–2023.
• Coordinator:
  Frank Valencia.
• Type of funding:
  The project provides funds for mobility between France and Colombia and dissemination of the results.
• Other partners:
  Sorbonne University and Universidad Javeriana de Cali, Colombia.
• Objective:
  This projects aims at studying the phenomenon of “Group Polarization”; the tendency for a group to learn or acquire beliefs or to make decisions that are more extreme than the initial inclinations of its members.

PROMUEVA

Participants: Frank Valencia, Carlos Pinzon Henao.

• Title:
  Computational Models for Polarization on Social Networks Applied To Colombia Civil Unrest.
• Duration:
  2022–2026.
• Coordinator:
  Frank Valencia.
• Source of funding:
  Minciencias - Ministerio de Ciencia Tecnología e Innovación, Colombia.
• Other partners:
  Universidad Javeriana de Cali, Colombia. Universidad del Valle, Colombia.
• Objective:
  This projects aims at developing computational frameworks for modeling belief evolution and measuring polarization in social networks.

10.2.1 Visits of international scientists

10.2.2 Visits to international teams

10.3.1 Horizon Europe

10.3.2 H2020 projects

10.3.3 Other european programs/initiatives

iPOP

Participants: Catuscia Palamidessi, Sami Zhioua, Héber Arcolezi, Sayan Biswas, Ruta Binkyte-Sadauskiene, Karima Makhlouf.

• Web Page:
  Link
• Title:
  Interdisciplinary Project on Privacy
• Program:
  PEPR Cybersecurity
• Duration:
  1 October 2022 - 30 September 2028
• Coordinators:
  Antoine Boutet (Insa-Lyon) - Vincent Roca (Inria)
• Partners:
  • Inria
  • CNRS
  • CNIL
  • INSA-Centre Val de Loire (CVL)
  • INSA-Lyon
  • Université Grenoble Alpes
  • Université de Lille
  • Université Rennes 1
  • Université de Versailles Saint-Quentin-en-Yvelines
• Inria COMETE contact:
  Catuscia Palamidessi
• Description:
  Digital technologies provide services that can greatly increase quality of life (e.g. connected e-health devices, location based services or personal assistants). However, these services can also raise major privacy risks, as they involve personal data, or even sensitive data. Indeed, this notion of personal data is the cornerstone of French and European regulations, since processing such data triggers a series of obligations that the data controller must abide by. This raises many multidisciplinary issues, as the challenges are not only technological, but also societal, judiciary, economic, political and ethical. The objectives of this project are thus to study the threats on privacy that have been introduced by these new services, and to conceive theoretical and technical privacy-preserving solutions that are compatible with French and European regulations, that preserve the quality of experience of the users. These solutions will be deployed and assessed, both on the technological and legal sides, and on their societal acceptability. In order to achieve these objectives, we adopt an interdisciplinary approach, bringing together many diverse fields: computer science, technology, engineering, social sciences, economy and law.

FedMalin

Participants: Catuscia Palamidessi, Sami Zhioua, Héber Arcolezi, Sayan Biswas, Ruta Binkyte-Sadauskiene, Karima Makhlouf.

• Web Page:
  Link
• Title:
  Federated MAchine Learning over the INternet
• Program:
  Inria Challenge
• Duration:
  1 October 2022 - 30 September 2026
• Coordinators:
  Aurélien Bellet and Giovanni Neglia
• Partners:
  • ARGO (Inria Paris)
  • COATI (Inria Sophia)
  • COMETE (Inria Saclay)
  • EPIONE (Inria Sophia)
  • MAGNET (Inria Lille)
  • MARACAS (Inria Lyon)
  • NEO (Inria Sophia)
  • SPIRALS (Inria Lille)
  • TRIBE (Inria Saclay)
  • WIDE (Inria Rennes)
• Inria COMETE contact:
  Catuscia Palamidessi
• Description:
  In many use-cases of Machine Learning (ML), data is naturally decentralized: medical data is collected and stored by different hospitals, crowdsensed data is generated by personal devices, etc. Federated Learning (FL) has recently emerged as a novel paradigm where a set of entities with local datasets collaboratively train ML models while keeping their data decentralized. FedMalin aims to push FL research and concrete use-cases through a multidisciplinary consortium involving expertise in ML, distributed systems, privacy and security, networks, and medicine. We propose to address a number of challenges that arise when FL is deployed over the Internet, including privacy and fairness, energy consumption, personalization, and location/time dependencies. FedMalin will also contribute to the development of open-source tools for FL experimentation and real-world deployments, and use them for concrete applications in medicine and crowdsensing.

DIFPRIPOS

Participants: Catuscia Palamidessi.

• Title:
  Making PostgreSQL Differentially Private for Transparent AI
• Program:
  ANR blanc.
• Duration:
  2023–2026
• Coordinator:
  Jen-François Couchot (Université de Franche-Comté).
• Inria COMETE PI:
  Catuscia Palamidessi.
• Other partners:
  Université de Franche-Comté, LIRIS / INSA-Lyon, The DALIBO cooperative society, and LIFO / INSA-CVL.
• Objective:
  The general objective is to implement and to evaluate a "privacy preserving" approach for interpreting SQL queries in the sense of differential confidentiality that can be integrated into PostgreSQL.

11.1.1 Scientific events: organisation

• Catuscia Palamidessi has co-organized and co-chaired:
  • the Fourth AAAI Workshop on Privacy-Preserving Artificial Intelligence. Washington DC, USA. February 2023.
  • the session on formal methods at the Franco-Japanese workshop on Cybersecurity. Bordeaux, France. November 29th - December 1st, 2023.
• Sami Zhioua, Catuscia Palamidessi, and other members of Comete have organized and co-chaired the Second Ethical AI @Comete workshop. Palaiseau, France. November 23-24, 2023.
• Frank Valencia has organized and chaired the Promueva Workshop on Models for Social Networks at École Polytechnique, Sorbonne Paris Nord, and IRCAM. June 12-23, 2023.
• Frank Valencia has co-organized and co-chaired the Promueva Public Workshop on Polarization in Social Networks and AI Impact, at Universidad Javeriana Cali, August 23, 2023.
• Sami Zhioua has been member of the Organizing Committee of the European Workshop on Algorithmic Fairness (EWAF’23) June 2023
• Héber Hwang Arcolezi has co-organized the 13th French Workshop on Privacy, at University Bourgogne Franche-Comté, June 12-15, 2023.

11.1.2 Scientific events: selection

• Catuscia Palamidessi is/has been program committee member of:
  • PPAI 2024. The 5th AAAI Workshop on Privacy-Preserving Artificial Intelligence
  • PETS 2024, the international conference on Privacy Enancing Technologies.
  • FOSSACS 2024, the International Conference on Foundations of Software Science and Computation Structures.
  • CSF 2024, the international IEEE Symposium on Computer Security Foundations.
  • LICS 2023, the international ACM/IEEE Conference on Logic In Computer Science.
  • CSF 2023, the international IEEE Symposium on Computer Security Foundations.
  • SDS 2023. The 10th IEEE Swiss Conference on Data Science.
  • WiL 2023. The Women in Logic Workshop.
• Frank Valencia has been program committee member of:
  • ICLP-DC 2023, the Doctoral Program International Conference on Logic Programming.
  • FOSSACS 2023, the International Conference on Foundations of Software Science and Computation Structures
• Héber Hwang Arcolezi is/has been program committee member of:
  • PETs 2024.
  • ICLR Tiny Paper Track 2024.
  • NeurIPS 2023.
  • CCS Poster Track 2023.
  • ICLR Tiny Paper Track 2023.
  • ECML / PKDD 2023.
  • FAccT 2023.
  • PPAI Workshop 2023.
  • SDS 2023.
• Sami Zhioua has been program committee member of the IEEE Afro-Mediterranean Conference on Artificial Intelligence, 2023.

11.1.3 Journal

Catuscia palamidessi is member of the editorial board of:

• (Since 2022) ACM Transactions on Privacy and Security.
• (Since 2022) TheoretiCS.
• (Since 2020) IEEE Transactions on Dependable and Secure Computing.
• (Since 2020) Journal of Logical and Algebraic Methods in Programming, Elsevier.
• (Since 2015) Acta Informatica, Springer.
• (Since 2006) Mathematical Structures in Computer Science, CUP.

11.1.4 Invited talks

• Catuscia Palamidessi has been keynote invited speaker at:
  • ACM CODASPY 2023. The 13th ACM Conference on Data and Application Security and Privacy. April 24 - 26, 2023. Charlotte, NC, United States.
  • MobiliT.AI 2023. The Annual Forum on Artificial Intelligence for critical applcations. 30-31 May 2023. MEETT Toulouse, France.
  • HYPER 2023. Workshop on Hyperproperties. 18 July 2023. Paris, France.
  • The PhD day for PhD students in ICST of the plateau de Saclay (Paris Saclay University and IP Paris). 20 june 2023. AgroParisTech, Palaiseau, France.
  • The Franco-Japanese workshop on Distributed Ledger Technologies. 14-15 November, 2023. Tokio, Japan.
• Sami Zhioua has been invited speaker at:
  • Ethical Public Robots and AI (EPURAI), Paris, France, 2023.
  • LIX École Polytechnique Monthly Seminar, Palaiseau, France, 2023.
  • Tau Team monthly seminar Université Paris-Saclay, 2023.

11.1.5 Leadership within the scientific community

Catuscia palamidessi is:

• President of SIGLOG, the ACM Special Interest Group on Logic and Computation.
• Co-chair of the of the 6th edition of the CNIL-Inria Privacy Award.
• Member of steering committees of:
  • (Since 2016) CONCUR, the International Conference in Concurrency Theory.
  • (Since 2015) EACSL, the European Association for Computer Science Logics.

11.1.6 Scientific expertise

Catuscia Palamidessi has been/is:

• Member of the Estonian Research Council for the evaluation process of the research funding applications in 2023, in the fields of Mathematics, Computer Science and Informatics.
• Member of the Romanian Research Council for the evaluation process of the research funding applications in the Innovation and Development program for 2023.
• Member of the Scientific Committee of ANR - AAPG 2024. Evaluation of project proposal in the context of Artificial Intelligence and Data Science - CES 23.
• Member of the Board of Trustees of the IMDEA Software Institute, Madrid, Spain. Since 2021.
• Member of the Sci. Adv. Board of CISPA, Helmholtz Center for Information Security. Saarbruecken, Germany. Since 2019.

11.1.7 Research administration

Catuscia palamidessi has been chair of the Scientific Committee of Inria Saclay during 2021-23.

11.2.1 Teaching

• Catuscia Palamidessi has given a Tutorial on Privacy at the GDR SRD Summer School on Distributed Learning, Lyon, France, September 2023.
• Frank Valencia has been teaching since 2019 Concurrency Theory and Computability at the Master's program of Computer Science at the University Javeriana Cali for a total of 128 hours per year.
• Héber Hwang Arcolezi was a teaching assistant for the INF361 "Introduction à l informatique" course at École Polytechnique, from April to June 2023.
• Sami Zhioua has given the following courses:
  • CSE 101 : Computer Programming I
    • year:
      2023-2024
    • School/University:
      École Polytechnique
    • Role:
      TD main instructor
  • CSE 102 : Computer Programming II
    • year:
      2023-2024
    • School/University:
      École Polytechnique
    • Role:
      TD main instructor
  • INF473X - Modal d'informatique - Cybersecurity - The Hacking Xperience
    • year:
      2023-2024
    • School/University:
      École Polytechnique
    • Role:
      TD main instructor and grader
  • Éthique dans l'apprentissage machine
    • year:
      2023-2024
    • School/University:
      Aivancity
    • Role:
      Course design and main instructor
  • CSE 103 : Introduction to Algorithms
    • year:
      2022-2023
    • School/University:
      École Polytechnique
    • Role:
      TD main instructor
  • C319: Fair Machine Learning
    • year:
      2022-2023
    • School/University:
      Aivancity
    • Role:
      Course design and main instructor (with Ruta Binkyte)

11.2.2 Supervision

Supervision of PhD students

• (2023-) Ramon Goncalves Gonze . Co-supervised by Catuscia palamidessi and Mario Alvim. Subject: Tension between privacy and utility in Census data.
• (2022-) Andreas Athanasiou . Co-supervised by Catuscia palamidessi and Kostantinos Chatzikokolakis. Subject: The shuffle model for metric differential privacy.
• (2021-) Karima Makhlouf . Co-supervised by Catuscia palamidessi and Heber Hwang Arcolezi . Subject: Relation between privacy and fairness in machine learning. Karima received the Best Poster Award at the workshop on Computing, Data, and Artificial Intelligence organized by the IPP doctoral schools in 2022.
• (2020-23) Ruta Binkyte-Sadauskiene . Co-supervised by Catuscia palamidessi and Sami Zhioua. Subject: Advancing Ethical AI: Methods for fairness enhancement leveraging on causality and under privacy constraints. Ruta defended her thesis in December 2023 and she is now a Postdoctoral Fellow at CISPA, Germany.
• (2020-23) Carlos Pinzon Henao . Co-supervised by Catuscia palamidessi, Frank Valencia and Pablo Piantanida. Subject: Exploring fairness and privacy in machine learning. Carlos defended his thesis in December 2023 and he is currently considering various job offers. He received the accessit to the 2023 Doctoral Prize awarded by the University of Paris Saclay and the Polytechnic Institute of Paris, for his work 9, which constitutes an important part of his thesis.
• (2019-23) Ganesh Del Grosso Guzman . Co-supervised by Catuscia palamidessi and Pablo Piantanida. Subject: Leakage of sensitive data from deep neural networks. Ganesh defended his thesis in November 2023 and he is now working as Researcher and Developer at Ericsson Telecommunications Inc.
• (2020-23) Sayan Biswas . Supervised by Catuscia palamidessi. Subject: Understanding and optimizing the trade-off between privacy and utility from a foundational perspective. Sayan defended his thesis in October 2023 and he is now a Postdoctoral Fellow at EPFL, Switzerland.
• (2019-23) Federica Granese . Co-supervised by Catuscia Palamidessi, Pablo Piantanida and Daniele Gorla. Subject: Towards securing machine learning algorithms. Federica defended her thesis in April 2023 and she is now a Postdoctoral Fellow in AI at IRD - UMMISCO, Sorbonne Université. Her paper 5, which constitute an important part of her thesis, was accepted as hotspot presentation at NeurIPS 2020. Only 5% of the paper accepted at NeurIPS have this privilege.

Supervision of postdocs and junior researchers

• (2022–2023) Szilvia Lestyan, postdoc.
• (2022–23) Selene Cerna, postdoc.
• (2022–23) Héber Hwang Arcolezi, postdoc.
• (2021–) Sami Zhioua, researcher CDD.
• (2020–) Gangsoo Zeong, junior researcher.

11.2.3 Juries

Catuscia Palamidessi has been:

• Member of the jury for the HDR defense of Benjamin Smith. October 2023.
• Member of the jury for the PhD defense of Angelo Saadeh. July 2023.
• Member of the jury for the PhD defense of Chrysoula Kosma. December 2023.
• Rapporteur and member of the jury for the PhD defense of Yakini Tchouka. December 2023.
• Rapporteur and member of the jury for the PhD defense of Sara Taki. December 2023.

International journals

• 11 articleG.Guilherme Alves, F.Fabien Bernier, M.Miguel Couceiro, K.Karima Makhlouf, C.Catuscia Palamidessi and S.Sami Zhioua. Survey on Fairness Notions and Related Tensions.EURO journal on decision processes2023HALDOI
• 12 articleM. S.Mário S. Alvim, B.Bernardo Amorim, S.Sophia Knight, S.Santiago Quintero and F.Frank Valencia. A Formal Model for Polarization under Confirmation Bias in Social Networks.Logical Methods in Computer ScienceMarch 2023HALDOIback to textback to textback to text
• 13 articleH. H.Héber H. Arcolezi, S.Sébastien Gambs, J.-F.Jean-François Couchot and C.Catuscia Palamidessi. On the Risks of Collecting Multidimensional Data Under Local Differential Privacy.Proceedings of the VLDB Endowment (PVLDB)165January 2023, 1126 - 1139HALDOIback to text
• 14 articleS.Sayan Biswas and C.Catuscia Palamidessi. PRIVIC: A privacy-preserving method for incremental collection of location data.Proceedings on Privacy Enhancing Technologies20241October 2023, 582–596HALDOIback to text
• 15 articleG.Ganesh Del Grosso, G.George Pichler, C.Catuscia Palamidessi and P.Pablo Piantanida. Bounding Information Leakage in Machine Learning.Neurocomputing5342023, 1-17HALDOIback to text
• 16 articleN.Natalia Díaz-Rodríguez, R.Rūta Binkytė, W.Wafae Bakkali, S.Sannidhi Bookseller, P.Paola Tubaro, A.Andrius Bacevičius, S.Sami Zhioua and R.Raja Chatila. Gender and sex bias in COVID-19 epidemiological data through the lens of causality.Information Processing and Management603May 2023, 103276HALDOIback to text
• 17 articleN.Natasha Fernandes, A.Annabelle Mciver, C.Catuscia Palamidessi and M.Ming Ding. Universal optimality and robust utility bounds for metric differential privacy.Journal of Computer SecurityJuly 2023, 1-42HALDOIback to text
• 18 articleF.Filippo Galli, K.Kangsoo Jung, S.Sayan Biswas, C.Catuscia Palamidessi and T.Tommaso Cucinotta. Advancing Personalized Federated Learning: Group Privacy, Fairness, and Beyond.SN Computer Science46October 2023, 831HALDOIback to text
• 19 articleC.Carlos Pinzón, C.Catuscia Palamidessi, P.Pablo Piantanida and F.Frank Valencia. On the incompatibility of accuracy and equal opportunity.Machine LearningMay 2023HALDOIback to text

Invited conferences

• 20 inproceedingsC.Catuscia Palamidessi. Local Methods for Privacy Protection and Impact on Fairness.CODASPY 2023 - Thirteenth ACM Conference on Data and Application Security and PrivacyCharlotte NC, United StatesACMApril 2023HALDOI

International peer-reviewed conferences

• 21 inproceedingsM. S.Mário S. Alvim, N.Natasha Fernandes, B. D.Bruno D Nogueira, C.Catuscia Palamidessi and T. V.Thiago V A Silva. On the Duality of Privacy and Fairness.International Conference on AI and the Digital Economy (CADE 2023),CADE 2023 - International Conference on AI and the Digital EconomyVenice, ItalyJune 2023, p. 46 – 48HALback to text
• 22 inproceedingsH. H.Héber Hwang Arcolezi, S.Selene Cerna and C.Catuscia Palamidessi. On the Utility Gain of Iterative Bayesian Update for Locally Differentially Private Mechanisms.Lecture Notes in Computer ScienceDBSec 2023 - 37th IFIP Annual Conference on Data and Applications Security and PrivacyLNCS-13942Data and Applications Security and Privacy XXXVIISophia Antipolis, FranceSpringer Nature SwitzerlandJuly 2023, 165-183HALDOIback to text
• 23 inproceedingsH. H.Héber Hwang Arcolezi, K.Karima Makhlouf and C.Catuscia Palamidessi. (Local) Differential Privacy has NO Disparate Impact on Fairness.Lecture Notes in Computer ScienceDBSec 2023 - 37th IFIP Annual Conference on Data and Applications Security and PrivacyLNCS-13942Data and Applications Security and Privacy XXXVIISOPHIA ANTIPOLIS, FranceSpringer Nature SwitzerlandJuly 2023, 3-21HALDOIback to textback to text
• 24 inproceedingsH. H.Héber Hwang Arcolezi, C.Catuscia Palamidessi, C.Carlos Pinzón and S.Sébastien Gambs. Frequency Estimation of Evolving Data Under Local Differential Privacy.EDBT 2023 - 26th International Conference on Extending Database TechnologyIoánnina, GreeceMay 2023, 512-525HALDOIback to text
• 25 inproceedingsS.Selene Cerna and C.Catuscia Palamidessi. On the Application and Impact of differential privacy and Fairness in Ambulance Engagement Time Prediction.Proceedings of the Tiny Papers Track at ICLRICLR 2023 - The First Tiny Papers Track at ICLR 2023Proceedings of the Tiny Papers Track at ICLRKigali, Rwanda2023HAL
• 26 inproceedingsK.Konstantinos Chatzikokolakis, G.Giovanni Cherubin, C.Catuscia Palamidessi and C.Carmela Troncoso. Bayes Security: A Not So Average Metric.Proceedings of the IEEE 36th Computer Security Foundations Symposium (CSF)CSF 2023 - 36th IEEE Computer Security Foundations SymposiumProceedings of the IEEE 36th Computer Security Foundations Symposium (CSF)Dubrovnik, CroatiaIEEE2023HALDOIback to text
• 27 inproceedingsF.Filippo Galli, S.Sayan Biswas, K.Kangsoo Jung, T.Tommaso Cucinotta and C.Catuscia Palamidessi. Group privacy for personalized federated learning.Proceedings of the 9th International Conference on Information Systems Security and Privacy - ICISSPLisbon, PortugalSCITEPRESS - Science and Technology Publications2023, 252-263HALDOIback to text
• 28 inproceedingsD.Daniele Gorla, L.Louis Jalouzot, F.Federica Granese, C.Catuscia Palamidessi and P.Pablo Piantanida. On the (Im)Possibility of Estimating Various Notions of Differential Privacy (short paper).Proceedings of the 24th Italian Conference on Theoretical Computer ScienceICTCS 2023 - The 24th Italian Conference on Theoretical Computer Science3587Proceedings of the 24th Italian Conference on Theoretical Computer SciencePalermo, Italy2023, 219--224HAL
• 29 inproceedingsM.Mireya Jurado, R. G.Ramon Goncalves Gonze, M. S.Mário S Alvim and C.Catuscia Palamidessi. Analyzing the Shuffle Model Through the Lens of Quantitative Information Flow.Proceedings of the IEEE 36th Computer Security Foundations Symposium (CSF)CSF 2023 - 36th IEEE Computer Security Foundations SymposiumProceedings of the IEEE 36th Computer Security Foundations Symposium (CSF)Dubrovnik, CroatiaIEEEMay 2023, 423-438HALDOIback to text
• 30 inproceedingsS.Sebastian Simon, C.Cezara Petrui, C.Carlos Pinzón and C.Catuscia Palamidessi. Obfuscation Padding Schemes that Minimize Rényi Min-Entropy for Privacy.Lecture Notes in Computer ScienceISPEC 2023 - The 18th International Conference on Information Security Practice and ExperienceLNCS-14341Information Security Practice and ExperienceCoppenhagen, DenmarkSpringer Nature SingaporeNovember 2023, 74-90HALDOIback to text

Doctoral dissertations and habilitation theses

• 31 thesisR.Rūta Binkytė. Advancing Ethical AI: Methods for fairness enhancement leveraging on causality and under privacy constraints.Ecole Polytechnique (EDX)December 2023HAL
• 32 thesisS.Sayan Biswas. Understanding and optimizing the trade-off between privacy and utility from a foundational perspective.Ecole Polytechnique (EDX)October 2023HAL
• 33 thesisG.Ganesh Del Grosso. Leakage of Sensitive Data from Deep Neural Networks.Ecole Polytechnique (EDX)November 2023HAL
• 34 thesisF.Federica Granese. Towards Securing Machine Learning Algorithms through Misclassification Detection and Adversarial Attack Detection.Ecole Polytechnique (EDX); Sapienza University of RomeApril 2023HAL
• 35 thesisC.Carlos Pinzón. Exploring fairness and privacy in machine learning.Ecole PolytechniqueDecember 2023HAL

Reports & preprints

• 36 miscR.Rūta Binkytė, L.Ljupcho Grozdanovski and S.Sami Zhioua. On the Need and Applicability of Causality for Fair Machine Learning.November 2023HAL
• 37 miscR.Rūta Binkytė, S.Sami Zhioua and Y.Yassine Turki. Dissecting Causal Biases.January 2024HAL
• 38 miscK.Karima Makhlouf, H.Héber Hwang Arcolezi, S.Sami Zhioua, G. B.Ghassen Ben Brahim and C.Catuscia Palamidessi. On the Impact of Multi-dimensional Local Differential Privacy on Fairness.December 2023HAL
• 39 miscR.Raluca Panainte, Y.Yassine Turki and S.Sami Zhioua. A Web Application Software for Causal-based Machine Learning Discrimination Estimation.February 2024HAL
• 40 miscC.Carlos Pinzón and K.Kangsoo Jung. Fast Python sampler for the von Mises Fisher distribution.August 2023HAL
• 41 miscS.Sami Zhioua and R.Rūta Binkytė. Shedding light on underrepresentation and Sampling Bias in machine learning.December 2023HAL