Scientific Publications

Abstract: Text-to-image generative models are becoming increasingly popular and accessible to the general public. As these models see large-scale deployments, it is necessary to deeply investigate their safety and fairness to not disseminate and perpetuate any kind of biases. However, existing works focus on detecting closed sets of biases defined a priori, limiting the studies to well-known concepts. In this paper, we tackle the challenge of open-set bias detection in text-to-image generative models presenting OpenBias, a new pipeline that identifies and quantifies the severity of biases agnostically, without access to any precompiled set.
OpenBias has three stages. In the first phase, we leverage a Large Language Model (LLM) to propose biases given a set of captions. Secondly, the target generative model produces images using the same set of captions. Lastly, a Vision Question Answering model recognizes the presence and extent of the previously proposed biases. We study the behavior of Stable Diffusion 1.5, 2, and XL emphasizing new biases, never investigated before. Via quantitative experiments, we demonstrate that OpenBias agrees with current closed-set bias detection methods and human judgement.

Type of Publication: conference paper

Title of Journal: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024

Abstract: We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node re-lations in an end-to-end fashion for challenging graph-constrained architectural layout generation tasks. The proposed graph-Transformer-based generator includes a novel graph Transformer encoder that combines graph convolutions and self-attentions in aTransformer to model both local and global interactions across con-nected and non-connected graph nodes. Specifically, the proposed connected node attention (CNA) and non-connected node attention(NNA) aim to capture the global relations across connected nodes and non-connected nodes in the input graph, respectively. Theproposed graph modeling block (GMB) aims to exploit local vertex interactions based on a house layout topology. Moreover, we pro-pose a new node classification-based discriminator to preserve thehigh-level semantic and discriminative node features for different house components. To maintain the relative spatial relationships between ground truth and predicted graphs, we also propose anovel graph-based cycle-consistency loss. Finally, we propose anovel self-guided pre-training method for graph representation learning. This approach involves simultaneous masking of nodesand edges at an elevated mask ratio (i.e., 40%) and their subsequent reconstruction using an asymmetric graph-centric autoencoder architecture. This method markedly improves the model’s learn-ing proficiency and expediency. Experiments on three challenginggraph-constrained architectural layout generation tasks (i.e., houselayout generation, house roof generation, and building layout gen-eration) with three public datasets demonstrate the effectiveness of the proposed method in terms of objective quantitative scoresand subjective visual realism. New state-of-the-art results are established by large margins on these three tasks.

Type of Publication: Journal Article

Title of Journal: IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(6), 4298-4313, 2024.

Abstract: Atmospheric radiative transfer models (RTMs) are widely used in satellite data processing to correct for the scattering and absorption effects caused by aerosols and gas molecules in the Earth’s atmosphere. As the complexity of RTMs grows and the requirements for future Earth Observation missions become more demanding, the conventional lookup-table (LUT) interpolation approach faces important challenges. Emulators have been suggested as an alternative to LUT interpolation, but they are still too slow for operational satellite data processing. Our research introduces a solution that harnesses the power of multifidelity methods to improve the accuracy and runtime of Gaussian process (GP) emulators. We investigate the impact of the number of fidelity layers, dimensionality reduction, and training dataset size on the performance of multifidelity GP emulators. We find that an optimal multifidelity emulator can achieve relative errors in surface reflectance below 0.5% and performs atmospheric correction of hyperspectral PRISMA satellite data (one million pixels) in a few minutes. Additionally, we provide a suite of functions and tools for automating the creation and generation of atmospheric RTM emulators.

Type of Publication: publication

Title of Journal: IEEE Transactions on Geoscience and Remote Sensing, 61, 1-10, 2023.

Abstract: We explore brokerage between traders in an online learning framework. At any round t, two traders meet to exchange an asset, provided the exchange is mutually beneficial. The broker proposes a trading price, and each trader tries to sell their asset or buy the asset from the other party, depending on whether the price is higher or lower than their private valuations. A trade happens if one trader is willing to sell and the other is willing to buy at the proposed price.

Previous work provided guidance to a broker aiming at enhancing traders’ total earnings by maximizing the gain from trade, defined as the sum of the traders’ net utilities after each interaction. In contrast, we investigate how the broker should behave to maximize the trading volume, i.e., the total number of trades.

We model the traders’ valuations as an i.i.d. process with an unknown distribution. If the traders’ valuations are revealed after each interaction (full-feedback), and the traders’ valuations cumulative distribution function (cdf) is continuous, we provide an algorithm achieving logarithmic regret and show its optimality up to constant factors.

If only their willingness to sell or buy at the proposed price is revealed after each interaction (2-bitfeedback), we provide an algorithm achieving polylogarithmic regret when the traders’ valuations cdf is Lipschitz and show that this rate is near-optimal.

We complement our results by analyzing the implications of dropping the regularity assumptions on the unknown traders’ valuations cdf. If we drop the continuous cdf assumption, the regret rate degrades to Θ(√T) in the full-feedback case, where T is the time horizon. If we drop the Lipschitz cdf assumption, learning becomes impossible in the 2-bit feedback case.

Type of Publication: publication

Abstract: We study the role of contextual information in the online learning problem of brokerage between traders. At each round, two traders arrive with secret valuations about an asset they wish to trade. The broker suggests a trading price based on contextual data about the asset. Then, the traders decide to buy or sell depending on whether their valuations are higher or lower than the brokerage price. We assume the market value of traded assets is an unknown line ar function of a d-dimensional vector representing the contextual information available to the broker. Additionally, we model traders’ valuations as independent bounded zero-mean perturbations of the asset’s market value, allowing for potentially different unknown distributions across traders and time steps. Consistently with the existing online learning literature, we evaluate the performance of a learning algorithm with the regret with respect to the gain from trade. If the noise distributions admit densities bounded by someconstant L, then, for any time horizon T:

• If the agents’ valuations are revealed after each interact ion, we provide an algorithm achieving O(LdlnT) regret, and show a corresponding matching lower bound of Ω(LdlnT).
• If only their willingness to sell or buy at the proposed price is revealed after each interaction, we provide an algorithm achieving O(√LdTlnT) regret, and show that this rate is optimal (up to logarithmic factors), via a lower bound of Ω(√LdT).

Type of Publication: publication

Abstract: In online bilateral trade, a platform posts prices to incoming pairs of buyers and sellers that have private valuations for a certain good. If the price is lower than the buyers’ valuation and higher than the sellers’ valuation, then a trade takes place. Previous work focused on the platform perspective, with the goal of setting prices maximizing the gain from trade (the sum of sellers’ and buyers’ utilities). Gain from trade is, however, potentially unfair to traders, as they may receive highly uneven shares of the total utility. In this work we enforce fairness by rewarding the platform with the fair gain from trade, defined as the minimum between sellers’ and buyers’ utilities. After showing that any no-regret learning algorithm designed to maximize the sum of the utilities may fail badly with fair gain from trade, we present our main contribution: a complete characterization of the regret regimes for fair gain from trade when, after each interaction, the platform only learns whether each trader accepted the current price. Specifically, we prove the following regret bounds: Θ(ln T ) in the deterministic setting, Ω(T)in the stochastic setting, and ̃Θ(T2~3)in the stochastic setting when sellers’ and buyers’ valuations are independent of each other. We conclude by providing tight regret bounds when, after each interaction, the platform is allowed to observe the true traders’ valuations.

Type of Publication: publication

Abstract: Collaborative self-supervised learning has recently become feasible in highly distributed environments by dividing the network layers between client devices and a central server. However, state-of-the-art methods, such as MocoSFL, are optimized for network division at the initial layers, which decreases the protection of the client data and increases communication overhead. In this paper, we demonstrate that splitting depth is crucial for maintaining privacy and communication efficiency in distributed training. We also show that MocoSFL suffers from a catastrophic quality deterioration for the minimal communication overhead. As a remedy, we introduce Momentum-Aligned contrastive Split Federated Learning (MonAcoSFL), which aligns online and momentum client models during training procedure. Consequently, we achieve state-of-the-art accuracy while significantly reducing the communication overhead, making MonAcoSFL more practical in real-world scenarios.

Type of Publication: publication

Title of Journal: International Conference on Learning Representations (ICLR), Vienna Austria, 7-11.05.2024

Abstract: Class-incremental learning is becoming more popular as it helps models widen their applicability while not forgetting what they already know. A trend in this area is to use a mixture-of-expert technique, where different models work together to solve the task. However, the experts are usually trained all at once using whole task data, which makes them all prone to forgetting and increasing computational burden. To address this limitation, we introduce a novel approach named SEED. SEED selects only one, the most optimal expert for a considered task, and uses data from this task to fine-tune only this expert. For this purpose, each expert represents each class with a Gaussian distribution, and the optimal expert is selected based on the similarity of those distributions. Consequently, SEED increases diversity and heterogeneity within the experts while maintaining the high stability of this ensemble method. The extensive experiments demonstrate that SEED achieves state-of-the-art performance in exemplar-free settings across various scenarios, showing the potential of expert diversification through data in continual learning.

Type of Publication: publication

Title of Journal: International Conference on Learning Representations (ICLR), Vienna Austria, 7-11.05.2024