Publication Types:

Towards replication-robust analytics markets

2025ArticlePreprint
T Falconer, P Pinson, J Kazempour
preprint, under review
Publication year: 2025

Despite widespread adoption of machine learning, many firms face the common challenge of relevant datasets being distributed amongst market competitors whom are reluctant to share information. Accordingly, recent works propose analytics markets as a way to provide monetary incentives for collaboration, where agents share features and are rewarded based on their contribution to improving the predictions of others. These contributions are determined by their relative Shapley value, computed by treating features as players and their interactions as a cooperative game. However, this setup is known to incite agents to strategically replicate their data and act under multiple false identities to increase their own revenue whilst diminishing that of others, which limits the viability of these markets in practice.
In this work, we develop an analytics market robust to such strategic replication for supervised learning problems. We adopt Pearl’s do-calculus from causal inference to refine the cooperative game by differentiating between observational and interventional conditional probabilities. As a result, we derive Shapley value-based rewards that deter replication by design.

Predict-and-optimize robust unit commitment with statistical guarantees via weight combination

2025ArticlePreprint
R. Xie, Y. Chen, and P. Pinson
preprint, under review
Publication year: 2025

The growing uncertainty from renewable power and electricity demand brings significant challenges to unit commitment (UC). While various advanced forecasting and optimization methods have been developed to predict better and address this uncertainty, most previous studies treat forecasting and optimization as separate tasks. This separation can lead to suboptimal results due to misalignment between the objectives of the two tasks. To overcome this challenge, we propose a robust UC framework that integrates the forecasting and optimization processes while ensuring statistical guarantees. In the forecasting stage, we combine multiple predictions derived from diverse data sources and methodologies for an improved prediction, aiming to optimize the UC performance. In the optimization stage, the combined prediction is used to construct an uncertainty set with statistical guarantees, based on which the robust UC model is formulated. The optimal robust UC solution provides feedback to refine the forecasting process, forming a closed loop. To solve the proposed integrated forecasting-optimization framework efficiently and effectively, we develop a neural network-based surrogate model for acceleration and introduce a reshaping method for the uncertainty set based on the optimization result to reduce conservativeness. Case studies on modified IEEE 30-bus and 118-bus systems demonstrate the advantages of the proposed approach.

Load forecasting model trading: A cost-oriented and auction-based approach

2025ArticlePreprint
D. Qin, P. Pinson, Y. Wang
preprint, under review
Publication year: 2025

Energy management costs can be reduced by increasing load forecasting accuracy. Many studies use multiple data sources to potentially yield improved forecasts. However, data owners may be hesitant to share their data due to privacy concerns and a lack of monetary incentives. Here, we establish a model trading market to encourage collaboration, in which a buyer can purchase advanced load forecasting models that have been trained by sellers’ data to facilitate his decision-making. Specifically, we first propose a cost-oriented loss function that links to the buyer’s decision-making problem, which serves as a basis for market participants to evaluate the model quality and design their utility functions. Then, an iterative model auction mechanism is proposed to orchestrate selfish market participants to reach a consensus on the social welfare-maximizing solution, where the model quality and price for trading are determined through a distributed process. Furthermore, we propose a model adaptation strategy, including model fine-tuning and ensembling, for the buyer to enhance the applicability of purchased models to his decision-making problem. Experiments for building energy management are conducted based on public datasets. Results show that our approach converges to the socially optimal point and every participant can benefit from the market: sellers are compensated for providing models; and, the buyer can greatly reduce operational costs by employing the purchased models.

How long is long enough? Finite-horizon approximation of energy storage scheduling problems

2025ArticleJournal paperPreprint
E. Prat, R. M. Lusby, J. M. Morales, S. Pineda, P. Pinson
preprint, under review
Publication year: 2025

Energy storage scheduling problems, where a storage is operated to maximize its profit in response to a price signal, are essentially infinite-horizon optimization problems as storage systems operate continuously, without a foreseen end to their operation. Such problems can be solved to optimality with a rolling-horizon approach, provided that the planning horizon over which the problem is solved is long enough. Such a horizon is termed a forecast horizon. However, the length of the planning horizon is usually chosen arbitrarily for such applications. We introduce an easy-to-check condition that confirms whether a planning horizon is a forecast horizon, and which can be used to derive a bound on suboptimality when it is not the case. By way of an example, we demonstrate that the existence of forecast horizons is not guaranteed for this problem. We also derive a lower bound on the length of the minimum forecast horizon. We show how the condition introduced can be used as part of an algorithm to determine the minimum forecast horizon of the problem, which ensures the determination of optimal solutions at the lowest computational and forecasting costs. Finally, we provide insights into the implications of different planning horizons for a range of storage system characteristics.

Do we actually understand the impact of renewables on electricity prices? A causal inference approach

2025ArticleJournal paperPreprint
D. Cacciarelli, P. Pinson, F. Panagiotopoulos, D. Dixon, L. Blaxland
preprint, under review
Publication year: 2025

The energy transition is profoundly reshaping electricity market dynamics. It makes it essential to understand how renewable energy generation actually impact electricity prices, among all other market drivers. These insights are critical to design policies and market interventions that ensure affordable, reliable, and sustainable energy systems. However, identifying causal effects from observational data is a major challenge, requiring innovative causal inference approaches that go beyond conventional regression analysis only. We build upon the state of the art by developing and applying a local partially linear double machine learning approach. Its application yields the first robust causal evidence on the distinct and non-linear effects of wind and solar power generation on UK wholesale electricity prices, revealing key insights that have eluded previous analyses. We find that, over 2018-2024, wind power generation has a U-shaped effect on prices: at low penetration levels, a 1 GWh increase in energy generation reduces prices by up to 7 GBP/MWh, but this effect gets close to none at mid-penetration levels (20–30%) before intensifying again. Solar power places substantial downward pressure on prices at very low penetration levels (up to 9 GBP/MWh per 1 GWh increase in energy generation), though its impact weakens quite rapidly. We also uncover a critical trend where the price-reducing effects of both wind and solar power have become more pronounced over time (from 2018 to 2024), highlighting their growing influence on electricity markets amid rising penetration. Our study provides both novel analysis approaches and actionable insights to guide policy makers in appraising the way renewables impact electricity markets.

Load forecasting model trading: A cost-oriented and auction-based approach

2024ArticlePreprint
D. Qin, P.Pinson, Y. Wang
preprint, under review
Publication year: 2024

Energy management costs can be reduced by increasing load forecasting accuracy. Many studies use multiple data sources to potentially yield improved forecasts. However, data owners may be hesitant to share their data due to privacy concerns and a lack of monetary incentives. Here, we establish a model trading market to encourage collaboration, in which a buyer can purchase advanced load forecasting models that have been trained by sellers’ data to facilitate his decision-making. Specifically, we first propose a cost-oriented loss function that links to the buyer’s decision-making problem, which serves as a basis for market participants to evaluate the model quality and design their utility functions. Then, an iterative model auction mechanism is proposed to orchestrate selfish market participants to reach a consensus on the social welfare-maximizing solution, where the model quality and price for trading are determined through a distributed process. Furthermore, we propose a model adaptation strategy, including model fine-tuning and ensembling, for the buyer to enhance the applicability of purchased models to his decision-making problem. Experiments for building energy management are conducted based on public datasets. Results show that our approach converges to the socially optimal point and every participant can benefit from the market: sellers are compensated for providing models; and, the buyer can greatly reduce operational costs by employing the purchased models.

Fairness by design in shared-energy allocation problems

2024ArticlePreprint
Z Fornier, V Leclėre, P Pinson
preprint, under review
Publication year: 2024

This paper studies how to aggregate prosumers (or large consumers) and their collective decisions in electricity markets, with a focus on fairness. Fairness is essential for prosumers to participate in aggregation schemes. Some prosumers may not be able to access the energy market directly, even though it would be beneficial for them. Therefore, new companies offer to aggregate them and promise to treat them fairly. This leads to a fair resource allocation problem.

We propose to use acceptability constraints to guarantee that each prosumer gains from the aggregation.
Moreover, we aim to distribute the costs and benefits fairly, taking into account the multi-period and uncertain nature of the problem. Rather than using financial mechanisms to adjust for fairness issues, we focus on various objectives and constraints, within decision problems, that achieve fairness by design. We start from a simple single-period and deterministic model, and then generalize it to a dynamic and stochastic setting using, e.g., stochastic dominance constraints.

Privacy-preserving convex optimization: When differential privacy meets stochastic programming

2023ArticleJournal paperPreprint
V. Dvorkin, F. Fioretto, P. Van Hentenryck, P. Pinson, J. Kazempour
preprint, under review
Publication year: 2023

Convex optimization finds many real-life applications, where – optimized on real data – optimization results may expose private data attributes (e.g., individual health records, commercial information, etc.), thus leading to privacy breaches. To avoid these breaches and formally guarantee privacy to optimization data owners, we develop a new privacy-preserving perturbation strategy for convex optimization programs by combining stochastic (chance-constrained) programming and differential privacy. Unlike standard noise-additive strategies, which perturb either optimization data or optimization results, we express the optimization variables as functions of the random perturbation using linear decision rules; we then optimize these rules to accommodate the perturbation within the problem’s feasible region by enforcing chance constraints. This way, the perturbation is feasible and makes different, yet adjacent in the sense of a given distance function, optimization datasets statistically similar in randomized optimization results, thereby enabling probabilistic differential privacy guarantees. The chance-constrained optimization additionally internalizes the conditional value-at-risk measure to model the tolerance towards the worst-case realizations of the optimality loss with respect to the non-private solution. We demonstrate the privacy properties of our perturbation strategy analytically and through optimization and machine learning applications.

On the design of decentralised data markets

2023ArticleJournal paperPreprint
A. Manzano Kharman, C. Jursitzky, Q. Zhou, P. Ferraro, J. Marecek, P. Pinson, R. Shorten
preprint, under review
Publication year: 2023