Filter by type:

1 2 3 8

Wind energy forecasting with missing values within a fully conditional specification framework

2022ArticlePreprint
H. Wen, P. Pinson, J. Gu, Z. Jin
preprint, under review
Publication year: 2022

Wind power forecasting is essential to power system operation and electricity markets. As abundant data became available thanks to the deployment of measurement infrastructures and the democratization of meteorological modeling, extensive data-driven approaches have been developed within both point and probabilistic forecasting frameworks. These models usually assume that the dataset at hand is complete and overlook missing value issues that often occur in practice. In contrast to that common approach, we rigorously consider here the wind power forecasting problem in the presence of missing values, by jointly accommodating imputation and forecasting tasks. Our approach allows inferring the joint distribution of input features and target variables at the model estimation stage based on incomplete observations only. We place emphasis on a fully conditional specification method owing to its desirable properties, e.g., being assumption-free when it comes to these joint distributions. Then, at the operational forecasting stage, with available features at hand, one can issue forecasts by implicitly imputing all missing entries. The approach is applicable to both point and probabilistic forecasting, while yielding competitive forecast quality within both simulation and real-world case studies. It confirms that by using a powerful universal imputation method based on fully conditional specification, the proposed universal imputation approach is superior to the common impute-then-predict approach, especially in the context of probabilistic forecasting.

Variable heat pricing to steer the flexibility of heat demand response in district heating systems

2022ArticleJournal paper
L. Bai, J. Wang, P. Pinson
Electric Power Systems Research 212, art. no. 108383
Publication year: 2022

Trading data for wind power forecasting: A regression market with Lasso regularization

2022ArticleJournal paper
L. Han, P. Pinson, J. Kazempour
Electric Power Systems Research 212, art. no. 108442
Publication year: 2022

To share or not to share? Alternative views on a future of collaborative forecasting

2022ArticleJournal paper
P. Pinson
Foresight 67(7), pp. 8-15
Publication year: 2022

Distributed data refers to information that flows from different sources and possibly different owners. Getting top value from distributed data requires a paradigm shift towards collaborative forecasting. Alternative frameworks exist to support collaborative forecasting, from collaborative analytics to data markets, and from analytics markets to prediction markets. While we should accept that not all data will be openly shared, rethinking forecasting processes with modern communication, distributed computation, and a market component could yield substantial improvements in forecast quality while unleashing new business models

Stochastic control and pricing for natural gas networks

2022ArticleJournal paper
V. Dvorkin, A. Ratha, P. Pinson, J. Kazempour
IEEE Transactions on Control of Network Systems 9(1), pp. 450-462
Publication year: 2022

Statistical downscaling of local wind speed based on objective definition of the set of regressor variables

2022ArticleJournal paperPreprint
G. Dantas, A. Costa, O. Vilela, P. Pinson
preprint, under review
Publication year: 2022

Robust scheduling with purchase of distributed predictive information

2022ArticleJournal paperPreprint
R. Xie, P. Pinson, Y. Chen
preprint, under review
Publication year: 2022

Robust scheduling is an essential way to cope with uncertainty. However, the rising unpredictability in net demand of distributed prosumers and the lack of relevant data make it difficult for the operator to forecast the uncertainty well. This leads to inaccurate, or even infeasible, robust scheduling strategies. In this paper, a novel two-stage robust scheduling model is developed, which enables the operator to purchase predictive information from distributed prosumers to enhance scheduling efficiency. An improved uncertainty set with a smaller variation range is developed by combining the forecasts from the operator and prosumers. Since the improved uncertainty set is influenced by the first-stage information purchase related decisions, the proposed model eventually becomes a case of robust optimization with decision-dependent uncertainty (DDU). An adaptive column-and-constraint generation (C&CG) algorithm is developed to solve the problem within a finite number of iterations. The potential failures of traditional algorithms in detecting feasibility, guaranteeing convergence, and reaching optimal strategies under DDU are successfully circumvented by the proposed algorithm. Case studies demonstrate the effectiveness, necessity, and scalability of the proposed method.

Reliability-aware probabilistic reserve procurement

2022ArticleJournal paper
L Herre, P. Pinson, S. Chatzivasileiadis
Electric Power Systems Research 212, art. no. 108345
Publication year: 2022

Regression markets and application to energy forecasting

2022ArticleJournal paper
P. Pinson, L. Han, J. Kazempour
TOP 30, pp. 533–573
Publication year: 2022

Energy forecasting has attracted enormous attention over the last few decades, with novel proposals related to the use of heterogeneous data sources, probabilistic forecasting, online learning, etc. A key aspect that emerged is that learning and forecasting may highly benefit from distributed data, though not only in the geographical sense. That is, various agents collect and own data that may be useful to others. In contrast to recent proposals that look into distributed and privacy-preserving learning (incentive-free), we explore here a framework called regression markets. There, agents aiming to improve their forecasts post a regression task, for which other agents may contribute by sharing their data for their features and get monetarily rewarded for it. The market design is for regression models that are linear in their parameters, and possibly separable, with estimation performed based on either batch or online learning. Both in-sample and out-of-sample aspects are considered, with markets for fitting models in-sample, and then for improving genuine forecasts out-of-sample. Such regression markets rely on recent concepts within interpretability of machine learning approaches and cooperative game theory, with Shapley additive explanations. Besides introducing the market design and proving its desirable properties, application results are shown based on simulation studies (to highlight the salient features of the proposal) and with real-world case studies.

Privacy-preserving convex optimization: When differential privacy meets stochastic programming

2022ArticleJournal paperPreprint
V. Dvorkin, F. Fioretto, P. Van Hentenryck, P. Pinson, J. Kazempour
preprint, under review
Publication year: 2022

Convex optimization finds many real-life applications, where – optimized on real data – optimization results may expose private data attributes (e.g., individual health records, commercial information, etc.), thus leading to privacy breaches. To avoid these breaches and formally guarantee privacy to optimization data owners, we develop a new privacy-preserving perturbation strategy for convex optimization programs by combining stochastic (chance-constrained) programming and differential privacy. Unlike standard noise-additive strategies, which perturb either optimization data or optimization results, we express the optimization variables as functions of the random perturbation using linear decision rules; we then optimize these rules to accommodate the perturbation within the problem’s feasible region by enforcing chance constraints. This way, the perturbation is feasible and makes different, yet adjacent in the sense of a given distance function, optimization datasets statistically similar in randomized optimization results, thereby enabling probabilistic differential privacy guarantees. The chance-constrained optimization additionally internalizes the conditional value-at-risk measure to model the tolerance towards the worst-case realizations of the optimality loss with respect to the non-private solution. We demonstrate the privacy properties of our perturbation strategy analytically and through optimization and machine learning applications.

Participation and data valuation in IoT data markets through distributed coalitions

2022ArticlePreprint
S. R. Pandey, P. Pinson, P. Popovski
preprint, under review
Publication year: 2022

Pandemics and forecasting: The way forward through the Taleb-Ioannidis debate

2022ArticleJournal paper
P. Pinson, S. Makridakis
International Journal of Forecasting 38(2), pp. 410-412
Publication year: 2022

Online decision-making for trading wind energy

2022Journal paperPreprint
M. A. Muñoz, P. Pinson, J. Kazempour
preprint, under review
Publication year: 2022

On the efficiency of energy markets with non-merchant storage

2022ArticleJournal paperPreprint
L. Frölke, E. Prat, P. Pinson, R. M. Lusby, J. Kazempour
preprint, under review
Publication year: 2022

Energy market designs with non-merchant storage have been proposed in recent years, with the aim of achieving optimal integration of storage. In order to handle the time linking constraints that are introduced in such markets, existing works commonly make simplifying assumptions about the end-of-horizon storage level. This work analyses market properties under such assumptions, as well as in their absence. We find that, although they ensure cost recovery for all market participants, these assumptions generally lead to market inefficiencies. Therefore we consider the design of markets with non-merchant storage without such simplifying assumptions. Using an illustrative example, as well as detailed proofs, we provide conditions under which market prices in subsequent market horizons fail to reflect the value of stored energy. We show that this problem is essential to address in order to preserve market efficiency and cost recovery. Finally, we propose a method for restoring these market properties in a perfect-foresight setting.

On the design of decentralised data markets

2022ArticleJournal paperPreprint
A. Manzano Kharman, C. Jursitzky, Q. Zhou, P. Ferraro, J. Marecek, P. Pinson, R. Shorten
preprint, under review
Publication year: 2022

On machine learning-based techniques for future sustainable and resilient energy systems

2022ArticleIn press/Available onlineJournal paper
J. Wang, P. Pinson, S. Chatzivasileiadis, M. Panteli, G. Strbac, V. Terzija
IEEE Transactions on Sustainable Energy, in press/available online
Publication year: 2022

North Sea energy islands: Impact on national markets and grids

2022ArticleJournal paper
A. Tosatto, X. M. Beseler, J. Østergaard, P. Pinson, S. Chatzivasileiadis
Energy Policy 167, art. no. 112907
Publication year: 2022

Multi-stage linear decision rules for stochastic control of natural gas networks with linepack

2022ArticleJournal paper
V. Dvorkin , A. Botterud , D. Mallapragada , J. Kazempour, P. Pinson
Electric Power Systems Research 212, art. no. 108388
Publication year: 2022

Forecasting: theory and practice

2022ArticleJournal paper
Fotios Petropoulos and co-authors
International Journal of Forecasting 38(3), pp. 705-871
Publication year: 2022

Editorial: Epidemics and forecasting with focus on COVID-19

2022ArticleJournal paper
P. Pinson
International Journal of Forecasting 38(2), pp. 407-409
Publication year: 2022

Distributionally robust trading strategies for renewable energy producers

2022ArticleJournal paperPreprint
P. Pinson
preprint/under review
Publication year: 2022

Renewable energy generation is to be offered through electricity markets, quite some time in advance. This then leads to a problem of decision-making under uncertainty, which may be seen as a newsvendor problem. Contrarily to the conventional case for which underage and overage penalties are known, such penalties in the case of electricity markets are unknown, and difficult to estimate. In addition, one is actually only penalized for either overage or underage, not both. Consequently, we look at a slightly different form of a newsvendor problem, for a price-taker participant offering in electricity markets, which we refer to as Bernoulli newsvendor problem. After showing that its solution is similar to the classical newsvendor problem, we then introduce distributionally robust versions, with ambiguity possibly about both the probabilistic forecasts for power generation and the chance of success of the Bernoulli variable. All these distributionally robust Bernoulli newsvendor problems admit closed-form solutions. We finally use simulation studies, as well as a real-world case-study application, to illustrate the workings and benefits from the approach.

Continuous and distribution-free probabilistic wind power forecasting: A conditional normalizing flow approach

2022ArticleJournal paper
H. Wen, P. Pinson, J. Ma, J. Gu, Z. Jin
IEEE Transactions on Sustainable Energy 13(4), pp. 2250-2263
Publication year: 2022

We present a data-driven approach for probabilistic wind power forecasting based on conditional normalizing flow (CNF). In contrast with the existing, this approach is distribution-free (as for non-parametric and quantile-based approaches) and can directly yield continuous probability densities, hence avoiding quantile crossing. It relies on a base distribution and a set of bijective mappings. Both the shape parameters of the base distribution and the bijective mappings are approximated with neural networks. Spline-based conditional normalizing flow is considered owing to its non-affine characteristics. Over the training phase, the model sequentially maps input examples onto samples of base distribution, given the conditional contexts, where parameters are estimated through maximum likelihood. To issue probabilistic forecasts, one eventually maps samples of the base distribution into samples of a desired distribution. Case studies based on open datasets validate the effectiveness of the proposed model, and allows us to discuss its advantages and caveats with respect to the state of the art.

Chance-constrained economic dispatch of generic energy storage under decision-dependent uncertainty

2022ArticlePreprint
N. Qi, P. Pinson, M. R. Almassalkhi, L. Cheng, Y. Zhuang
preprint/under review
Publication year: 2022

An asynchronous online negotiation mechanism for real-time peer-to-peer electricity markets

2022ArticleJournal paper
Z. Guo, P. Pinson, S. Chen, Q. Yang, Z. Yang
IEEE Transactions on Power Systems 37(3), pp. 1868-1880
Publication year: 2022

A network-aware market mechanism for decentralized district heating systems

2022ArticleJournal paper
L. Frölke, T. Sousa, P. Pinson
Applied Energy 36, art. no. 117956
Publication year: 2022
1 2 3 8