Cascading Failure Sequence Prediction Using Generative Pre-trained Transformers in Power Networks

Predicting the behavior of large power networks in rare cascading failures is important for resiliency evaluation, as these events pose a high risk. The state space of the initial and potential subsequent events is, however, enormous, and the process has non-local and non-monotonicity properties, making the problem computationally hard. The failure cascade unfolds according to local and global physical constraints, leading to pairwise and nontrivial higher order failure interactions across cascade generations. We propose using generative pre-trained (GPT) models to learn the underlying constraints as a network grammar to predict subsequent line failures and complete the failure sentence after an initial incident. The model is a transformer-based machine learning tool with a designed architecture for line embedding, transformer, and projection layers,
in which antecedent failures serve as tokens to predict consequent failures. In the pre-training stage, the model learns the network’s physical topology from data generated by random walkers. Next, the model is trained on diverse failure-cascade sequence data, with initial failures induced by the most frequently observed initial failure motifs. The model successfully predicts failure unfolding on test data, as measured by the area under the precision-recall curve. Furthermore, by dissecting the attention weights, we interpret them and show how the model captures the network’s influential motifs.