Usage-and Risk-Aware Falsification Testing for Cyber-Physical Systems

Falsification testing is a popular method for efficiently identifying inputs that witness the violation of cyber-physical system (CPS) specifications. The generated counterexamples are used to locate and explain faults and debug the system. However, CPS rarely operate in unconstrained environments and not all counterexamples have the same value. On one hand, faults resulting from common system usage are more likely to happen in practice than faults triggered by esoteric inputs. On the other hand, some faults can have more severe consequences than others. Together, the probability and the severity of a fault determine its risk, an important aspect that is neglected by the existing falsification testing techniques. We propose a new falsification testing methodology that is aware of the system’s expected usage and the severity associated to different faulty behaviors. Given a user profile in the form of a stochastic hybrid automaton, an associated severity degree measure, an executable black-box implementation of the CPS and its formalized requirements, we provide a test generation method that (1) uses efficient randomized methods to generate multiple violating traces, and (2) estimates the probability and the expected severity, and hence the expected risk of each counterexample, thus providing their ranking to the engineer.