We propose an adversarial, time-varying test-synthesis procedure for safety-critical systems without requiring specific knowledge of the underlying controller steering the system. From a broader test and evaluation context, determination of difficult tests of system behavior is important as these tests would elucidate problematic system phenomena before these mistakes can engender problematic outcomes, e.g. loss of human life in autonomous cars, costly failures for airplane systems, etc. Our approach builds on existing, simulation-based work in the test and evaluation literature by offering a controller-agnostic test-synthesis procedure that provides a series of benchmark tests with which to determine controller reliability. To achieve this, our approach codifies the system objective as a timed reach-avoid specification. Then, by coupling control barrier functions with this class of specifications, we construct an instantaneous difficulty metric whose minimizer corresponds to the most difficult test at that system state. We use this instantaneous difficulty metric in a game-theoretic fashion, to produce an adversarial, time-varying test-synthesis procedure that does not require specific knowledge of the system's controller, but can still provably identify realizable and maximally difficult tests of system behavior. Finally, we develop this test-synthesis procedure for both continuous and discrete-time systems and showcase our test-synthesis procedure on simulated and hardware examples.

我们提出了一种对抗性的、时变的测试综合过程。 无需特定基础知识即可实现安全关键型系统 控制器控制系统。从更广泛的测试和评估背景来看， 确定系统行为的困难测试非常重要，因为这些测试 将在这些错误可能产生之前阐明有问题的系统现象 有问题的结果，例如自动驾驶汽车造成的生命损失，代价高昂 飞机系统的故障等。我们的方法建立在现有的、 测试和评估文献中基于模拟的工作，通过提供 与控制器无关的测试-综合过程，提供了一系列 用于确定控制器可靠性的基准测试。要实现 因此，我们的方法将系统目标编码为定时到达-避免 规格。然后，通过将控制屏障函数与这类 规范，我们构造了一个瞬时难度度量，它的最小化 对应于该系统状态下最难的测试。我们用这个 以博弈论的方式进行瞬时难度度量，以产生 对抗性的、时变的测试综合程序，不需要 系统控制器的特定知识，但仍可被证明识别 对系统行为进行可实现且难度最大的测试。最后，我们 开发用于连续时间和离散时间的测试综合程序 系统，并展示我们在模拟和硬件上的测试综合过程 举个例子。