We test our provengo tool using our provengo tool. That shouldn’t surprise anyone, since we care about quality and provengo is a very good tool for test suite design and execution (among other things). But what techniques do we use? And how?
We test provengo using a layered model – a provengo model that uses other provengo models. This structure allows us to easily exercise all the features we want to test, without making the test model grow too much. At the top level, there’s a state machine model. At each state, the model uses our command line (CLI) library to invoke provengo on others models (mostly from our TechDemos repository).
The top-level state machine, without the automation layer, is shown below:
Recall that each test scenario starts at the “start” node on the left, and then continues along the arrows until it gets to a location from which there are no outgoing arrows. So it’s easy to see that the model drawn above contains many scenarios that are typical for a provengo user, such as create ⇒ sample ⇒ ensemble ⇒ gen-script command sequence.
Here’s a screen capture (+ some titles) showing a test. The clip was edited for speed.
Just like our clients, we find that using provengo allows us to easily create and maintain test suites that cover system functionalities properly. We know which functionalities are covered and which are not, and so, when all tests pass, we know that the new version is ready for release.
