TestPlan

From Riley Consulting Wiki
Jump to: navigation, search

Test Plan for Storm Project

The RedX does not enforce any kind of testing requirements other than a fairly cursory examination of any code to be pushed live by QA. The Storm Project Manager (Brandon) and I came up with the following plan to test the product.

Phase 1
The first phase of testing comes from the developers themselves. Each developer is responsible for writing unit tests for any piece of code that is to be committed to the Repository. Both the code and the tests are not allowed to be committed until another developer has reviewed both the code and the tests. Ideally, the tests and the code are each written by a different programmer, but a detailed design document was not created at the beginning of the project and so it is difficult to create tests when the specs are evolving as the product is being created. The requirements are fixed, the specs are not.
It is believed that the majority of bugs that can arise and the area of the product that needs to be the most stable is the C++ module written to interface with Freeswitch (the phone-switch server that actually places the calls and connects them). That is the heart of the product and must be bullet-proof. The design for that module is not finalized, but following is the best stab I can give at writing the things that need to be tested:

Class: freeswitch_telecom
Requirements: Must emit the channel_answered, channel_bridged and channel_hungup events.
Log methods must log the user_id, channel_id, log_level, file_name, line_number, timestamp and log_message to any configured database.
Class: dialable
Requirements: If dial() is called, the dialable moves through its states to "Parked" where it emits the "Answered" event. If the dialable is a child dialable, it tries to bridge to its parent.
If bridge() is called with no parameters, the dialable pulls the first dialable from its priority queue and dials is, as long as the "can_reload" flag is true.
If bridge() is called with a dialer as the parameter, the callee dialable accepts that dialable, calls dial() on it and then transitions to the "Bridging" state, emitting the "Bridging" event and waiting for the dialed dialable to attempt to bridge.
Every state change must emit an event, the events being "Dialing", "Answered", "Bridging", "Bridged", "Hanging-up", "Hungup" and "Error"
If hangup() is called at any point, the dialable must transition from whatever state it is in to "Hanging-up", then to "Hungup".
If dial() is called twice on a dialable, there is no effect unless the dialable has reached and end state (Hungup and Error), in which case an Exception is thrown.
If bridge() is called twice in a row, the second bridge command is discarded.
If the "can_reset" flag is false and the "can_reload" flag is true, the dialable dials each dialable in its priority queue until one of them bridges or the end of the queue is reached. At that point the dialable moves to the "Hungup" state and is destroyed.
If the "can_reset" flag is true, a dialable returns to the parked state after bridging or attempting to bridge and waits for the bridge() command to either pop the next dialable off the priority queue or dial the passed in dialable.
Every event emitted from a dialable must contain the user_id of that dialable and the phone(s) that are active in that dialable.

Phase 2
Once the code is unit tested, peer-reviewed and committed to the dev master repository, the code is pushed to a staging server that QA has access to. They will then test the code according to the requirements doc in a black-box fashion. Any bugs are sent back to the dev team, whose first responsibility is to fix the bugs and re-submit to QA. Once QA has approved the code, it is pushed to the production server.
Phase 3
In the case of Storm, once the code is on the production server, it will go through a beta test phase with less than 200 but more than 100 real customers. They will have access to our bug-reporting software to log bugs that they find. Those user bugs will be vetted by QA before they come to the dev team to make sure that the bugs are legitimate and to hopefully reduce the number of duplicate bugs reported.