|
|
|
IS&T Release Engineering assists project managers, business analysts and software engineers in ensuring that deployed applications, APIs, Software-as-a-Service (SaaS) and Platform-as-a-Service (PaaS) offerings are production-ready. Our testing engineers build automated API tests, performance tests and regression tests, offer consultation regarding testing strategies and provide assistance in the use of our testing tools: Blazemeter for performance testing and Veracode for security analysis. Developers provide their own unit tests for code as part of the development process.
This checklist is intended as a tool for project managers and business analysts as they move through the phases of software development projects and SaaS & PaaS implementations.
Questions? Email release-engineering@mit.edu.
| Project Milestone | Testing Activities | |
|---|---|---|
| Project approved for discovery | ▢ Testing resources included as part of discovery project discussion. Business Analyst works with product owner to document system requirements. | |
| Project team documents system requirements | ▢ Requirements are captured as acceptance criteria for use as functional test cases. | |
| Project Manager creates project plan | ▢
Project Manager meets with Release Engineering team manager to consider testing needs for the project. ▢ Decision regarding manual functional testing vs. automated functional testing. |
|
| Development begins | ▢
For SaaS & PaaS applications, development phase will include configuration of the vendor product and may include API development work.
▢ Developer creates function tests for APIs. ▢ Developer checks in with Release Engineering regarding function tests for APIs, for awareness. Consultation available if needed. ▢ Developer works with Release Engineering to validate API performance in isolation from the application. ▢ Explore whether Veracode security scanning will be available for the application. If so, Release Engineering creates an application entry on http://veracode.mit.edu. ▢ For Java applications, developer integrates Veracode into Integrated Development Environment (IDE). ▢ Configure Travis to execute API tests automatically for each application build. Consult with Release Engineering as needed. |
|
| Determination of server resource requirements | ▢
If this is a net new application, determine production resourcing requirements. ▢ Confirm that resources in test are on par with those in production, in order to enable accurate performance testing. |
|
| One feature or component of the application is complete | ▢
Release Engineering validates API unit tests. ▢ Release Engineering validates integration of application with APIs and other backend services. ▢ If automated function tests will be created for this application, coding begins when one feature or component of the application is complete. |
|
| Application available in Test environment | Performance Testing: | |
| ▢
BA or product owner determines test cases. ▢ BA or product owner works with Release Engineering to determine expected response times. ▢ Load testing: Validate that average response times are consistent with product owner expectations and IS&T guidelines. ▢ Stress testing: Increase load incrementally to determine the point at which the application fails to respond. |
||
| Systems Integration Testing (SIT): | ||
| ▢ Developers and BAs work with Release Engineering to validate integration of application unit tests, API function tests and other backend services. | ||
| Automated Functional Testing, if applicable: | ||
| ▢
Validate that UI behaves as expected, based on a set of specified test cases. ▢ BA validates automated regression tests and documents test cases that are not included in automated regression tests and will need manual testing. |
||
| Security Testing: | ||
| ▢ Project Manager reviews Veracode security testing results in conjunction with developer(s) and Manager of Design and Development. | ||
| Go-Live and Ongoing Maintenance | Automated Functional Testing: | |
| ▢ Automated regression tests are available to be run for all application builds, to ensure that code changes do not break existing application functionality. | ||
| API Testing: | ||
| ▢ API tests are run at build time as part of Travis Continuous Integration. | ||
| Performance Testing: | ||
| ▢ Performance tests are available to be re-run as needed. | ||
| Security Testing: | ||
| ▢ In some cases, it may be advisable to re-run security tests on a yearly basis. | ||
Download Checklist