Many customers who are interested in a content management system make the mistake of only considering price and features during the selection process.
Whether one will be satisfied with the purchase in the long run is not defined by the sales person who once presented the system but by the factors that can hardly be perceived in a presentation. Among those are:
- quality of the software
- innovation and continually ongoing development
- training, service and hotline
In two blogs on the topics ‘Quality Assurance’ and ‘Hotline’ we try to focus on these factors. SCHEMA would like to introduce those two departments at SCHEMA to you and for a start has put together some “Q & A” based on an interview with Ms Kelley, head of the QA department. The TechSupport department will follow soon… The quality assurance department currently (September 2013) has 6 fully employed members. All colleagues are on one or another level as ISTQB Certified Tester.
What does the quality assurance department at SCHEMA do and what is tested?
The tasks of QA are mainly comprised of the following core themes:
- Testing the standard software ST4 and ST4 DocuManager, ST4 BasicLine, ST4 Labeling Manager, ST4 ValiManager
- Testing the external software supported by ST4, like e.g. operating systems, databases, editors, translation memory systems, production software like e.g. Antenna House, InDesign, MS Word, Adobe Frame Maker or also the various HTML browsers. An at all times complete listing of the supported third party software can also be found in the “Third-Party Software Document’ which customers can download any time from the customer area on our website.
- As the development process within SCHEMA is carried out according to an extended V-model, for new developments internal functional specifications are created and released. In this phase QA is already involved in order to review these.
What actually are the specific challenges?
One needs to realize that more than 200 man years of development effort have been invested in SCHEMA ST4. We have a major release about once a year and sometimes service packs within the year. This accounts for a number of functions. We are dealing with a highly integrated system and many functions interact with one another resp. depend on each other. First of all one must not lose track.
Furthermore SCHEMA ST4 is software that can also be put into proper operation by third parties (i.e. customers or partners) by creating metadata, filters, productions etc. by themselves without our direct help. The product can be extensively adjusted via the user interface and be adapted to customer demands. This configurability must be tested as well.
SCHEMA ST4 is used in many different industrial sectors and fields of application. From this many different use cases and various priorities of application result. One more thing that needs to be mirrored in our test cases.
And finally: We support a high number of third party software in multiple languages and operating system versions as well. The combinatorics resulting from this is demanding. The user interface of SCHEMA ST4 in the standard version is available in German, English, French and Japanese.
And how is it tested?
This is certainly not the right platform for a complete overview of test theory but we at SCHEMA professionally and systematically cover all ways of testing that are relevant for us. Among these are functional testing, interface testing, integration testing, user interface testing, system tests, installation tests, load testing, performance testing, unit tests.
The whole procedure is a combination of manual and automatic tests. The automated tests are partly carried out via scripts or via professional test automation software SilkTest by Borland. This is complemented by daily, automated Compile Builds and fully automated, standardized tests for database connection, for initialization, starting of the ST4 Services clients. All test environments exist on VMware images. Herewith many different environments (operating systems, databases, editors, TM systems) can be provided and the cooperation of ST4 with them be tested.
Here individual modules as well as the integration to a total system in the various product variants are automatically checked for functionality. The installer programs are automatically installed on various operating systems and tested for the features laid down in the specification.
All functional specification documents, test plans and test protocols are conclusively logged and saved in our internal SCHEMA ST4-based QA database. This is by the way also a precondition for us to carry out and pass audits held by customers.