The assessment process covers three phases – Groundwork, Testing and Reporting. The Groundwork phase defines the test set for the site or application. The Testing phase performs the actual testing on the site or application and codifies all the testing results. The reporting phase analyzes all the assessment results, creates the report and delivers the final report results.
Groundwork Phase
For any given system a test set must be defined to serve as the testing baseline for the system. The test set includes two components – the module list and the use case list. The module list defines the set of individual modules within a system that will be tested using SSB BART Group’s automated and manual testing methodology. For web based systems, the module list is defined by selecting representative pages from the system. The use case list defines the assumed use cases for the document set.
Module Selection – For each system to be tested, a set of fifteen modules are selected for testing. The modules are selected to provide a sampling of the various different elements within the system. These modules are selected using targeted sampling, rather than random sampling. Targeted sampling selects modules for testing based on the type of modules likely to create problems with assistive technology. For example, within a web based systems modules that provide data tables, charts and graphs, and other complex user interface elements are selected for testing.
Use Case Selection – Use case development is the process of developing a list of use cases for a system. The use cases define the core functionality that users are expected to interact with in a system. For example, consider a user who is attempting to download a specific form from a web site. The use cases associated with this user are: searching within the site to locate the form, downloading the form, and completing the form.
Testing Phase
During the testing phase the actual tests are performed on the system. The testing process includes automated testing, manual testing and end-user assistive technology testing. The different types of tests are designed to provide redundant sets of results so that all issues within the system are captured.
Automated Analysis – As part of the testing process the various different modules are tested using automated testing tools like InFocus™. Depending on the system, different automated testing tools are used. In some systems, such as hardware based systems, no automated testing tools can be used and this phase is skipped.
End-user Analysis – Individual with disabilities using leading Assistive Technologies (AT) test the end-to-end user experience of the system. This testing focuses on the overall usability of the system with various types of assistive technologies. As part of the end-user analysis, users ensure that the use cases defined can be successfully and easily executed.
Manual Review Analysis – A set of manual tests is executed against each module within the system. The manual testing results provide the bulk of the accessibility testing results.
Reporting Phase
During the reporting phase the final report is assembled and the data set is analyzed. The reporting phase includes analysis, prioritization, draft report creation and final delivery.
Prioritization Analysis – During the analysis phase all the issues detected in testing are identified and mapped into the final report framework.
Prioritization – All accessibility violations are analyzed and ranked based on their impact on users with disabilities (Severity), frequency of occurrence (Frequency), how difficult they are to correct (Tractability), and how likely the violations are to be noticed (Noticeability). These metrics enable an organization to prioritize correction activities to ensure maximum return for any efforts invested in addressing the issues found in a report.
Draft Report Creation – Authoring and editing of the content of the written report. Upon delivery of the written report a formal review and comment period begins. Typically the review and comment period lasts for one week during which developers can review the recommended implementation and ensure the results can effectively be implemented. Often, there are many routes to achieve accessibility within a given system and the review phase allows developers to make certain that the most efficient route is determined.
Final Delivery – Web based delivery of the final report and findings overview. During this phase a report is presented online to the development team and the core issues within the report are reviewed.
Assessment Overview
