Why and How to Start Manual Accessibility Testing
November 12, 2020
How many times have you seen a tweet like this?
Proud to say my website is fully accessible! #a11y
I don’t want to diminish the effort it takes to reach a perfect accessibility score on Lighthouse. That is a great first step and feel free to pat yourself on the back about it!
However, the important thing to know is that the work does not stop there. A 100 does not mean your site is fully accessible.
Automated testing can only catch about 40-50% of accessibility issues.
In This Article
- Automated testing can only test for programmatic issues, without usability considerations.
- What Can you Do?
Automated testing can only test for programmatic issues, without usability considerations.
Automated tests only work for those things that can be tested programmatically and objectively, such as the existence of alt tags, whether color contrast ratios are high enough, and the existence of form labels.
However, even these things are subjective.
If alt tags are present for all images, the alt tag test will pass, but that doesn’t mean the alt text is actually appropriate for those images.
It is extremely difficult to automatically test color contrast for text with a gradient or photograph background.
The form labels that are present might not be appropriate for the fields they are labeling.
These are just a few examples of the many things that still require a human touch to verify accessibility.
What Can You Do?
Fortunately, there are manual web accessibility testing tools readily available and literally at your fingertips!
If you’re just getting started with manual accessibility testing, I recommend focusing on:
- Keyboard testing.
- Screen reader testing.
Keyboard Testing for Accessibility
One key element of web accessibility is that it should be possible to perform every action using only the keyboard.
Keyboard accessibility affects two types of users:
- People who cannot physically use a mouse, or who have difficulty using a mouse. This can be due to a permanent, temporary, or situational disability (e.g. a person who experiences hand tremors, or someone holding a baby in their right arm while navigating using a keyboard with their left).
- Blind users who can physically use a mouse, but don’t because they can’t see where the cursor is located.
Remember Your Sighted Keyboard Users
A common mistake—and one I have made myself—is to forget that there are many sighted (i.e. not blind) people who navigate the web using only their keyboard.
It must be possible for both sighted AND non-sighted users to perform all functions using the keyboard.
While a screen reader will announce the element that has focus to a blind user, a sighted user needs to be able to see where their focus is.
The main thing to keep in mind here is that visible focus styles are key. None of that outline: none; crap (without adding other visible styles)!
Click here to read a great blog post from Deque on how to design useful and usable focus indicators.
How to Get Started with Keyboard Testing
Open up your webpage and try to navigate with your keyboard!
The basics of keyboard navigation are:
- Use your TAB key to advance focus from item to item on the page.
- Use SHIFT + TAB to go backwards on the page.
- Use ENTER to activate interactive elements such as links and buttons.
- Use arrow keys to move between radio buttons or checkboxes within a group.
- Use SPACE to select or deselect a checkbox.
Expected Behavior
You should be able to navigate to and interact with all focusable elements on the page.
Focusable elements are those that are interactive by default, such as links, buttons, and form fields, as well as anything to which you have added tabindex="0" to add it to the logical focus order.
It is NOT normal to be able to focus on non-interactive elements, so don’t worry that your focus skips things like paragraphs, lists, images, etc. Don’t manually add anything to the focus order unless absolutely necessary.
Your focus should go to all focusable elements on the page in a logical focus order.
For more on keyboard testing and expected behavior, check out this article on keyboard accessibility from WebAIM.
Screen Reader Testing for Accessibility
Screen readers are an assistive technology that allow blind users to interact with computers by announcing the elements on the page to the user and allowing the user to navigate through the page with their keyboard.
Why You Should Test with a Screen Reader
It’s important to test with a screen reader to truly get a feel for whether your site is accessible to blind users. You should test for two main things:
- Are you able to access and interact with all elements of your site using only a keyboard?
- Can you understand the interface from the audio alone?
Keyboard operability can vary when navigating with a screen reader activated. For example, I have seen cases where buttons could be activated with the enter key when tabbing through a site without a screen reader turned on, but that could not be activated when using a screen reader.
Since not everyone navigating your site with a keyboard will be using a screen reader, it is still also important to test this without a screen reader, as described above.
In addition, UI elements that make sense visually might need some tweaking to make them understandable to screen reader users. Some common examples of issues that can pop up include:
- error messages that aren’t programmatically associated with their fields (but appear next to those fields visually),
- radio button groups with some hidden buttons that announce the full number of buttons to screen reader users, including the hidden ones,
- and help text that is not programmatically associated with what it is trying to help with.
Which Screen Readers to Use and How to Use Them
The three most popular screen readers on the market today are NVDA, JAWS, and VoiceOver (see the WebAIM Screen Reader User Survey conducted in 2019).
VoiceOver comes with all Apple devices by default, so you’ll probably be testing with VoiceOver if you have a Mac laptop or desktop.
For Windows, the most popular choices are NVDA and JAWS. JAWS is paid software, but NVDA is free and open source, so it is a good choice to use for testing.
Click here for guides to getting started with each of these screen readers.
Why You Should Also Test on Mobile (If Possible)
In addition to testing on a laptop or desktop computer, I highly recommend testing your website on a mobile device.
The screen readers available for mobile devices vary slightly from their desktop counterparts. I have run into some challenging cases where functionality works with desktop screen readers, but not using a screen reader on a mobile device.
It is important to test on a real device to find these issues, instead of just testing on a narrow browser window or in a device emulator on your desktop.
For mobile testing, the most popular combinations are iOS with VoiceOver and Android with TalkBack.
Click here for a guide to using VoiceOver on iOS.
Click here for a guide to using TalkBack on Android.
Conclusion
Manual testing is key for ensuring your website is accessible. Automated testing is a great way to get started, but it can only test for about half the potential issues.
Adding keyboard testing and screen reader testing to your workflow can have a huge impact on the accessibility of your websites and web apps. Testing with a screen reader can be frustrating at first, but the more you practice, the easier it will become.
Check out these resources (also linked above) for getting started with keyboard and screen reader testing:
- WebAIM on keyboard accessibility.
- Guides to using a variety of screen readers, written for people using them for manual testing.
- Getting started with mobile screen reader testing using VoiceOver on iOS.
- Getting started with mobile screen reader testing using TalkBack on Android.
Will you be incorporating manual accessibility testing into your workflow? Let me know on Twitter!