Evergreen DokuWiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Accessibility Testing ======

Testing for accessibility involves a combination of automated testing tools and manual testing procedures using your keyboard. You can add screen readers, switches, or other devices as necessary, but you can find the majority of problems without any extra equipment or software.

First, use a browser extension to catch basic problems with HTML structure, form labels, color contrast, and link and button names. Then try navigating the screen using your keyboard to tab between items. These two steps alone will uncover many issues. You can then go further by setting up a screen reader or using browser bookmarklets to highlight specific problems.

  * [[https://web.dev/how-to-review/|How To Do an Accessibility Review]], Rob Dodson, web.dev
  * [[https://blog.pope.tech/2023/03/01/a-beginners-guide-to-manual-accessibility-testing/|A beginner’s guide to manual accessibility testing]], Whitney Lewis
  * [[https://www.youtube.com/watch?v=LHUmoPJgOEU|Quick Start for Evaluating and Testing Web Accessibility, WebAIM]] - YouTube, 56 minutes
  * [[https://testingaccessibility.com/accessibility-reviews|Accessibility review videos by Marcy Sutton]] ~20 minutes each

You can examine problems as they come up in the browser extensions, or [[accessibility:requirements#understanding-wcag-and-atag|start with a checklist]] to tackle things in a more logical order.

===== Testing with Browser Extensions =====

Browser extensions are great for testing the overall accessibility of a page and finding straightforward errors. However, these tools can’t test criteria that require subjective consideration, and they can sometimes give false positives. 


  * [[https://www.deque.com/axe/devtools/|Axe DevTools]]
    * See the [[https://docs.deque.com/devtools-html/4.0.0/en/devtools-igt|Intelligent Guided Tests features]] for tables, forms, and interactive controls
  * [[https://developer.chrome.com/docs/lighthouse/overview/|Lighthouse]]
    * See [[https://github.com/GoogleChrome/lighthouse/issues/9507|how Lighthouse scores relate to WCAG 2.1]]
    * You can export Lighthouse’s accessibility report to share with colleagues
  * [[https://wave.webaim.org/extension/|WAVE]]
  * [[https://hacks.mozilla.org/2019/10/auditing-for-accessibility-problems-with-firefox-developer-tools/|Firefox Accessibility Inspector]]
  * [[https://accessibleit.disability.illinois.edu/skip-to-chrome-and-firefox-extensions/|SkipTo extension]] or [[https://www.accessibility-developer-guide.com/setup/helper-tools/browser-extensions/headingsmap/|HeadingsMap]]

===== Testing Keyboard Navigation =====

Keyboard navigation is not enabled by default in some browsers.

  * [[https://stackoverflow.com/a/11713537|How to enable keyboard tabbing in Firefox on Macs]]
  * In Chrome, to go Settings > Accessibility. Turn on “Navigate pages with a text cursor” and “Show a quick highlight on the focused object.”
  * In Safari, go to Settings > Advanced and check “Press Tab to highlight each item on a webpage” ([[https://twitter.com/springbroken/status/1552718321899954179|screenshot]])

Place your cursor in the address bar and then press the tab key to move sequentially through focusable items. Use Shift-Tab to go back. Press Enter or Space to open dropdowns or activate buttons. Within dropdown menus and tab sets, you should be able to use the arrow keys to navigate the subsections. Use Escape to close dropdowns, popups, and modals.

Make a note of any controls that can’t be reached using the keyboard, or any areas where you lose track of which item has focus. Items should be highlighted in more or less the same order as a person would read them; if your focus jumps to other areas of the page out of the expected order, make a note of that as well.

==== References: ====

[[https://dev.to/steady5063/keyboard-testing-the-a11y-enhancement-to-your-definition-of-done-2n7c|Keyboard Testing: The A11y Enhancement to Your Definition of Done]]

===== Testing with Screen Readers =====

Getting started with screen readers can be challenging because they lack visible controls. Find a cheat sheet of keyboard shortcuts for the software you’re using, and have it ready before you begin.

[[https://www.sarasoueidan.com/blog/testing-environment-setup/|Setting up a screen reader testing environment on your computer, Sara Soueidan]]

[[https://accessibility.huit.harvard.edu/manual-testing-accessibility#sr|Getting started with screen reader testing, Harvard University Digital Accessibility team]]

Guides to keyboard shortcuts in screen readers:

  * [[https://dequeuniversity.com/screenreaders/survival-guide|Desktop Screen Readers Survival Guide - Basic Keyboard Shortcuts]] Shows a combined chart of common shortcuts in all major screen readers
  * [[https://dequeuniversity.com/screenreaders/jaws-keyboard-shortcuts|JAWS Keyboard Shortcuts]]
  * [[https://dequeuniversity.com/screenreaders/nvda-keyboard-shortcuts|NVDA Keyboard Shortcuts]]
  * [[https://dequeuniversity.com/screenreaders/voiceover-keyboard-shortcuts|VoiceOver Keyboard Shortcuts]]

To speed up testing, you can turn on the screen reader’s feature to display a transcript of the page. This shows exactly what the screen reader will announce. For Macs, the [[https://www.smashingmagazine.com/2021/06/automating-screen-reader-testing-macos-autovo/|Auto VO]] tool automates VoiceOver tests. The transcript will update live as you move through interactive controls.

===== Testing Color Contrast =====

The various accessibility testing browser extensions will flag instances of insufficient color contrast. To test new color combinations, there are numerous tools. The most basic are:

  * [[https://webaim.org/resources/contrastchecker/|WebAIM Color Contrast checker]]
  * [[https://webaim.org/resources/linkcontrastchecker/|WebAIM Link Contrast checker]]

To find new accessible color combinations, try:

  * [[http://web-accessibility.carnegiemuseums.org/design/color/|Carnegie Museums’ color accessibility guidelines]]
  * [[https://contrast-finder.tanaguru.com/|Tanaguru contrast finder]]
  * [[https://geenes.app/editor/accessibility|Geenes]]

==== Stylesheets and Bookmarklets for Specific Issues ====

Some developers have assembled stylesheets that you can apply to your pages to highlight specific accessibility problems.

  * [[https://a11y-tools.com/bookmarklets/|A11y Audit bookmarklets]]
  * [[https://ffoodd.github.io/a11y.css/|A11y.css]]
  * [[https://absichtbar.de/public/test/zugangstest_missing.css|Annika Brinkman’s CSS warnings stylesheet]]

==== Automated Testing ====

  * [[https://docs.deque.com/devtools-html/4.0.0/en/cli-home|axe DevTools HTML CLI]]
  * [[https://www.smashingmagazine.com/2021/06/automating-screen-reader-testing-macos-autovo/|Automating Screen Reader Testing On macOS Using Auto VO]]

===== UI Checklist for Accessibility =====

==== Context ====

Accessibility problems can often be spotted by just looking for things that seem glitchy or odd to you. If you have a few minutes, a keyboard, or an accessibility testing browser extension, you could help spot even more! Tips and details on some optional tools are available on the Evergreen wiki under Accessibility Testing. The details in the other sections of this page can be used in conjunction with this list. 

Regardless of what tools you use to spot the bugs, once a bug is spotted you should submit a Launchpad (LP) bug report about it! Anyone can create an account and report bugs. You do not need to be an expert or know the cause of the bug you found; someone else can come and add more information to your report as comments. Make sure you include your version number for Evergreen when reporting. More information on LP tags and submission can be found on the wiki page [[dev:bug_wrangler:faq|Bug Wrangler FAQ]]. 

Community test servers are available if you want to test on a system without any customizations or confirm that a suspected bug is not just a glitch specific to your computer. Details about the test systems can be found on the [[:community_servers|Community Demo Servers]] wiki page. You can also email the list servs if you’re not sure about the bug and see if anyone has advice or insight.

==== Using the Checklist ====

This checklist is intended to help you spot signs of common accessibility problems. The list has been broken out into some broad categories, however there is overlap between the categories. If your answer is no to any of these questions, that is an indication there may be an accessibility issue on that interface. Follow up with a bug report!

=== General ===

  * Does the layout of the page make sense for how you use it?
  * Are the layout pieces in proportion to each other?
  * Does the page work how you expect it to?
  * Are colors used only where necessary and do the colors used match their purposes? 
  * Do the words on the page make sense in context?

=== Words ===

  * Does the specific part of the title on the browser tab match the page name?
  * Does the title in the browser tab go from most specific information to least specific?
  * Does the page have a single blue title area, and does it make sense for the page?
  * Do the words match between the page title and any menu links that bring you to the page?
  * Do any error messages make sense and give you enough information to resolve the issue?
  * If there is an error, confirmation, or other type of status message does it stay until dismissed?
  * Do you receive a message when there is an empty state such as “nothing to display?”
  * If your system uses translations, are the labels and text correct?

=== Buttons ===

  * Are the buttons the correct size for their location and purpose?
  * Are the buttons placed in the expected location?
  * Where appropriate, are the buttons in line with the related fields?
  * Do the words and/or icons on the button make sense?
  * If the button only has an icon, is there a tooltip to indicate the purpose of the button?
  * Is the color of the button consistent with the color of similar buttons elsewhere?
  * If the button is red or yellow, is this an appropriate use of these colors?
  * When you use your keyboard, can you tab to the button?
  * When you tab to a button does it get a focus outline?
  * Once you have tabbed to a button, does pressing Enter activate it?
  * If the button changes state based on actions or changes on screen, does this work as you expect?

=== Grids ===

  * Where appropriate, do the columns on the grid have filters?
  * Do the column values display?
  * Do the column values appear as you expected them to?
  * Do any column values that are links open into new tabs?
  * Are the options available on the function bar (grid actions) the ones you would expect?
  * Do the grid settings work as expected?

=== Forms ===

  * Do the labels on the form fields make sense?
  * Does clicking on the form field label place the cursor in the input field or check the box?
  * Is there help or descriptive text where appropriate?
  * Does the text or label match the context without the need for insider jokes/knowledge?
  * Can the form be completed and submitted without the use of a mouse?
  * If pressing Enter in the input should submit the form, does it?