Thinking about WCAG Conformance in a different way #724
Replies: 4 comments 8 replies
|
Several very good ideas in here
|
|
One request for qualification , Wendy. You write
I am not sure what you mean. Can you illustrate that? And to address one of your proposals for new ways of looking at WCAG:
There are two aspects that come to mind when I hear deterministic:
Regarding the notion that "each behaviour is attached to a means to achieve it", I believe it is not so easy. WCAG 3 will also have different ways of achieving conformance to a requirement, and the gray areas won't disappear. The reported horror of regulators towards nuance, percentages, or whatever is used to render those grey areas, may untimately induce us to gloss over them and corral us in deterministic simplicity. It is a shame, but perhaps it needs to be accepted as an unavoidable outcome artefact of a conformance standard. |
|
Thanks @detlevhfischer I'll try to answer:
Whenever we talk about conformance considerations such as scoring, levels, progress, etc. inevitably it feels like conversation gets derailed by "if we do that, people will game the system" or "regulators will interpret this as that and so we need to make sure they don't by...". I don't intend to suggest these aren't things to consider, but I do find it tends to shut down the conversation. A professor of mine introduced me to the concept of exploring bad ideas in order to get to good ones, and I think I want this discussion to be a space where we explore some of these things without fear of getting shut down. I (personally) hold the potentially controversial view that we have no control over what people do, and so gamificiation should not be a concern because it is up to the person/entity whether they want to game the system, and it's up to the regulators to set up guardrails against that. We simply offer a system for the reporting of information. Similarly for regulators, I think we should offer helpful guidance on how to interpret WCAG 3 and interpret conformance reporting, but we have the same control issue – we can offer as much guidance as we want, it is going to be the discretion of the regulator whether they take it or not. With that in mind, what does a conformance model look like that just focuses on how testing is performed, how results are reported, and how to interpret those results? |
|
In the spirit of offering up a potentially bad idea that might lead towards a good one, I did have an idea of how we might take the concept of WCAG as a part, or many parts, and what that might look like. Especially when we have such a large number of provisions and a lot of diversity within the provision set. WCAG as "sets" of provisionsIf we take a high-level look at all of the WCAG3 provisions, we have sets within the whole (note: example provisions are only meant to be illustrative of the set, they are completely up for debate):
The breakdown into sets does not answer the conformance question, but I do think it makes talking about conformance a bit easier in the sense that now we can potentially look at measurements that are tailored to the nature of the set, instead of trying to find a measure that somehow captures a very broad range of considerations. The individual measures can be further normalized to offer comparison between sets, testable units, or products as a whole. Smaller sets (assertions and system) may be easiest to simply assess conformance by % of whole. Larger sets such as the deterministic/subjective provisions might benefit from determining levels or progression that can be easier mapped to user expectations or needs. Sample(Again, bad idea leading to good, so focus on the concepts not the content): If I have a small e-commerce website with around 30 pages to assess for WCAG 3 conformance, I have to assess 30 pages against ~220 provisions. Since WCAG3 has provision sets, I can further break down some of my testing process depending on the set. For the Deterministic provisions, I can run a tool across my entire codebase to look for failing provisions and address those by creating issues automatically. For the System/Architectural provisions, I look at our website builder tools and check that my tools meet the requirements, my video player module allows upload of captions and audio descriptions, my CMS offers necessary tools and checks for content accessibility. For Assertions, I can look at the set and assess what assertions apply to us, and look at potential assertions we can implement. This leaves the Subjective provisions, which require a more detailed approach. To make the process less daunting I determine the testable units of my website by user tasks/flows. This narrows down the 30 pages to 8 sections:
I can then work through the subjective provisions section by section, tracking passes and failures and logging them. I can also re-run the deterministic set through each section as it helps with testing. Based on the results for each section, I can review them to get a picture of how each section of my site is performing against each other. I can then take those results in combination with others to produce my accessibility statement/conformance report. (Note: Completely made up numbers I haven't taken a chance to try and sort provisions into the sets) MyStore.ca Conformance Report
Assertions: 20/29 provisions met
Deterministic/Subjective Results by section of website, for more information on levels (link to WCAG3/some kind of informative documentation):
|
|
The basic question from me is: What benefit is there from including this is normative conformance? What is presented is a useful looking report, but apart from (perhaps) tagging provisions with a type, it looks like it could all be done in informative reporting guidance. On the other hand, it would need to be in conformance if:
Perhaps there's another reason I can't think of? |
|
While I originally conceived this as a normative conformance model, I'm with you, I think this is one way to do an informative one. I definitely wouldn't consider the types as levels themselves, but as I suggested, especially for what might be the larger type groups, we might consider levels within those just to break up what could be an overwhelming set of requirements. I also don't think we are serving anyone by requiring a reporting structure. |
One way of approaching this would be:
For me this is the "KISS" approach, get on with crafting the provisions with a simple conformance model, and provide guidance for everything else... |
|
@alastc Hmmm. I am afraid that would result in a real "all or nothing" or else complete disharmony. If we did this, each regulator, legislator, or someone setting policy for company or other entity would have two choices. Either require all, which would immeasurably increase difficulty in conforming OR pick and choose what to require and lose all the harmony we have with every place picking different sets of requirements. |
|
@GreggVan, this approach is similar to your conformance proposal, so I guess you're disagreeing with yourself?? Le me check: Apart from having 2 levels in requirements, your proposal didn't have anything else in normative conformance, the rest was informative material. Am I missing something there? I was also assuming we'd have robust guidance for policy makers (and methodology & reporting), to mitigate the disharmony. I was also assuming we'd divide what is now "supplemental" into requirements and recommendations. I was sort of posting this as a strawman, but given how hard it is to get agreement on anything above this, it seems like a useful fallback. |
|
@alastc I was having this exact thought looking at the survey yesterday and trying to think of how each suggestion would fit into normative or informative models. If there is anything we've learned from WCAG2, a big challenge is that testing and implementing the success criteria is highly dependent on context, and SCs can be easier or harder to manage depending on a huge number of factors. SCs can also be more or less important depending on different factors as well. For WCAG3 we know this complexity exists, we know it is why there is literally no such thing as a "WCAG2 conformant" website (with some exceptions obviously but we're speaking at scale), and we want to design a conformance model that somehow balances knowing that all provisions are important to someone with knowing there has to be measurable criteria to assess and report on. We also need some way to reliably tell implementers what to do to make products accessible and tell users when the products they want to use are accessible in a way that meets their needs. With that in mind and looking at all the proposals like progress to conformance, levels, paths, severity, etc. I think we actually may have no choice but to abandon the idea of a normative conformance model for WCAG3 and instead focus on informative documentation that can support methodologies that reflect the complexity. WCAG 3 "conformance" becomes like any other technical spec, you either do or do not conform to the provisions, and we can provide normative guidance on testing the provisions to ascertain conformance. The informative documentation can then accommodate how to approach and report on conformance, and the flexibility of doing this in informative documentation means we can offer multiple approaches, update them as needed, and provide as much background or supplemental information as we want. Sure, from a regulatory or policy perspective it could mean that they just look at the WCAG3 document and ignore the guidance, but it could also mean policy that considers the complexity of accessibility for different industries, organization sizes, or user groups. |
I'd say "the idea of a more complex conformance model", rather than not having one at all. But a big part of that would be using the guidance to normalise accessibility as an ongoing journey, and always needing to look at the detail of conformance reporting rather than the yes/no answer at the top. Perhaps a useful approach would be to include a simple conformance model and work on the informative guidance. Once that is drafted, if there are aspects of that which appear to be useful to be in the normative section, great, we can move that in. |
|
+1 to "the idea of a more complex conformance model" that is really what I mean. The upside with moving the "how" of quantifying and explaining a product's accessibility to informative documentation means we can offer multiple options or approaches that may suit some use cases better than others. We can even pull from the models we see in the regulatory and policy space today. Potentially once we take the pressure off the idea of creating the normative conformance model and instead let ourselves get creative with informative guidance, we'll see some things emerge that could be normative, I totally agree. I do think the work around levels is a strong contender, especially if we can settle on a way to score provisions that makes arranging them into levels an easier task. |
Uh oh!
There was an error while loading. Please reload this page.
I want to continue the thread we started in the meeting today (June 23) and create some space for a little bit of creative imagining when it comes to WCAG/accessibility conformance.
Before anything, I want to lay out some ground rules for this discussion to flourish:
We have been living in a world for the last 20+ years where WCAG has been more or less the sole measure of a website or application's accessibility. It has been written into laws, regulations, corporate and government standards. It has been the basis of multiple software products, companion specifications, and has sort of crystallized into how we think about measuring and achieving digital accessibility. It has taken on a near-mythic status as the yardstick for accessibility, while every single person who works with it every day agrees it is only one part of the equation.
What if we explore a model where WCAG is just one part of how we measure and assess a product's accessibility? Where WCAG is a standard for achieving accessible content, but exists as a node in a broader ecosystem of standards that focus on different aspects of digital accessibility.
Thinking from this perspective, I believe we can look at WCAG and its conformance in new ways:
We already have a concept of the broader ecosystem, we have satellite standards like ATAG, UAAG, EPUB Accessibility, EN 301 549, etc. We have existing models to explore as comparison, such as the ISO/IEC 27000 family of information security standards. We also within the WCAG3 work have the concepts that could form foundations for companion specifications, such as assertions.
Most importantly to this discussion, I want to imagine what WCAG looks like when we don't try to out-game bad actors or create mitigations for every possible regulatory scenario. I want to imagine what WCAG looks like when we think about how we can create an environment that is supportive of creators building accessible experiences and maintaining or improving them over time. A model that rewards effort and supports learning.
All reactions