forcedDisplay and visibility="hidden"

[nigelmegitt]

Currently the specification for itts:forcedDisplay in both IMSC 1.0.1 and IMSC 1.1 suggests in the introduction that it only applies to content for which the value of tts:visibility="visible" but the normative text does not actually make that distinction.

We have a use case for using itts:forcedDisplay="true" and tts:visibility="hidden" for the same content: the idea is that when text is burned into video and has no equivalent sound an "invisible" caption is forced on to trigger reading by a screen reader, i.e. producing an audible presentation without producing "any visible rendering".

We want to do this in the short to medium term without having to advance to full adoption of condition, and this (direction of text to a screen reader) is not a candidate for the use of tta:speak which is itself not conditional on there being an active screen reader.

I don't think this alone is adequate to trigger the release of a new version of IMSC, however if one is being prepared for any reason, I would propose changing the opening sentence from:

itts:forcedDisplay can be used to hide content whose computed value of tts:visibility is "visible" when the processor has been configured to do so via the application parameter displayForcedOnlyMode.

to

itts:forcedDisplay can be used to prevent visible rendering of content when the processor has been configured to do so via the application parameter displayForcedOnlyMode, regardless of the computed value of tts:visibility.

and add a note:

Note: Applications that send presented text to a screen reader can be configured to do so even when normal display of captions is disabled by the user, without making any visible mark, by using itts:forcedDisplay="true" in conjunction with tts:visibility="hidden". This can be used to cause on-screen text with no audio equivalent to be presented in an alternative medium for users who cannot read that text.

[palemieux]

when text is burned into video and has no equivalent sound

Do you mean like the high school example at https://www.w3.org/TR/ttml-imsc1.1/#forced-content , or something else?

[nigelmegitt]

@palemieux this is a different example. Here the visible text is not provided as part of the TTML document, for rendering purposes, but is already present in the pixels of the video. A consequence of this antipattern is that the text is not accessible to screen reader users, therefore we want to send it in the TTML for non-visible presentation that does get exposed to screen readers. Since it needs to be processed and "presented" even if subtitle presentation is apparently switched off, it is appropriate to use itts:forcedDisplay="true" and since we don't want it to result in any visible presentation, it also seems appropriate to use tts:visibility="hidden".

[palemieux]

Here the visible text is not provided as part of the TTML document

Hmmm. What do you mean by "here"? In the referenced annex, Lycee is provided as part of the TTML document, as a translation of the high school text burned in the video.

[nigelmegitt]

What do you mean by "here"?

By "here" I meant, in the new scenario described in this issue, which, incidentally, doesn't involve translation at all.

[palemieux]

My understanding of the use case:

• the listener is visually impaired, speaks English and uses a screen reader
• the video displays a scene with a a sign containing the text "danger: minefield" (in English)
• the audio does not contain any mention of "danger: minefield"
• the text "danger: minefield" should be sent to the user's screen reader during the scene

Is this the correct understanding?

[nigelmegitt]

Exactly right @palemieux .

[palemieux]

@cconcolato Input on how you would handle the use case at #484 (comment) ?

[palemieux]

@nigelmegitt It feels to me like this is new experience: it is neither captions for the hard-of-hearing (which are visible) nor audio description for the visually-impaired (which is heard). Is this something that the BBC does today, or plans to do in the future?

[nigelmegitt]

@palemieux possibly, it's something a bit like audio description for the visually-impaired, but not the complete experience. BBC currently publishes quite a large number of clips with this visual style of storytelling where the key information is presented as "narrative text", and this is seen as the quickest way to making them accessible for audiences who can't read the text.

[palemieux]

It see two distinct issues here:

1. the introductory text is misleading: itts:forcedDisplay applies regardless of the computed value of tts:visibility. [ed.: I agree this should be corrected.]

2. the proposed Note at forcedDisplay and visibility="hidden" #484 (comment) however hints at an application that goes beyond the intent of itts:forcedDisplay, and has implications for authors and processors, i.e. using itts:forcedDisplay="true" in conjunction with tts:visibility="hidden" and transmitting hidden text to screen readers. [ed.: this deserves a longer discussion, and likely a solution that goes beyond the combination of itts:forcedDisplay="true" in conjunction with tts:visibility="hidden".]

@nigelmegitt Do you wish to split the issue, or keep it as one?

[nigelmegitt]

@palemieux Happy to keep it as one for the time being, at least until we discuss it, noting we agreed to put this on the agenda for TPAC 2019 in our weekly call yesterday.

My reason is that I think a very reasonable outcome might be that sending subtitle/caption text to a screen reader continues to be delegated to the player as an implementation-specific behaviour, as now, in which case only the changes I propose would need to be made.

Of course another reasonable option is to be explicit about what player behaviour should be in relation to screen readers, in which case further edits would be needed and a separate issue would likely be opened.

As a data point, the BBC's Standard Media Player has sent subtitles and captions to screen readers for some time now, by default, using aria-live="polite" and role="region" on the subtitle container div in the HTML rendering.

[palemieux]

Of course another reasonable option is to be explicit about what player behaviour

@nigelmegitt I do not think it is necessary to be explicit about player behavior. It is however necessary to be explicit about the semantics of elements and attributes. I fairly certain that tts:visibility="hidden" Indicates that the element is not visible, perceivable, or interactive to any user -- to use the HTML definition (https://www.w3.org/TR/wai-aria/ and https://html.spec.whatwg.org/multipage/interaction.html#the-hidden-attribute). If the element is intended to be provided to a screen reader even though it is not rendered, then another attribute/element/track would be necessary.

[ed.: added reference]

[nigelmegitt]

Ah, I had assumed that complete removal from the tree would be done by setting display: none; or tts:display="none". MDN suggests that a combination of properties can be used to make content visually invisible but available to screen readers, however in the linked page none of them includes setting visibility.

It's unfortunate that the HTML spec doesn't specify which CSS property is equivalent to making an element hidden in HTML. The fact that the note talks about setting the display property to override the hidden HTML attribute makes me think that the opposite is true too, that setting display: none; is equivalent to setting the hidden attribute, but I don't think this is clear from that part of the specification anyway.

In the ARIA documentation I see that in all the examples they match the value of the ARIA attribute to an equivalent value of the visibility property, for example in aria-hidden - this makes me think they are not intrinsically linked and that maybe it is possible to vary them independently. There's also an explicit mention of display: none; content never being rendered regardless of anything else, which is what I would expect.

If it is legitimate to use visibility in the way I proposed, then we should not add anything else; however if it is true that setting visibility: hidden forces the content to be hidden from the screen reader too, then I agree we may need to specify something special.

[palemieux]

however if it is true that setting visibility: hidden forces the content to be hidden from the screen reader too, then I agree we may need to specify something special.

I suggest exploring approaches in that direction, e.g. a dedicated attribute.

Note that, in the case of forced text, it turns out that, in many scenarios, a dedicated track is created -- instead of using itts:forcedDisplay. This is necessary because the forced text shown on the screen changes when the non-forced text is displayed, e.g. there is less space for the forced text. So it might be the same here: a dedicated track might be needed for the "text-for-the-visually-impaired" use case.

[skynavga]

@nigelmegitt you probably know this, but display is very distinct from visibility, in both CSS and TTML: display controls whether (and how) content layout occurs, such that if it does occur, then it (generally) consumes layout space (in visual media), while visibility controls whether marks generated for that space are rendered (presented) or not; e.g.,

<span>Foo</span><span tts:display="none">Bar</span><span>Baz</span>

has no space between "Foo" and "Baz", while

<span>Foo</span><span tts:visibility="hidden">Bar</span><span>Baz</span>

has space (the width of "Bar") between "Foo" and "Baz";

the reason I'm making this comment is because I was not sure you made this distinction in your comment above;

[nigelmegitt]

Yes @skynavga I was aware of that; I did not make the distinction because I was only concerned with which property matches the HTML @hidden attribute, and because in the use case I have described, it isn't likely that there would be some inline hidden text in amongst non-hidden text. However, taking that semantic into account, the WHATWG specification of the @hidden attribute appears to fit more closely with display: none; than visibility: hidden;.

@palemieux I understand some content providers have made that choice; in this use case the forced text does not affect the non-forced text, so it is likely to be significantly easier to distribute a single TTML file and have the player switch presentation of the non-forced parts. If we can make it work, forced seems like a good fit.

[palemieux]

If we can make it work, forced seems like a good fit.

@nigelmegitt Sounds good. Can you propose an approach that is consistent with TTML, IMSC, ARAI, HTML, WCAG and MAUR?

[nigelmegitt]

Can you propose an approach that is consistent with TTML, IMSC, ARIA, HTML, WCAG and MAUR?

@palemieux I had thought that proposing tts:visibility="hidden" and having the player put the content into an ARIA live region would achieve that, so that's my baseline proposal until anyone tells me it doesn't work.

[palemieux]

had thought that proposing tts:visibility="hidden" and having the player put the content into an ARIA live region would achieve that, so that's my baseline proposal until anyone tells me it doesn't work.

The proposal as it stands does not work IMHO since it changes the semantics of tts:visibility="hidden", which today means not visible, perceivable, or interactive to any user.

Can you propose an alternative/tweak?

[nigelmegitt]

tts:visibility="hidden", which today means not visible, perceivable, or interactive to any user.

@palemieux I haven't found any specification text to support this, looking at TTML2, and the derivations at XSL-FO and CSS - at the moment my understanding is that setting visibility to hidden is not the same as setting the hidden attribute in HTML, and only affects visual presentation.

I've been experimenting with this in HTML and CSS and JS on Firefox and so far my experiments seem to bear this out(apologies I have to stop this soon and haven't got a clear shareable example yet):

• A span where @hidden=true has been set becomes visible when setting display: inline;
• A span where @hidden=true has been set does not become visible when setting visibility: visible;
• A span with visibility: hidden reports @hidden = false.
• A span with display: none reports @hidden = false.

In the first two cases, querying@hidden after changing the CSS property results in true . So a hidden thing that is visible and has display: inline is still rendered.

The only consistent thing I've found is that setting @hidden = false on an element makes it disappear.

Right now it is not clear to me how this is supposed to work, but I haven't managed to work my way around to understanding how the @hidden attribute's description relates directly to anything in CSS. One thing I notice is that it includes the text "User agents should not render elements that have the hidden attribute specified." - interesting "should" there!

[palemieux]

@nigelmegitt Have you considered defining an explicit attribute, akin to forcedDisplay, but for screen readers? Any downside? Would a potential next step be to seek feedback from CSS and the accessibility WG?

[nigelmegitt]

Have you considered defining an explicit attribute

@palemieux I'd consider a new attribute if I felt certain that there is no existing way to get the same effect; I'm not really convinced of that yet, but it certainly wouldn't harm to get feedback from those groups. I'll ask them.

[nigelmegitt]

Further evidence: w3c/csswg-drafts#560 and https://developer.mozilla.org/en-US/docs/Web/CSS/visibility#Accessibility_concerns suggest that as you stated @palemieux setting visibility: hidden; also should hide the content from the screen reader, and that other people have asked for this too.

[css-meeting-bot]

The Timed Text Working Group just discussed forcedDisplay and visibility="hidden" imsc#484.

The full IRC log of that discussion

<nigel> Topic: forcedDisplay and visibility="hidden" imsc#484
<nigel> github: https://github.com//issues/484
<atai> Nigel: This was discussed in CSS WG and with APA
<atai> ...there is a view that CSS speech spec is tended for this kind of purpose
<gkatsev> -> https://www.w3.org/TR/css3-speech/#speaking-props-speak speak css property
<atai> ...erika from CSS WG Commented if you set visibility to hidden but speak property ? it is supposed to be sent to the screen reader
<atai> ...it was not clear from me from the spec that it should behave that way
<atai> gkatsev: spec is ambiguous
<atai> Nigel: I now think there is something needed
<atai> ...CSS WG had volunteers to look at it
<atai> gkatsev: Currently this is defined in CSS3 speech
<atai> nigel: It is a buggy spec
<cyril> rrsagent, pointer
<RRSAgent> See https://www.w3.org/2019/09/18-tt-irc#T01-58-07
<cyril> rrsagent, draft minutes
<RRSAgent> I have made the request to generate https://www.w3.org/2019/09/18-tt-minutes.html cyril
<atai> ...values for speak property are possibly wrong
<atai> gkatsev: The alway value is confusing and needs update
<atai> nigel: they may have thought of the difference between visibility and display but it is not clear
<atai> ...action: we need to wait for the result of the a11y task force
<atai> ...after we have an outcome we can see what to do next
<atai> glenn: in ttml we ended up merging the properties
<atai> ...we have values like normal, fast
<atai> Nigel: There is a difference in CSS:
<atai> ...if you use speech representation that is how it should be styled
<atai> glenn: I thought auto is display none
<atai> Nigel: The idea in for CSS speak that you can direct screen reader for audio representation even if they
<atai> ...do not have a pixel representation
<atai> ...I now see that is a retired note in annotation in the spec
<atai> Summary: Wait what the CSS a11y TF come up with

[palemieux]

@nigelmegitt After reviewing the thread above, I do not see a concrete action for IMSC 1.3. Do you have a different opinion?