"Screen reader"

Jan 18, 2022 Last reply: 4 years ago 8 Replies
[Not exactly, but close enough, semantically.]

I'm starting to design the application to "read" programs to visually impaired users/authors (legally blind, macular degeneration, diabetic retinopathy, etc.). In general, it's a simple problem (given a formal grammar for the language).



The problem is deciding how much detail to convey, audibly. Remembering that the user/author needs to be able to determine how the compiler will treat (or *has* treated) his code.



E.g., the simplest approach is to read all identifiers (deciding on an algorithm to make those unpronounceable ones pronounceable!) and punctuation. This provides *all* of the information that the compiler will see (act on) so a practiced user can recognize incorrect/missing/absent syntax.



But, it's tedious -- esp when you want to read at rates above normal speaking rates (try reading a piece of code this way, REALLY fast!)



I can tweek the syntax (for my applets, but not for system code -- though the latter is much less likely to be reviewed by such folks) to reduce/eliminate extra punctuation as well as alter how certain punctuation is pronounced, based on context.



E.g., eliminate "==" for equality test in favor of "="; pronounce that as "is equal to" in that context and "gets" when used in assignment. Of course, this restricts the syntax of the language -- but, applets code tiny/trivial algorithms... akin to using a "BASIC" enhanced with support for multiple threads, concurrency, IPC, etc.



Similarly, eliminating commas in favor of whitespace (you don't "hear" commas so having to explicitly mention their presence -- so you can deduce their absence -- is extra work that a sighted author wouldn't encounter).



I can also slip into different voices based on context. E.g., quoted constants (so you don't have to pronounce the quotation marks).



As I suspect damn near everyone reading this is using their eyes and likely has *zero* experience with a screen reader, my question relies on *imagination*; imagine having someone read their code to you over the phone (a common metric for readability). What would make this easiest/most expeditious/least prone to error/misunderstanding?



Another option is to allow the user to selectively invoke a "spell that" mode (I use this elsewhere to address sequences of letters/symbols that aren't prnncbl)



Keep in mind that writing code is already tedious for such a user; reading what he's written shouldn't make this (much) worse!


Sometimes even experienced sighted programmers can't figure that out by looking, the rules for operator precedence and variable initialization in e.g. C++ can be convoluted as anything, how many different ways are there to do an initialization in C++ like 15? It's like Bubba Gump shrimp company you got value initialization, direct initialization, copy initialization, move initialization, list initialization, zero initialization, aggregate initialization, static initialization, dynamic initialization, deferred initialization...and some stuff I'm forgetting...

If I write:

struct Inner { int arr[2]; };

Then can you tell me by looking how the compiler treats each of these. The first two aren't the same, and the last one doesn't even strictly make sense but they added a rule (brace elision) so it does.

int main() { Inner inner1; Inner inner2 { }; //Since C++11 only! Inner inner3 { { 1, 2 } }; Inner inner4 { 1, 2 }; }

Fun times.

For certain languages (see above) I would avoid altering _anything_, unless you fully intend to restrict the syntax of the language too; the majority of people don't know the whole of a language like C++ or even Java well enough to be 100% sure they aren't breaking something.

int a, b = 0;

is valid code, it doesn't mean the same as:

int a, b; a = b = 0;

which means the same as:

int a = 0; int b = 0;

but

int a b = 0;

is just an error. If you eliminate commas how they going to know whether they typed valid code or an error?

If they're using a modern OOP language and having trouble digesting the content of a method or class function, or dozing off while it's read aloud it means the method is too big.

Inside methods is where the bulk of stuff you can't just tab through the hierarchy like the index of a book lives and if it's undigestable via TTS for a sight-impared person there's a high probability it's undigestable to someone who can read it, also - good standards and practices apply to all coders, regardless of disability status.

That is to say I'd consider carefully whether I want to make writing a code-decruftifier/intent-determiner part of my job description, also.

IME, "existing languages" are too cumbersome to annotate in this manner -- as well as the issues you've addressed, here.

Target audience isn't "engineers" or "programmers" or even "coders" but, rather, "Ma & Pa Public". They want to be able to piece together an applet using bits of stuff that they've seen elsewhere without concern for idiosyncrasies of some language designed for (sighted!) programmers. Or, use some code snippet that someone has posted as appropriate for their desired task.

"If Marijane comes to the front door, let her in"

"When I go to bed, make sure the lights are off in the house -- except the nightlight in little Billy's bedroom and the front porch light".

"When the afternoon sun starts shining through the living room windows, close the blinds, there".

"When I open the garage door to pull the car in, alert me to anything that might be in the car's path before I run the risk of running over it".

"If more than 0.25 inch of rainfall, today, then defer the irrigation cycle for the cacti (but not the roses!)."

"If Fred calls, tell him 'I'm Out. Call me at 555-1234' and offer to take a message".

Because you wouldn't use a language that assigned significance to commas beyond that of delimiters. The whole point is not to *need* punctuation because punctuation complicates presentation.

a = 0 b = 0

is not going to tax *any* coder due to the absence of the comma (and semicolons)

Applets are a "page" (whatever that means). An applet typically ties together a bunch of preexisting services in a (very) high level form. E.g., recognizing that "Marijane" is at the door, doing whatever it takes to invite her in and then ensuring the locks on the door are released. Afterwards, verifying that the door is re-closed, etc.

A "programmer" would explicitly indicate each of these things. A "user" shouldn't have to.

Think in terms of the attitude BASIC exhibited when it dealt with "programmers" (they don't need to understand how numbers are represented, don't need to understand how to talk to other programs, don't need to know how to synchronize with other events, etc.).

As a result, what they write is reduced to only simplistic notions expressed at a very high level.

OK I guess I was a bit unclear as to whether this was for your own custom sandbox or programming languages in general.

C++ is a particularly pathological example of why deducing intent in a very "expressive" (euphemism) language is a tough problem.

If you are trying to make a professional app for say, windows and starting from scratch, you should probably check out one of the MANY screen readers and computer navigation programs already out there.

The one I am semi familiar with is called JAWS and has been out for decades now. They work but can always be better. And OS's seem to change a lot. It will be a pain in the ass I have a feeling but should be a fun learning experience.

formatting link
boB

Yes, I've used JAWS, NVDA, BRLTTY (with a braille display), "Narrator" (MS) and EMACSpeak. (I also looked at Mercator but it was an idea lacking legs.)

With the exception of EMACSpeak, all are very general purpose and include provisions that aren't necessary if you know you are reading "code" and the code's syntax is tightly controlled. Ideally, you just want to *listen* and not have to interact with the screen reader's parsing of the code (so, you design the grammar to avoid ambiguities and "silent" features -- can you hear if I've placed a comma, semicolon and/or space on this line?)

That is a proverbial understatement!

Have you ever tried reading USENET, for example, with a screen reader? Or email? And, those are largely "fixed format" applications! Reading free-form code is a different challenge, entirely!

[a screen reader that knows to pause on encountering a comma or period or other such punctuation is "working properly". Failing to make the actual punctuation mark knowable to the user has higher consequences when it, itself, is significant -- to the content/code]

Can you see which of these lines of text are replies vs. quotes? What if someone (upthread) in the conversation had used ')' for the quote introducer? Or, ':'?

Yes, it *is* a learning experience. And, more folks should try it just to see what the obstacles are (and what may lie ahead for aging/diabetic folks in their futures)!

There are some interesting gotchas lurking in the undergrowth. Back in the bad old days when variable names had very limited length it was not uncommon to zap the vowels out of variable names. This works OK in your own native language but gets confusing when it isn't.

I first ran into it in French where stars were abbreviated TL. (eToiLe)

Variable names that are essentially unpronouceable will be an issue and the more awkward ones might have to be spelt out.

Punctuation like ";", ",", ".", and ":" have significance beyond being mere punctuation in some computer languages. Likewise other escape sequences and operators "||" vs "|", "*" and "&" in C for example.

Having distinct distinguishable voices for main code, one for embedded strings and another for comments might make it a lot clearer to the listener and also break up the otherwise monotonous single voice. Women's voices are easier to listen to and clearer to understand.

I know someone who uses one. It is very clunky especially on Usenet posts where it reads quite a bit of eg. quotation junk. It would drive me crazy not to be able to flash read whole chunks at a time.

I think the next generation of personal assistants might be going in the other direction allowing a natural language interface to let people describe what they want their assistant to do for them rather than having to code in a primitive text based language or IFTTT.

Think a slightly more upmarket version of Alexa or Siri. Already some of the apps for that can be quite potent in chosen domains. Ours remembers which bins to put out this week far better than I can.

Join the Discussion

Have something to add? Share your thoughts — no account required.

Didn't find your answer?

Ask the community — no account required