Hi all,
Regarding the implementation of support for screen-reader, Orca on Linux,
I am planning to use qt-at-spi in order to bridge QAccessible API’s to the
AT-SPI 2 protocol enabling MuseScore to be used with Orca.
I also added that to the proposal, but If you guys have any better
alternatives please let me know.
Best Regards!
Samadhi.
On 22 March 2016 at 20:13, Samadhi Poornima <
[email protected]> wrote:
> Hi all,
>
> Taking into account all the facts you guys pointed out, I created my draft
> proposal. You can view it here.
>
> https://docs.google.com/document/d/1ofiRCitgEps9N8rvWYcCjmgdyu4QvUO-RBFKhoduXX4/edit?usp=sharing
>
>
> Waiting for some feedback!
>
> Best Regards,
> Samadhi.
>
> On 21 March 2016 at 21:16, Marc Sabatella <[email protected]> wrote:
>
>> Andrei explained things well. I'll just add a couple of things:
>>
>> Whether we modify the existing "next element" command or add a new one is
>> not important; actually we should get user feedback on that. If we do add
>> a new one, there would be a choice between a single command that traverses
>> all commands (and thus one would use *instead* of the existing command if
>> one wanted to traverse all elements), or perhaps a command that just
>> traverses the "children" of a given element. Any of these designs could
>> conceivably work. And indeed, there is no single "child" concept in the
>> Element class; different elements have different children. We *could*
>> conceivably add a new virtual function "children" to the Element class and
>> instantiate it as necessary for each sub-class that actually has child
>> elements.
>>
>> Marc
>>
>> On Mon, Mar 21, 2016 at 3:47 AM Andrei Tuicu <[email protected]>
>> wrote:
>>
>>> Sorry, accidentally hit send without finishing my email.
>>>
>>> 2016-03-21 11:35 GMT+02:00 Andrei Tuicu <[email protected]>:
>>>
>>>> Hello,
>>>>
>>>> I'm sorry if I wasn't clear before.
>>>>
>>>> 2016-03-21 5:44 GMT+02:00 Samadhi Poornima <
>>>> [email protected]>:
>>>>
>>>>> Hi all,
>>>>>
>>>>>
>>>>>
>>>>> Here are some points that I need to clarify with you guys.
>>>>>
>>>>>
>>>>> 2) Enable the use of keyboard to navigate to a text element or
>>>>> an articulation or a crescendo marking.
>>>>>
>>>>> Use the existing “next element” code to traverse the "annotations"
>>>>> for each ChordRest segment, the Articulations for each ChordRest, etc.
>>>>> Here
>>>>> what is remaining, according to my opinion, is to implement a method to
>>>>> dynamically alter the string which is displayed in the status bar to
>>>>> include attached elements. Andrei, please do share your original plan
>>>>> to implement this. I think it will be helpful here.
>>>>>
>>>>>
>>>> First of all, there is a distinction that needs to be made between what
>>>> is in the status bar and what the screen reader reports.
>>>> The screen-reader usually says more then the it is in the status bar.
>>>> In the status bar you will see only the selected element, the screen-reader
>>>> will also say informations about attached elements.
>>>> Visual impaired musicians will rely on screen-reader feedback only,
>>>> they don't have access to the status bar itself. The status bar info is for
>>>> non-visually impaired users. I hope that I explained this part clearly. If
>>>> not please ask about specific points where you are not sure.
>>>>
>>>> Looking at the code from here:
>>>> https://github.com/musescore/MuseScore/blob/master/libmscore/element.h#L587
>>>>
>>>> - Status bar: accessibleInfo
>>>> - Screen-reader: screenReaderInfo + accessibleExtraInfo (attached
>>>> elements)
>>>>
>>>>
>>>> Ok, at this point here is what you have:
>>>>
>>>> - Screen-reader feedback when selecting an element (for *all*
>>>> elements, including annotations, articulations, text elements, etc.).
>>>> - Screen-reader feedback for all the attached elements when
>>>> selecting notes/rests/barlines/clefs (might be others, but I don't
>>>> remember
>>>> specifically)
>>>> - Status bar info on the selected element (for all elements).
>>>> - A "nextElement" command for reading a score, that will select
>>>> step by step notes/rests/barlines/clefs, basically all the elements that
>>>> are standalone (non-attached). Using this command a user can traverse
>>>> the
>>>> score and find out about all the elements that are there, because the
>>>> screen-reader will tell them about the current selected element and
>>>> about
>>>> all the elements attached to it.
>>>>
>>>> I don't think Marc was saying that you should modify the existing code
>>>> for "nextElement" command (Marc, please correct me if I'm wrong here). You
>>>> can look and see how I did things there in order to help you with your
>>>> implementation.
>>>>
>>>> What you have to do:
>>>>
>>>> Create a command that can traverse *all* the elements. You don't need
>>>> to change the existing code in order to have screen-reader feedback, just
>>>> if you want to improve it, otherwise just selecting elements is enough.
>>>> Users will use this command for editing the score, as opposed to the
>>>> previous one which is used for reading only. My plan to implement this was
>>>> as following: Look at the score as a tree data structure (which it is -
>>>> each element has "children" and a parent: the Score object is the root, its
>>>> children are the measures; the measures have as children segments, etc.
>>>> Marc can give you a crash course in the structure of the score, he
>>>> explained it to me as well) and implement a preorder traversal of this
>>>> tree. My idea of algorithm was as following:
>>>>
>>>> 1. if nothing is selected: select the first element (you have code
>>>> for selecting the first element)
>>>> 2. if the current selected element has children, select the first
>>>> one.
>>>> 3. else go to parent and select the next child.
>>>>
>>>> I don't remember if there is an actual field in the Element class
>>>> "children", so each type of element has children defined differently. There
>>>> will also be some other corner cases for Spanners object, but you will
>>>> figure this things out as you go.
>>>>
>>>>
>>>>> 5) Support for other screen-readers- VoiceOver on OS X and also
>>>>> Orca on Linux.
>>>>>
>>>>> When implementing support for VoiceOver on OS X , I will face a
>>>>> problem since I do not own a Macintosh
>>>>> <https://en.wikipedia.org/wiki/Macintosh> computer. Any word of
>>>>> advice on this issue will be helpful.
>>>>>
>>>>> As Lasconic said, if you don't have a Macintosh don't include this in
>>>> your proposal. Focus on making everything work smooth with NVDA, then look
>>>> at Orca (which is free and open source). If you finish everything else and
>>>> have time to spare you can try to get an OS X. My advice is to try and look
>>>> at ways to virtualise an OS X rather than trying to get an actual Macintosh
>>>> computer. I don't know much about Apple products, but you may be able to
>>>> find trial versions for OS X and there are free solutions for
>>>> virtualisation that support it like VMWare player and VirtualBox. If not,
>>>> you could try cloud solutions. I don't know if Amazon has support for OS X,
>>>> but there is http://www.macincloud.com/ (never tried it, just googled
>>>> it) and they seem to have a student pay plan. What I'm trying to say is
>>>> that there might be solutions to get access for free to a OS X just for a
>>>> number of day which are necessary to take a look at VoiceOver.
>>>> JAWS is another alternative of screen reader on Windows that has a lot
>>>> of users. It has a trial version that requires you to restart your computer
>>>> at every 30 minutes or so, but at least you don't have to pay for it. :)
>>>>
>>>> In addition, I add the following point to my list.
>>>>>
>>>>>
>>>>> 6) Make the palettes accessible to the screen reader.
>>>>>
>>>>> If we have the implementation for QAccessibleWidget for the Palette
>>>>> object already, I think it remains only to implement the part which
>>>>> gathers
>>>>> the information from the selected element and creates the string which is
>>>>> displayed in the status bar and exposed to the Screen Reader.
>>>>>
>>>>> I think what I said was a bit confusing, in the code from my branch
>>> you have to following:
>>>
>>> - support for traversing the palettes using just the keyboard
>>> - screen-reader feedback for the selected element
>>>
>>> In addition you have in the MuseScores master branch from the offical
>>> repository:
>>>
>>> - support for adding the selecting element when Enter/Return is
>>> pressed (implemented by Marc)
>>>
>>> What you have to do here is:
>>>
>>> - rebase/cherry pick my commits on the master branch and resolve the
>>> eventual conflicts
>>> - see if there are improvements to be made, or bugs to fix
>>>
>>> And aside from this comes the idea for searchable palettes that you will
>>> have to implement from scratch.
>>> As explained at 2), there is no direct correlation between the status
>>> bar and the screen reader feedback. The status bar is extra.
>>> Once the QAccessibleWidget is implemented for a QWidget,
>>> QAccessibleInterface for QObject and registered you have screen-reader
>>> support. You might have to implement yourself a QAccessibleWidget depending
>>> on how you design your search feature for the palettes.
>>>
>>> Hope this helps!
>>> Good luck!
>>>
>>> Andrei
>>>
>>>
>>>>
>>>>>
>>>>> Best Regards,
>>>>>
>>>>> Samadhi.
>>>>>
>>>>> On 20 March 2016 at 20:30, Lasconic <[email protected]> wrote:
>>>>>
>>>>>> Just want to add that Braille output is a huge project by itself if
>>>>>> we want to do it from scratch... So I wouldn't focus on this during a
>>>>>> GSoC
>>>>>> project. Your point 1, 2, 4 and 5 are more focused on MuseScore's
>>>>>> accessibility.
>>>>>>
>>>>>> Regarding braille though...
>>>>>> We could of course integrate with freedots, but the project is
>>>>>> currently inactive. However it's the shortest road.
>>>>>> I currently operate http://musicxml2braille.appspot.com/. Doing a
>>>>>> REST API call from a plugin to get Braille out is probably doable. But,
>>>>>> of
>>>>>> course, the plugin needs to be accessible... Not sure it's the case for
>>>>>> any
>>>>>> plugin right now.
>>>>>>
>>>>>> lasconic
>>>>>>
>>>>>> 2016-03-20 7:48 GMT+04:00 Marc Sabatella <[email protected]>:
>>>>>>
>>>>>>> On Sat, Mar 19, 2016 at 1:51 AM Samadhi Poornima <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>
>>>>>>>> 2). Enable the use of keyboard to navigate to a text element or an
>>>>>>>> articulation or a crescendo marking.
>>>>>>>>
>>>>>>>
>>>>>>> I think there is probably a pretty straightforward model for how
>>>>>>> this could work, based on the existing "next element" code Andrei added
>>>>>>> (which navigates clefs and barlines and some other elements, but not
>>>>>>> articulations, text, or crescendos) but making sure to also traverse the
>>>>>>> "annotations" for each ChordRest segment, the Articulations for each
>>>>>>> ChordRest, etc. Here as elsewhere, feedback from actusal blind users
>>>>>>> would
>>>>>>> be useful.
>>>>>>>
>>>>>>>> 3). Braille output- I plan on doing this by calling the FreeDots
>>>>>>>> executable once the user enters the command to import the file to
>>>>>>>> braille.
>>>>>>>> However, this would not be the optimum solution to this since FreeDots
>>>>>>>> is
>>>>>>>> written in java.
>>>>>>>>
>>>>>>>
>>>>>>> There are other concerns here too. FreeDots is a good thing, but it
>>>>>>> is pretty limited in what it can do. There are other automatic or
>>>>>>> semi-automatic MusicXML->Braille converter, each of which also has
>>>>>>> limitations but different progrsams have different strengths and
>>>>>>> weaknesses. There are a few people out there in the world who have done
>>>>>>> fairly extensive study of these issues that you might want to consult,
>>>>>>> also
>>>>>>> it might be good to contact the develoeprs of these tools to get a
>>>>>>> sense of
>>>>>>> their plans for the future to see which tool might make the most sense
>>>>>>> for
>>>>>>> us to focus our integration efforts on. Frankly, I'm not 100%
>>>>>>> convinced it
>>>>>>> wouldn't someday be worth implementing out own Braille export, working
>>>>>>> directly from our own score representation (although I recognize that
>>>>>>> this
>>>>>>> becomes one more thing to maintain over time, so I don't make this
>>>>>>> statement lightly).
>>>>>>>
>>>>>>> Marc
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Transform Data into Opportunity.
>>>>>>> Accelerate data analysis in your applications with
>>>>>>> Intel Data Analytics Acceleration Library.
>>>>>>> Click to learn more.
>>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
>>>>>>> _______________________________________________
>>>>>>> Mscore-developer mailing list
>>>>>>> [email protected]
>>>>>>> https://lists.sourceforge.net/lists/listinfo/mscore-developer
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Transform Data into Opportunity.
>>>>>> Accelerate data analysis in your applications with
>>>>>> Intel Data Analytics Acceleration Library.
>>>>>> Click to learn more.
>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
>>>>>> _______________________________________________
>>>>>> Mscore-developer mailing list
>>>>>> [email protected]
>>>>>> https://lists.sourceforge.net/lists/listinfo/mscore-developer
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Transform Data into Opportunity.
>>>>> Accelerate data analysis in your applications with
>>>>> Intel Data Analytics Acceleration Library.
>>>>> Click to learn more.
>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
>>>>> _______________________________________________
>>>>> Mscore-developer mailing list
>>>>> [email protected]
>>>>> https://lists.sourceforge.net/lists/listinfo/mscore-developer
>>>>>
>>>>>
>>>>
>>> ------------------------------------------------------------------------------
>>> Transform Data into Opportunity.
>>> Accelerate data analysis in your applications with
>>> Intel Data Analytics Acceleration Library.
>>> Click to learn more.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
>>> _______________________________________________
>>> Mscore-developer mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/mscore-developer
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Transform Data into Opportunity.
>> Accelerate data analysis in your applications with
>> Intel Data Analytics Acceleration Library.
>> Click to learn more.
>> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
>> _______________________________________________
>> Mscore-developer mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/mscore-developer
>>
>>
>
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
_______________________________________________
Mscore-developer mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mscore-developer
