Re: [Libreoffice-qa] Extending subsequent tests with dogtail tests?

2019-03-07 Thread Markus Mohrhard 
Hey,

let me at least add some comments.

On Thu, Mar 7, 2019 at 5:42 PM Jean-Philippe MENGUAL <
jean-philippe.meng...@libreoffice.org> wrote:

> Hi,
>
>
> Le 06/03/2019 à 20:52, Samuel Thibault a écrit :
> > Hello,
> >
> > Markus Mohrhard, le lun. 25 févr. 2019 00:34:33 +0800, a ecrit:
> >> On a slightly related note I think that we have already quite a few
> tests for
> >> the accessibility UNO layer but as that layer is full of bugs many of
> the tests
> >> are disabled. It might be a good idea to work on these tests before
> actually
> >> trying to implement more complex tests that depend on lower layers
> working
> >> correctly.
> >
> > I'm not sure which piece you are referring to.  Is that the AWB?  I
> > indeed see some source code in toolkit/test/accessibility but no
> > reference to it.
>

There should be accessibility UNO API tests in most top level modules. E.g.
from a quick git grep Accessible -- qadevOOo/Jar_OOoRunner.mk:

qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessible \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleAction \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleComponent \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleComponent \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleContext \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleEditableText \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleEventBroadcaster \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleExtendedComponent \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleImage \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleSelection \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleTable \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleText \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/ifc/accessibility/_XAccessibleValue \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_basctl/AccessibleShape \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/AccessibleEditableTextPara_HeaderFooter \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/AccessibleEditableTextPara_PreviewCell \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sch/AccessibleDocumentView \
qadevOOo/Jar_OOoRunner.mk:qadevOOo/tests/java/mod/_sc/ScAccessibleCell \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessibleCsvCell \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessibleCsvGrid \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessibleCsvRuler \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessibleDocument \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessibleDocumentPagePreview \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessiblePageHeader \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessiblePageHeaderArea \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessiblePreviewCell \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessiblePreviewHeaderCell \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessiblePreviewTable \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sc/ScAccessibleSpreadsheet \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sd/AccessibleDrawDocumentView \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sd/AccessibleOutlineView \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sd/AccessibleSlideView \
qadevOOo/Jar_OOoRunner.mk:qadevOOo/tests/java/mod/_sm/SmEditAccessible \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_sm/SmGraphicAccessible \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleBrowseBox \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleBrowseBoxHeaderBar \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleBrowseBoxHeaderCell \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleBrowseBoxTable \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleBrowseBoxTableCell \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleIconChoiceCtrl \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleIconChoiceCtrlEntry \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleTabBar \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleTabBarPage \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleTabBarPageList \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleTreeListBox \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svtools/AccessibleTreeListBoxEntry \
qadevOOo/Jar_OOoRunner.mk:
qadevOOo/tests/java/mod/_svx/AccessibleControlShape \
qadevOOo/Jar_OOoRunner.mk:

[Libreoffice-qa] minutes of ESC call ...

2019-03-07 Thread Michael Meeks 
* Present:
+ Olivier, Miklos, Michael M, Christian, Eike, Caolan, Heiko,
  Michael W, Sophie, Michael S, Thorsten

* Completed Action Items:

* Pending Action Items:
+ add monitoring for build bot daily failure rates (Christian)
+ review/update GSOC ideas page! (Mentors)
  + https://wiki.documentfoundation.org/Development/GSoC/Ideas

* Release Engineering update (Christian)
+ 6.2.1 rc2 status
   + released as final yesterday
+ 6.2.2 rc1
   + pushed to mirror network the same day.
+ 6.2.2 rc2
   + next week – will tag towards end of the week.
+ 6.1.6 rc1: April 9
+ 6.3 alpha1: early May
+ Remotes
+ Android viewer
   + Mavern repo is not up-to-date will fix this week.
+ Online
   + 6.2.1 source release out. 

* Documentation (Olivier)
 + New Help
 + Some XSLT optimizations (ohallot)
 + Improvements in build robustness (part) (sberg)
 + Helpcontent2
 + Addressing menus changes for Impress/Draw (buovjaga)
 + Nomenclature changes (kompilainen)
 + Add release info for Calc fucntions (ohallot)
 + other improvements (libreofficiant)
 + Guides
 + resuming Calc guides review wiki page (T.Thibodeaux) 

* UX Update (Heiko)
+ Bugzilla (topicUI) statistics
248(248) (topicUI) bugs open, 240(240) (needsUXEval)
+ Updates:
BZ changes   1 week1 month   3 months   12 months  
 added  2(-4) 12(-5)29(-7)  87(-11)
 commented 23(-32)   127(-8)   401(-27)   1408(-32)
   removed  0(-3)  3(0)  7(0)   17(-1) 
  resolved  6(4)  16(2) 47(3)  127(3)  
+ top 10 contributors:
  Tietze, Heiko made 74 changes in 1 month, and 575 changes in 1 year
  Mehrbrodt, Samuel made 32 changes in 1 month, and 71 changes in 1 year
  Dieter Praas made 24 changes in 1 month, and 149 changes in 1 year
  Foote, V Stuart made 24 changes in 1 month, and 210 changes in 1 year
  Cor Nouws made 23 changes in 1 month, and 106 changes in 1 year
  Thomas Lendo made 21 changes in 1 month, and 326 changes in 1 year
  Xisco Faulí made 14 changes in 1 month, and 265 changes in 1 year
  Telesto made 12 changes in 1 month, and 58 changes in 1 year
  kompilainenn made 11 changes in 1 month, and 110 changes in 1 year
  Kainz, Andreas made 10 changes in 1 month, and 159 changes in 1 year

 + new tickets
   + zarro
   + is that because Xisco hands them to you normally ? (Michael)
   + probably – he’s on vacation (Heiko)
  + or it is far too usable already  

 + easyhacks
   + Make shortcuts extensionizable
 + https://bugs.documentfoundation.org/show_bug.cgi?id=123768

 + ack request
   + Remove access to Firefox Persona in favor of self-hosted extensions
 + https://bugs.documentfoundation.org/show_bug.cgi?id=123831
 + Moz changes API continuously – want themes of our own etc.
   => remove & ideally switch to new extensions

* Crash Testing (Caolan)
+ 7(+6) import failure, 2(-2) export failures
  - mst has additional fixes which will probably fix the last of the
red-lines in next build
  – but new asserts/crashes have emerged
- interesting new asserts ? Or what triggers them ?  
+ coverity issues
  - coverity is back up
  - but we now need c++17, which isn’t working yet
  - asked coverity, and “new version will be deployed in the coming few
months”
+ Google / ossfuzz:
  - 11 (+3) issues, 9 timeouts, 1 OOM
  + found other one – but not really serious.
+ ForcePoint, round 9
+ still some issues outstanding
– layout issues with additional asserts in ordinary docs.
- unclear if this is the original reason, or something new.
+ CNA process
+ positive progress.

* Crash Reporting (Heiko)
  + http://crashreport.libreoffice.org/stats/version/6.1.3.2
+ 905 (-134)
  + http://crashreport.libreoffice.org/stats/version/6.1.4.2
+ 1195 (-65)
  + http://crashreport.libreoffice.org/stats/version/6.1.5.2
+ 719 (+275)
  + http://crashreport.libreoffice.org/stats/version/6.2.0.3
+ 2532 (+718)
  + lots more issues on fresh:
+ Hike in the download numbers recently
+ not entirely clear where it comes from; nothing interesting.
+ mostly from Spain, .cz, Poland, distributed etc.
+ people are using LibreOffice more often it seems → crash rate.

* Hackfests & Events (Thorsten)
   + this year hackfests coming up:
+ Tirana perhaps
  + April 25th April 28th → Easter variously.
  + May 4th / 5th  → conflict with German meetings
  + https://forum.openlabs.cc/t/libreoffice-hackfest-tirana-2019/1658
  + poke Bjoern / Eike / Xisco to chase / decide…
+ Paris in Summer
  + first week of July Fri/Sat 5th /

Re: [Libreoffice-qa] Extending subsequent tests with dogtail tests?

2019-03-07 Thread Jean-Philippe MENGUAL 

Hi,


Le 06/03/2019 à 20:52, Samuel Thibault a écrit :

Hello,

Markus Mohrhard, le lun. 25 févr. 2019 00:34:33 +0800, a ecrit:

On a slightly related note I think that we have already quite a few tests for
the accessibility UNO layer but as that layer is full of bugs many of the tests
are disabled. It might be a good idea to work on these tests before actually
trying to implement more complex tests that depend on lower layers working
correctly.


I'm not sure which piece you are referring to.  Is that the AWB?  I
indeed see some source code in toolkit/test/accessibility but no
reference to it.


The focus handling can be easily integrated into the existing UI
testing infrastructure and might benefit there from some of the concepts that
should make them more stable


Good :) So could you plan to work on it?


How could such plan be scheduled in LO qa? Should we report a bug? Or 
open a wiki roadmap?


The problem now is that it is not possible to fix accessibility bugs, 
fix regressions from 4.2, integrating in the code non-regression tests, 
and do the same for the three major programs in free software. It is 
less a will problem than a resource problem, because even with funds, we 
do not have enough persons to work on this with skills related to 
Libreoffice and accessibility in general. As you know more and more 
persons go the web or backend techno, less in programming for desktop 
software.


So while we are ready to fix accessibility bugs, we need non-regression 
tests. And it is difficult to do both. The tool on which we are working, 
dogtail, is interesting because enables to test via the same framework 
different programs, without needing to know the code of each of one. I 
think gateway is possible between Libreoffice framework test and such 
tool. We also could imagine a dedicated machine with dogtail to test, 
but should be acceptd. Also, the thing is to know if LO is ready to 
prevent a release or a commit because introduces a regression in 
accessibility.


Well to sum up, beyond our effort, that we try general and 
cross-software, we would need to know how we can together set a kind of 
roadmap to implement accessibility in the existing frameworks, and add a 
layer to make common scenarios between our general tool and LO's one. 
Having that may be in the easy hacks to have contributions? Or b a 
specific TDF tender just like we did for labels?


Regards


Samuel
___
LibreOffice mailing list
libreoff...@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/libreoffice



--
Logo Hypra  JEAN-PHILIPPE MENGUAL
DIRECTEUR TECHNIQUE ET QUALITÉ
102, rue des poissonniers, 75018, Paris
Tel : +331 84 73 06 61  Mob : +336 76 34 93 37

jpmeng...@hypra.fr 
www.hypra.fr 
Facebook Hypra  Twitter Hypra
 Linkedin Jean-Philippe



___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: https://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/