Re: [Libreoffice-qa] automated testing of Base

[Terrence Enger]

On Thu, 2011-10-06 at 17:13 +0200, Alexander Thurgood wrote:
> Hi Terry,
> 
> Le 06/10/11 16:40, Terrence Enger a écrit :
> 
> I can only speak from my perspective of an "enterprise" user, and thus
> possibly biased ;-)
> 
> > Do we have a feeling for what databases are most used "for real" with
> > Base? Or how they are used? 
> ODBC is still used as an enterprise-wide solution in many places

Thank you.  That helps to keep my interest up.


___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[Alexander Thurgood]

Hi Terry,

Le 06/10/11 16:40, Terrence Enger a écrit :

I can only speak from my perspective of an "enterprise" user, and thus
possibly biased ;-)

> Do we have a feeling for what databases are most used "for real" with
> Base? Or how they are used? 
ODBC is still used as an enterprise-wide solution in many places because
of its generally good performance, usually much better than the
corresponding JDBC drivers especially with regard to OOo/LibO. The whole
Java encapsulation thing always had performance issues that ODBC simply
did not. That being said, there seem to be far more actively maintained
JDBC drivers around these days than ODBC drivers, probably because it
fell under the "write once/run anywhere" (uh-huh) motto that was touted
for years with Java development.

> Or what open bugs are most deserving of attention? 
Too many to be reasonably approached by one person in a lifetime ;-) 
Just kidding, but there are quite a few, as with the other modules, and
depending on who you ask, they _all_ deserve attention (grin).


> How much real use is through ODBC as compared to JDBC as compared to
> built-in drivers? 
Hmm, like I said, from what I have seen being used in various companies,
JDBC would appear to come out tops in usage terms because of its
platform/OS agnosticity.

The built-in "native" ones are however highly appreciated because they
appear seamless and don't depend on Java...and on the whole their
performance rocks, whereas JDBC performance within LO dives rapidly,
depending on the release version of Java, the driver itself, and the
poor memory management within Base where everything has to be loaded and
held in memory until you write out it all back out in the IOstream to
the file.

I had always implemented ODBC enterprise wide in earlier versions of OOo
on Linux because (1) there were no native drivers at the time, and (2)
performance was excellent with ODBC, but many ODBC solutions for some db
engines are only available for one or two OSes (for example on Mac, it
is a rather large PITA no thanks to Apple), which obviously is a serious
slug in the chest when you have heterogenous environments. That is why
the native cross-platform driver is so appealing :-)

Obviously, all of the above just represents my small window on the world
of DB connectivity that I've either experienced, implemented or come
across in the last 20 years.

Alex

___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[Terrence Enger]

On Thu, 2011-10-06 at 09:31 +0100, Michael Meeks wrote:
> Hi Drew,
> 
> On Thu, 2011-10-06 at 00:57 -0400, drew wrote:
> > > well, kind of - there is a number of features that do not work with ODBC
> > > - so I'm just curious what the reason for using SQLite and ODBC would be
> > > for a test bed?

One motivation is that ODBC data sources can encapsulate
credentials that I ought not to reveal to the whole world.

> 
>   So - the ideal test code that we have in the build is small, fairly
> self-contained, builds on all platforms and is fast to execute - so we
> can run it every build cycle. That militates against anything using
> Java, and hsqldb in particular. Sqlite OTOH is my preferred replacement
> for hsqldb, and getting it building and being used in the tree would be
> a great start IMHO.

Yes, anything that is part of every build cycle must be
fairly self-contained.  OTOH, it is central to the purpose
of Base that it reach out to the rest of the world: just
look at module name "connectivity".

> 
> > *smile*...maybe I should rephrase that - what could I do to help
> > generate some of the tests?
> 
>   Heh - so working with Terry & Bernhard to get some VM's setup for
> openQA would be a great idea I think; hopefully it's scaleable.

VMs let us choose what database systems are installed; and
working outside the build cycle relieves the requirement for
"fast", letting us put more value on "thorough" if we so
choose.

Do we have a feeling for what databases are most used "for
real" with Base?  Or how they are used?  Or what open bugs
are most deserving of attention?  How much real use is
through ODBC as compared to JDBC as compared to built-in
drivers?  In short, what will guide us through the many
things that VMs would let us do?


Heh, heh, heh.  I started this thread with the notion that I
would be risking little more than my own time, hopefully
bothering the experts only occasionally with questions like
"Is this worth looking at?" or "Okay, so what should I read
next?" or "Okay, so what should I do next?".  I have time
available unless a customer calls (I wish!), but as the
topic grows wider I see a danger of absorbing more expert
help than my anticipated contribution is worth.  Guidance
welcome.

Terry.


___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[Michael Meeks]

Hi Drew,

On Thu, 2011-10-06 at 00:57 -0400, drew wrote:
> > well, kind of - there is a number of features that do not work with ODBC
> > - so I'm just curious what the reason for using SQLite and ODBC would be
> > for a test bed?

So - the ideal test code that we have in the build is small, fairly
self-contained, builds on all platforms and is fast to execute - so we
can run it every build cycle. That militates against anything using
Java, and hsqldb in particular. Sqlite OTOH is my preferred replacement
for hsqldb, and getting it building and being used in the tree would be
a great start IMHO.

> *smile*...maybe I should rephrase that - what could I do to help
> generate some of the tests?

Heh - so working with Terry & Bernhard to get some VM's setup for
openQA would be a great idea I think; hopefully it's scaleable.

HTH,

Michael.

-- 
michael.me...@suse.com  <><, Pseudo Engineer, itinerant idiot

___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[Michael Meeks]

Hi Bernhard,

On Wed, 2011-10-05 at 21:41 +0200, Bernhard M. Wiedemann wrote:
> Yes, there are some spare cycles. If long tests need to be added, I 
> could make it schedule them only daily like the extensive mozilla tests.

Wonderful :-) The database bits are a pain to validate without your
approach I think.

> I had met Mechtilde at FOSDEM this year. She told me about a huge 
> OpenOffice/LibreOffice testsuite (libreoffice-testtool) that runs hours 
> or days. So far I had not come to setting it up on openQA.
> And I don't know how reliable it is in terms of false positives.

In a nutshell it is a fairly useless maintenance disaster - we are
working hard at producing much more reliable, in-code unit tests that
~every builder and tinderbox can run during the build, and that execute
quickly.

The "runs for days" element of the test-suite was mostly a function of
poor design and implementation - and most of the time was spent in
'sleep' calls ;-)

> If you have some script that can setup and run a test, I can integrate 
> it easily.

Cool - well that'd be for Terrence to look into I guess :-)

Thanks !

Michael.

-- 
michael.me...@suse.com  <><, Pseudo Engineer, itinerant idiot

___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[drew]

On Wed, 2011-10-05 at 23:56 -0400, drew wrote:
> On Wed, 2011-10-05 at 22:27 -0400, Terrence Enger wrote:
> > On Wed, 2011-10-05 at 11:00 +0100, Michael Meeks wrote:
> > > 
> > >   Ho hum ;-) is there an ODBC driver that sqlite provides that we could
> > > use (in conjunction with a SQLite backend (currently there is none)) to
> > > make a fast/light unit test for base ? it would be fantastic to have
> > > anything in this space at all really ...
> > 
> > FWIW, on ubuntu-natty, I installed some packages; I think
> > the relevant ones are sqlite3 and libsqliteodbc.
> > LibreOffice, both Ubuntu package 1:3.3.3-1ubuntu2 and local
> > build of commit id 0e1de41, is happy to work with an ODBC
> > connection.
> 
> Hello Terry.
> 
> well, kind of - there is a number of features that do not work with ODBC
> - so I'm just curious what the reason for using SQLite and ODBC would be
> for a test bed?

*smile*...maybe I should rephrase that - what could I do to help
generate some of the tests?

> 
> Thanks
> 
> //drew
> 
> ___
> List Name: Libreoffice-qa mailing list
> Mail address: Libreoffice-qa@lists.freedesktop.org
> Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
> Problems? 
> http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
> Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
> List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
> 


___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[drew]

On Wed, 2011-10-05 at 22:27 -0400, Terrence Enger wrote:
> On Wed, 2011-10-05 at 11:00 +0100, Michael Meeks wrote:
> > 
> > Ho hum ;-) is there an ODBC driver that sqlite provides that we could
> > use (in conjunction with a SQLite backend (currently there is none)) to
> > make a fast/light unit test for base ? it would be fantastic to have
> > anything in this space at all really ...
> 
> FWIW, on ubuntu-natty, I installed some packages; I think
> the relevant ones are sqlite3 and libsqliteodbc.
> LibreOffice, both Ubuntu package 1:3.3.3-1ubuntu2 and local
> build of commit id 0e1de41, is happy to work with an ODBC
> connection.

Hello Terry.

well, kind of - there is a number of features that do not work with ODBC
- so I'm just curious what the reason for using SQLite and ODBC would be
for a test bed?

Thanks

//drew

___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[Terrence Enger]

On Wed, 2011-10-05 at 11:00 +0100, Michael Meeks wrote:
> 
>   Ho hum ;-) is there an ODBC driver that sqlite provides that we could
> use (in conjunction with a SQLite backend (currently there is none)) to
> make a fast/light unit test for base ? it would be fantastic to have
> anything in this space at all really ...

FWIW, on ubuntu-natty, I installed some packages; I think
the relevant ones are sqlite3 and libsqliteodbc.
LibreOffice, both Ubuntu package 1:3.3.3-1ubuntu2 and local
build of commit id 0e1de41, is happy to work with an ODBC
connection.

Terry.


___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[Bernhard M. Wiedemann]

Michael Meeks wrote:

On Wed, 2011-10-05 at 10:33 +0100, Caolán McNamara wrote:

Locally installed databases raise questions, and the only
answers that come to mind boil down to a poor man's
substitute for an ODBC data source plus a driver manager
plus who-knows-what.  Is there anything here worth thinking
about?


I've wondered about e.g. using virtual box appliance's. Install Linux
under vbox, stick pre-installed postgres, mysql whatever else is
possible, onto it with pre configured databases of test data.


Ah ! good point, in which case we should use openqa:

http://openqa.opensuse.org/

The basic idea is to use a KVM virtual machine, and to do damage / OCR
on the screen, to allow keyboard input and validation of the results.

We currently use it to test our bare-metal openSUSE install flow, and
to provide time-lapse videos of that - it's rather sexy really ;-)
checkout:

http://openqa.opensuse.org/opensuse/video/openSUSE-DVD-x86_64-Build0319-gnome.ogv

All done headless, with no manual input anywhere near, and (IIRC) the
ability to suspend a test just where it failed, to allow it to be
tweaked and continued later ;-)


nearly:
while the test is running, I can interact with VNC
and recent disk images are kept after tests finished, allowing to 
re-start it and try things manually (because most of the time, you look 
at results of finished tests).




Bernhard would be the best guy to speak to wrt. more details and/or how
to get it setup. Bernhard - would we even have some spare hardware
cycles to host such tests ?


Yes, there are some spare cycles. If long tests need to be added, I 
could make it schedule them only daily like the extensive mozilla tests.



I had met Mechtilde at FOSDEM this year. She told me about a huge 
OpenOffice/LibreOffice testsuite (libreoffice-testtool) that runs hours 
or days. So far I had not come to setting it up on openQA.

And I don't know how reliable it is in terms of false positives.

If you have some script that can setup and run a test, I can integrate 
it easily.

E.g. the openQA-specific glue for mozmill tests is 70 lines total:
http://gitorious.org/os-autoinst/os-autoinst/blobs/master/consoletest.d/600_mozmill_setup.pm
http://gitorious.org/os-autoinst/os-autoinst/blobs/master/x11test.d/405_mozmill_run.pm

Ciao
Bernhard M.
___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[Terrence Enger]

On Wed, 2011-10-05 at 11:00 +0100, Michael Meeks wrote:
> Hi Terrence,
> 
> On Tue, 2011-10-04 at 14:50 -0400, Terrence Enger wrote:
> 
>   For automated testing the developer's list is prolly the better place
> to post.

Thank you.  I'll do that when I start a new thread.

> 
>   Quite :-) we'd strongly prefer a unit-test-like test harness such as we
> have in sc/qa/unit/ (and CppUnit makefiles in the toplevel).

That's good.  The cppunit infrastructure is the one I feel
most ready to start looking at.


At risk of getting ahead of myself, let me ask ...

Does this imply that a failed test breaks the build?  It is
of course a good thing that tests should be executed
frequently, but there are many things outside of LibreOffice
itself that can go wrong; indeed, the database can be half a
world away.

Consider a local database for which the required server job
is not running.  Is the answer the same?

> 
>   Ho hum ;-) is there an ODBC driver that sqlite provides that we could
> use (in conjunction with a SQLite backend (currently there is none)) to
> make a fast/light unit test for base ? it would be fantastic to have
> anything in this space at all really ...

A bunch of Google hits keep leading back to
.  The licence is "BSD-type",
whatever that means, and it has been tested on WinNT/2000
and Linux.

"Documentation/How Tos/Using SQLite With OpenOffice.org"

references that driver, and the page has lots of detail; at
first glance, nothing there contradicts what I think I
already know.  

This is encouraging.  I guess trying this might be the most
useful thing I can do right now.  Right?


That same page mentions an experimental SDBC driver for
sqlite.  Perhaps somebody with more imagination than I can
find a benefit in having two routes to the same database.


For what little it is worth, I have seen a reference to an
ODBC driver for HSQLDB, but (a) it was called "exprimental",
and (b) I have not found where the driver itself is
available.  Of course, we *know* that HSQLDB is installed,
and that makes a big difference.


Terry.


___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[Michael Meeks]

On Wed, 2011-10-05 at 10:33 +0100, Caolán McNamara wrote:
> > Locally installed databases raise questions, and the only
> > answers that come to mind boil down to a poor man's
> > substitute for an ODBC data source plus a driver manager
> > plus who-knows-what.  Is there anything here worth thinking
> > about?
> 
> I've wondered about e.g. using virtual box appliance's. Install Linux
> under vbox, stick pre-installed postgres, mysql whatever else is
> possible, onto it with pre configured databases of test data.

Ah ! good point, in which case we should use openqa:

http://openqa.opensuse.org/

The basic idea is to use a KVM virtual machine, and to do damage / OCR
on the screen, to allow keyboard input and validation of the results.

We currently use it to test our bare-metal openSUSE install flow, and
to provide time-lapse videos of that - it's rather sexy really ;-)
checkout:

http://openqa.opensuse.org/opensuse/video/openSUSE-DVD-x86_64-Build0319-gnome.ogv

All done headless, with no manual input anywhere near, and (IIRC) the
ability to suspend a test just where it failed, to allow it to be
tweaked and continued later ;-)

Bernhard would be the best guy to speak to wrt. more details and/or how
to get it setup. Bernhard - would we even have some spare hardware
cycles to host such tests ?

Thanks,

Michael.

-- 
michael.me...@suse.com  <><, Pseudo Engineer, itinerant idiot

___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[Michael Meeks]

Hi Terrence,

On Tue, 2011-10-04 at 14:50 -0400, Terrence Enger wrote:
> I am wondering about the possibility of automated testing of
> Base, particularly ODBC functionality.

For automated testing the developer's list is prolly the better place
to post.

> The first question here is how to attach LibreOffice to a
> database for the tests.

Quite :-) we'd strongly prefer a unit-test-like test harness such as we
have in sc/qa/unit/ (and CppUnit makefiles in the toplevel).

> Locally installed databases raise questions, and the only
> answers that come to mind boil down to a poor man's
> substitute for an ODBC data source plus a driver manager
> plus who-knows-what.  Is there anything here worth thinking
> about?

Ho hum ;-) is there an ODBC driver that sqlite provides that we could
use (in conjunction with a SQLite backend (currently there is none)) to
make a fast/light unit test for base ? it would be fantastic to have
anything in this space at all really ...

Thanks,

Michael.

-- 
michael.me...@suse.com  <><, Pseudo Engineer, itinerant idiot

___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

Re: [Libreoffice-qa] automated testing of Base

[Caolán McNamara]

On Tue, 2011-10-04 at 14:50 -0400, Terrence Enger wrote:
> I am wondering about the possibility of automated testing of
> Base, particularly ODBC functionality.
> 
> The first question here is how to attach LibreOffice to a
> database for the tests.

...

> Locally installed databases raise questions, and the only
> answers that come to mind boil down to a poor man's
> substitute for an ODBC data source plus a driver manager
> plus who-knows-what.  Is there anything here worth thinking
> about?

I've wondered about e.g. using virtual box appliance's. Install Linux
under vbox, stick pre-installed postgres, mysql whatever else is
possible, onto it with pre configured databases of test data.

C.

___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/

[Libreoffice-qa] automated testing of Base

[Terrence Enger]

I am wondering about the possibility of automated testing of
Base, particularly ODBC functionality.

The first question here is how to attach LibreOffice to a
database for the tests.

For ODBC, I can imagine setting up data sources with magic
names, something like LO_test_* or LO_autotest_* or
something.  The data source can encapsulate identificatino
of the database and credentials for access to the database,
so the credentials need not be published.  The existence of
the data source says to run the available tests.  (This does
not allow for tests applicable only to particular data
sources.)  Does this make sense, at least to the extent of
being worth thinking about?

Locally installed databases raise questions, and the only
answers that come to mind boil down to a poor man's
substitute for an ODBC data source plus a driver manager
plus who-knows-what.  Is there anything here worth thinking
about?

Thank you for your attention and advice.
Terry.


___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/