Dear Laszló,
Thank you for your e-mail.
(Sorry my local system does not seem to have the correct font for the
character in your name, and so I am not sure if I am writing the name
correctly.
I hope it will show correctly on your end.)
My comments inline:
On 2019/11/01 23:44, László Böszörményi (GCS) wrote:
Hi Chiaki,
On Thu, Oct 31, 2019 at 9:09 PM ISHIKAWA,chiaki <ishik...@yk.rim.or.jp> wrote:
I vouch that there seems to be a serious issue in libsqlite3 3.30.1-1.
It's not that fatal like it may seem so.
I wish that is the case.
I came here because it seems that I have an issue with sqlite3 on my
linux installation.
The problem manifested as database operation failures during thunderbird
mail client regression testing (called mach xcpshell-tests|)
Is there a documented way to get and run it locally?
There is a series of documents.
However, it is such a long winding path, it may take a few days to set
up and run correctly.
I can't really suggest that you take the steps outlined below casually.
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Thunderbird_build
My summary is as follows.
First you have to download the source for C-C (comm-central) AND M-C
(mozilla-central.)
M-C should be installed somewhere with mozilla directory name.
C-C needs to be installed as comm immediately below mozilla directory.
Then you have to configure the development environment.
Setting environment variables and configuration file and set an
environment variable, MOZCONFIG,
that points to the config file.
You have to specify the MOZ_OBJDIR that stores the generated binary:
this object directory MUST be
outside the source tree.
Before building,
you have to download a series of debian package header files and
libraries (i.e., development library packages) that are used by
the thunderbird build process.
There are a lot actually under Debian that we need to install on top of
regular installation.
*THEN*
We run the compilation process using so-called |../mach build| command
in the C-C topmost directory (comm) immediately below M-C (mozill)a
directory.
This command will likely suggest that we need to install clang c++
compiler and rust compiler with many packages.
It prints out the commands how to perform these actions. So I follow it.
Once the compilation is successful, we can finally run
xpcshell test by
cd $MOZOBJ || exit 1
... mozilla/mach --log-no-times xpcshell-test
where mach command is in the mozilla source directory (M-C).
Build takes about 1 hour on my Ryzen1700 with 16GB memory.
(Actually, Debian GNU/Linux amd64 image that runs inside VirtualBox that
runs on top Windows10 Pro 64 bit)
If you really are interested, I can help.
There are all sorts of minor issues that pop up from time to time when
we use open source tools and libraries.
Usually error messages are clear, but sometimes python and other script
language errors pop up and they are difficult to analyze.
I looks someone on Ubuntu did not see it this month and mozilla's
compilation and testing farm machines do not seem to see it. They run
CentOs if I am not mistaken.
Which version of CentOS? Its latest release, versioned 8 has SQLite3
3.26.0 if I'm not mistaken.
Oh, I was wrong. The compilation/test farm machines have transitioned to
Ubuntu sometime before.
I did not know that. Maybe we can know what sqlite3 version is working
there.
I see the following version info of OS. It seems a very old OS package.
[task 2019-10-31T22:44:44.290Z] ++ NAME=Ubuntu
[task 2019-10-31T22:44:44.290Z] ++ VERSION='16.04.5 LTS (Xenial Xerus)'
[task 2019-10-31T22:44:44.290Z] ++ ID=ubuntu
[task 2019-10-31T22:44:44.290Z] ++ ID_LIKE=debian
[task 2019-10-31T22:44:44.290Z] ++ PRETTY_NAME='Ubuntu 16.04.5 LTS'
[task 2019-10-31T22:44:44.291Z] ++ VERSION_ID=16.04
from
https://taskcluster-artifacts.net/OtJtVGBiQkax0LeOVJAcbw/0/public/logs/live_backing.log
This is from a build/test job submission result:
https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=e1c9f00a40034361a9a3b6c4abdf0807e22c5d7e
Mozilla has a rather nice build/test farm setup.
https://bugzilla.mozilla.org/show_bug.cgi?id=1589779
So I thought maybe the current installation of libsqlite3 on my linux PC
was to blame.
Might be, but SQLite3 3.30.1 should be safe.
I am still trying to find out what changed between latest August and
beginning of October.
It can be
- my PC environment changes (including package updates), or
- even testing environment change (but unlikely).
Since someone under Ubuntu told me the test works there without an
issue, I think
local package version issue is more like it.
Well, it looks that is the case if I read the exchanges here correctly.
My libsqlite3 seems to be affected: the version is 3.30.1-1
(The problem was not observed in August. I am not sure if the upgrade of
libsqlite3 happened during then and now.)
What do people suggest that best course of action under the circumstances?
Downgrade (to which version and how) or install newer libsqlite3 (where)?
All uploaded SQLite3 Debian packages should be archived[1]. Try
downgrading it to 3.29.0-2 first and see if it helps.
I will do so.
My local testing with "--verbose" option of "xpcshell-test" reveals many
hither-to-unnoticed issues because
the error/warning messages were not printed by default when unit tests
in xpcshell-test did not fail.
(Only "--verbose" forces the dump of buffered stderr/stdout output. The
default on mozilla build farm is terse).
I have noticed that there are lots of orphaned errors/warnings that need
attention.
But then this DB errors interfered in my quest to figure out what are
the causes of these outstanding errors.
That is why I wanted to eliminate these DB errors.
All I can say is that the same tests that deal with database that worked
for years suddenly broke in the last several weeks. So my bug report
would not be that useful :-(
May you have a date when you first experienced it? As sqlite3 3.30.0
is in the Debian archives since the fifth of October, this may narrow
down the issue a bit.
Since |xpcshell-test| is long time running task, I don't run it locally
often.
Maybe I should have.
The local involving DB succeeded on Aug 25.
But it fails on Oct 14.
I wish I ran xpchsell-test weekly :-(
Your timeline makes very suspicious of sqlite3 3.30.0 now.
Regards,
Laszlo/GCS
[1] http://snapshot.debian.org/package/sqlite3/
Thank you for your comment.
I will keep your tips in mind especially the downgrade to 3.29 while I
am struggling :-)
to cope with other hither-to-unnoticed errors/warnings.
At this moment, I have excluded the DB test from xpcshell-test run
temporarily until I can figure out.
Thank you again, and happy hacking (in the old-fashioned good sense of
the word).
Best Regards,
Chiaki
