Re: Bug#876901: QFINDTESTDATA uses __FILE__

2017-11-15 Thread Pino Toscano
In data mercoledì 15 novembre 2017 20:14:00 CET, Ximin Luo ha scritto:
> Pino Toscano:
> > Loose meanings do not imply neither the other way around, that you are
> > free to break because people cannot do anything with it.  Also,
> > considering the very same behaviour (short of the different wording in
> > documentations) that is established for decades, this is a "de facto"
> > standard.
> > 
> 
> It's not, Microsoft doesn't do it. Last time I checked, QT is supposed
> to work on Windows.

Yup, and the fact that they used __FILE__ means the approach works also
on Windows.

> > [..]
> > 
> >> https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html
> >> "__FILE__ and __LINE__ are useful in generating an error message [..]"
> >>
> >> https://msdn.microsoft.com/en-us/library/027c4t2s.aspx
> >> "Without /FC, the diagnostic text would look similar to this diagnostic 
> >> text:"
> >>
> >> The only examples I can find in compiler documentation of __FILE__
> >> usage, is for error messages.
> >> 
> >> [..]
> >> 
> >> http://c0x.coding-guidelines.com/6.10.8.html The presumed name of the
> >> current source file [..]
> >> https://software.intel.com/en-us/node/524489 Defined as a character
> >> string literal containing the name of the source file.
> >> https://www.ibm.com/support/knowledgecenter/SSAE4W_9.1.1/com.ibm.etools.iseries.langref.doc/ilcrefer13.htm
> >> A string literal representing the name of the file being compiled.
> >>
> >> You can even find threads online confirming __FILE__ is
> >> implementation-specific and not required by any standard to be a full
> >> path. You can even find threads of people complaining about the fact
> >> that __FILE__ sometimes has a full path, sometimes not - e.g.
> >> depending on what options you pass to the Microsoft compiler.
> > 
> > The texts above clearly said that __FILE__ contains the path passed to
> > the compiler, which is very different from saying it is purely
> > "implementation-defined".  [..]
> > 
> 
> That's false, you're misleading casual readers when you say this.
> *None* of them define it using the word "path", they all use the word
> "name" - except the GCC docs which say:

Now it is you that plays with words.  Really, you see "file name" used
as both path and without path.

> "path by which the preprocessor opened the file, not the short name
> specified in ‘#include’ or as the input file name argument."
> 
> Let me highlight that for you: "not [..] the input file name argument".

Let me highlight this for you:
  "path by which the preprocessor opened the file"
which implies it is an *existing* *path*.

-- 
Pino Toscano

signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Bug#876901: QFINDTESTDATA uses __FILE__

2017-11-15 Thread Pino Toscano
In data mercoledì 15 novembre 2017 19:29:00 CET, Ximin Luo ha scritto:
> Pino Toscano:
> > [..]
> > 
> >> In summary: in no document or standard, does it guarantee or imply
> >> that __FILE__ can be taken to represent a real filesystem path.
> >> Applications relying on this behaviour are broken and should not be
> >> upset when things don't work. As documented in multiple places,
> >> __FILE__ only has a very loose meaning, my patch fits within this
> >> loose meaning, and it is intended mostly for error messages.
> > 
> > ... this is your own interpretation.  Even if __FILE__ has a loose
> > meaning, you are still changing that small bit of meaning it has,
> > without even thinking twice.  Also, where it is written that __FILE__
> > is "only for error messages"?
> > 
> 
> The loose meaning, indicates that programs should not rely on it for
> other meanings, and should be open to fixing themselves if
> implementations change their specifics. This is a general principle
> that applies to all text in all standards, it is not "my own
> interpretation".

Loose meanings do not imply neither the other way around, that you are
free to break because people cannot do anything with it.  Also,
considering the very same behaviour (short of the different wording in
documentations) that is established for decades, this is a "de facto"
standard.

> And stop twisting my words. You literally just misquoted me to
> advance your own point. I said "intended mostly for error messages":

Point taken, but my argument remain: where it written it is "mostly for
error messages"?

> https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html
> "__FILE__ and __LINE__ are useful in generating an error message [..]"
> 
> https://msdn.microsoft.com/en-us/library/027c4t2s.aspx
> "Without /FC, the diagnostic text would look similar to this diagnostic text:"
> 
> The only examples I can find in compiler documentation of __FILE__
> usage, is for error messages.

These are just very short documentation snippets; these documentations
surely cannot cover all the cases, so the most prominent one is shown.

> There are no examples of any other usage. That strongly suggests it
> was originally intended for error messages and never intended to
> support file lookups. That's not the same thing as saying "it's
> intended only for error messages", but it does strongly imply that
> you shouldn't be surprised if your file lookups break with minor
> changes like my patch,

Err no: the fact that something is not documented does not make it
automatically bad.  By this logic, __FILE__ would be the very last
problems in sources of modern software.

> and it certainly gives you no right to go off on a rant about me
> "breaking" stuff.

My rant replies to another rant about __FILE__... done by you.

> http://c0x.coding-guidelines.com/6.10.8.html The presumed name of the
> current source file [..]
> https://software.intel.com/en-us/node/524489 Defined as a character
> string literal containing the name of the source file.
> https://www.ibm.com/support/knowledgecenter/SSAE4W_9.1.1/com.ibm.etools.iseries.langref.doc/ilcrefer13.htm
> A string literal representing the name of the file being compiled.
> 
> You can even find threads online confirming __FILE__ is
> implementation-specific and not required by any standard to be a full
> path. You can even find threads of people complaining about the fact
> that __FILE__ sometimes has a full path, sometimes not - e.g.
> depending on what options you pass to the Microsoft compiler.

The texts above clearly said that __FILE__ contains the path passed to
the compiler, which is very different from saying it is purely
"implementation-defined".  I never claimed that __FILE__ is a full path
in all the cases.  Here you are misquoting what I said in previous
emails.

> You need to get your facts straight ...
> [...]

Please stop ad-hominem attacks.
Please stop ignoring other people's opinion.
And yes, please get out of your own echo chamber.

> And make some concrete proposals.

I already did, multiple times; alas, you discarded them a priori, since
they don't fit your own solution.

-- 
Pino Toscano

signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Bug#876901: QFINDTESTDATA uses __FILE__

2017-11-15 Thread Pino Toscano
In data mercoledì 15 novembre 2017 10:57:00 CET, Ximin Luo ha scritto:
> Pino Toscano:
> > In data martedì 14 novembre 2017 11:14:00 CET, Ximin Luo ha scritto:
> >> You're using __FILE__ inappropriately, none of the documentation
> >> guarantees or implies that you can access __FILE__ as a real
> >> filesystem path. "Surely for relative paths" is your justification
> >> for your own broken behaviour.
> > 
> > Again, this is your own echo chamber: "__FILE__ is broken, everybody
> > using it is broken no matter what".
> > 
> 
> It's not my "own echo chamber", I pointed you to lots of docs that
> confirm your usage of __FILE__ is not appropriate here.

I don't see any of these documentations.  All you did was taking GCC
and MSVC documentations and declare they mean nothing.

> Adrian Bunk also mentioned the C89 to me on IRC yesterday. Again, the
> wording gives no guarantee that __FILE__ should represent a real
> filesystem path.

The default is to be what is passed to the compiler as argument, which
*is* a filesystem path.  If the source uses #line, then that is
invalidated, but this is another situation.

> If my previous words were a bit terse I am sorry for that, but I don't
> appreciate comments like "Any of your solutions get a big, fat, and
> giant nope" after trying to explain the problem and present you with
> no less than 4 alternative simple unintrusive solutions.

My comment comes after you keep ignoring whatever else is said to you.
I explained your solution changes a standard behaviour (that, like it
or not, it *is* that, and the fact that so far it was used for decades
like that means it is a portable and standard behaviour).

You (generic "you") cannot pretend to have discussion, when you
basically decided a priori (although in a very arguable way) what is
"right" and what is "wrong", and thus discarding any other opinion
which does not fit into your criterias.

Just like Lisandro, I'm all for reproducibility, but this does not mean
standard things must be broken "just because".  What you want to change
is:
a) standard (standard C, and standard compiler behaviour)
b) so widely used
so trying to find a solution
a) without looking around at least a good majority of the use cases
b) flagging all the usages as "invalid"
it does not work.  And no, it simply does not.  It hardly is a
discussion at all, rather an echo chamber for your ideas.

-- 
Pino Toscano

signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Bug#876901: QFINDTESTDATA uses __FILE__

2017-11-15 Thread Pino Toscano
In data mercoledì 15 novembre 2017 12:09:00 CET, Ximin Luo ha scritto:
> Lisandro Damián Nicanor Pérez Meyer:
> > [..]
> > 
> > Xi: you also mentioned that having to file hundreds of patchs seems 
> > impossible. Well, it seems so, but it is actually not that necessary. 
> > Please 
> > allow me to explain the idea.
> > 
> 
> Thanks for being less inflammatory than Pino.

Flame that you started, mind you.

> I agree that eventually all projects should move away from using
> __FILE__.

Again, why?  Just because __FILE__ gives results that break
reproducibility does not mean that it is wrong/bad -- the topic of this
bug is exactly one of the cases where it *does* make sense to use
__FILE__.

> This BUILD_PATH_PREFIX_MAP variable is only a stepping stone to that,
> just like SOURCE_DATE_EPOCH was a stepping stone to less projects
> using __DATE__ and __TIME__. It allows people to see the obvious
> benefit of a reproducible build, by actually achieving a large amount
> of reproductions, today. We did not need to file mass bug reports for
> __DATE__ and __TIME__ with SOURCE_DATE_EPOCH.

Again: comparing with __DATE__ and __TIME__ makes no sense, since they
were used in 99+% of the cases only for display purposes.  OTOH, this
thread shows __FILE__ it is not used (only) for that.

> You're implying that QT's use of __FILE__ for tests, as being
> discussed in this thread, is a "good use case"

Exactly, it *is*.

> That's not how I see it. As I pointed out various times already, the
> use of __FILE__ here is *also not appropriate*. You can consider
> these emails from me now, as "documenting/blogging" about "bad use
> cases".

OK, sure, we agree to disagree.  But ...

> In summary: in no document or standard, does it guarantee or imply
> that __FILE__ can be taken to represent a real filesystem path.
> Applications relying on this behaviour are broken and should not be
> upset when things don't work. As documented in multiple places,
> __FILE__ only has a very loose meaning, my patch fits within this
> loose meaning, and it is intended mostly for error messages.

... this is your own interpretation.  Even if __FILE__ has a loose
meaning, you are still changing that small bit of meaning it has,
without even thinking twice.  Also, where it is written that __FILE__
is "only for error messages"?

> The BUILD_PATH_PREFIX_MAP variable works around a lot of "bad use
> cases", but it doesn't cover this particular case. Some minor tweaks
> are needed to cover it, and I suggested them already.

So this is not a solution, but low quality workaround.

> But you guys continue to push the position "we're doing nothing
> wrong, it's other people doing stuff wrong, go talk to them instead".

Let's flip this: you continue to push the position "everybody using
__FILE__ is wrong, I won't listen to anybody".

> I'm sorry but it's not a convincing position for me to agree with.

Neither is yours, sorry.

> At the end of the day, all of these cases, including yours, ought to
> be fixed to not use __FILE__ at all.

Again, we agree to disagree.

-- 
Pino Toscano

signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Bug#876901: QFINDTESTDATA uses __FILE__

2017-11-14 Thread Pino Toscano
In data martedì 14 novembre 2017 13:47:56 CET, Holger Levsen ha scritto:
> On Tue, Nov 14, 2017 at 07:22:07AM +0100, Pino Toscano wrote:
> > > There are quite a lot of packages that use __FILE__ so forgive me for
> > > not checking every single use-case of it.
> > 
> > This.  When something is used so widely, then changing its behaviour
> > blindly is simply a no-no.
> 
> I tend to agree.
> 
> > > This will require us to patch hundreds of packages, and isn't
> > > realistic.
> > 
> > If "hundreds of packages" misuse __FILE__, then simply *fix them*.
> > Sure, it will require time, energy, etc, but I do not see other ways
> > around that without breaking standard behaviours.
> 
> also agreed.
> 
> > > Please, you try sending hundreds of patches, then I will take your
> > > "solution" more seriously.
> > My *solution* (without quotes) is more realistic than your blind
> > breakage.
> 
> Pino, could you be so kind and repeat your proposed solution here again,
> for the sake of clarity?!

It is basically what I wrote in the quoted part above (not this one, the
last where you replied "also agreed".  I will repeat it again, as asked:

| No, the solution is:
| a) *not* break what __FILE__ means
| b) remove the misuses of __FILE__ in packages (not the case of
| QFINDTESTDATA)

As also Lisandro explained a bit more in another email, this solution
takes a bit more time and effort, but it pays off (especially if this
is done directly upstream) as all the improper usages will be removed,
with little to no effort specific for Debian.

-- 
Pino Toscano

signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Bug#876901: QFINDTESTDATA uses __FILE__

2017-11-14 Thread Pino Toscano
In data martedì 14 novembre 2017 11:14:00 CET, Ximin Luo ha scritto:
> You're using __FILE__ inappropriately, none of the documentation
> guarantees or implies that you can access __FILE__ as a real
> filesystem path. "Surely for relative paths" is your justification
> for your own broken behaviour.

Again, this is your own echo chamber: "__FILE__ is broken, everybody
using it is broken no matter what".

> You can either accept my one line patch suggestion, or fix it some
> other way.

I am not interested in working around broken changes introduced blindly
for very doubtful reasons.

> I'm not going to change the GCC patch, it does nothing wrong.

Let me add also another POV to this approach: do you really expect
Debian to carry this important diversion for GCC upstream?  I really
doubt GCC will accept this.

-- 
Pino Toscano

signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Bug#876901: QFINDTESTDATA uses __FILE__

2017-11-13 Thread Pino Toscano
In data lunedì 13 novembre 2017 23:44:00 CET, Ximin Luo ha scritto:
> Pino Toscano:
> > [..]
> > 
> > This is not an annoyance, it is the crux of the problem!  __FILE__ is
> > something standard, with a very well defined behaviour, upon which
> > people rely on: you cannot trash it from one day to another like this,
> > saying "too bad" to all its users, even those using it appropriately.
> > It does not simply work.
> > 
> >> but the point is that the previous use does not generalise well.
> > 
> > To be honest: I already said multiple times that abuses of __FILE__
> > *must* be fixed.  However, there are valid use cases, and in this case
> > the only generalization is done by you, who declared __FILE__ as
> > "absolutely BAD".
> > 
> 
> It's an objective fact that tests using __FILE__ are unable to be
> generalised to be run outside of the build.

Yes, and this is already acknoledged.

> As Lisandro said in the other comment, "We do *not* want to run the
> tests outside the build."

This refers to Qt tests, and those using QFINDTESTDATA in particular,
for reasons I already explained in another reply.

> >> It's not a sledgehammer solution, it's tweakable and we can try to
> >> use its flexibility to un-break your tests.
> > 
> > No, it *is* a sledgehammer solution, since you are not even checking
> > what __FILE__ is used for, and thus blindly changing it without any
> > pre-research on its effect.  Again, this is not the correct approach
> > to fix the issue at hand.
> > 
> 
> There are quite a lot of packages that use __FILE__ so forgive me for
> not checking every single use-case of it.

This.  When something is used so widely, then changing its behaviour
blindly is simply a no-no.

> My patch fixes 1800 packages, and the amount of extra FTBFS it caused
> (such as this) is a blip in the stats, and a very minor one at that -
> failing tests, because a file was not found.

Using statistics to justify a bad behaviour change like this is not a
good way to go.  This "blip is the stats" ought to show you the
behaviour change broke *valid* use cases; also, this does not account
how many packages build, but will fail to work at runtime because of
this.

> (Yes it is RC, but technically it is minor and very easy to fix, but
> you're denying us the fix because you seem to feel that the standards
> are on your side.

To be honest, the only denial I see is your refusal to acknoledge your
behaviour change is a bad regression, and a no-go.

> Well let's take a look at the standards:
> [...]

Even with different wordings, both describe basically the same
behaviour.  And yes, none of them says that __FILE__ is a full path,
but surely for relative paths the combination of $srcdir + __FILE__
will give me the path to the source.

> You're forcing us to choose between fixing 1800 packages minus your
> QT packages, or fixing nothing.

And again, your position here is "black or white": either "we change
__FILE__ to our liking", or "we fix nothing".  There is the middle
ground of fixing misuses, which are *already* bugs floating in
packages, regardless of reproducible builds.

> I ask you to be slightly less absolutist.

I am already.  That is why I suggested to fix only what is broken in
packages.

> The solution I proposed is 1-line and should cause basically no
> disruption to anything else.

By your own admission above, there are so many usages of __FILE__ that
you did not check all of them.  OK, but in this case, how do you claim
your solution will not break anything else?  Just because something
else builds, then it does not imply it will run correctly at runtime.
Even beside this, this case is one *valid* use case of __FILE__ that
breaks your assumptions.

> >> If this isn't suitable to you, how else do you suppose we should try
> >> to make builds independent of (i.e. not embed) the build path?
> >> Suggestions are welcome.
> > 
> > I already wrote it in my email, and I repeat it again:
> > 
> > | No, the solution is:
> > | a) *not* break what __FILE__ means
> > | b) remove the misuses of __FILE__ in packages (not the case of
> > |QFINDTESTDATA)
> > 
> 
> This will require us to patch hundreds of packages, and isn't
> realistic.

If "hundreds of packages" misuse __FILE__, then simply *fix them*.
Sure, it will require time, energy, etc, but I do not see other ways
around that without breaking standard behaviours.

It is a bit like gets(): it is old, buggy, and obsolete, so sources
using it are buggy already.  Then, the proper way to fix them is to
switch them to use something else, not to make gets() a no-op in libc.

> Please, 

Re: Bug#876901: QFINDTESTDATA uses __FILE__

2017-11-13 Thread Pino Toscano
In data lunedì 13 novembre 2017 20:20:00 CET, Ximin Luo ha scritto:
> Pino Toscano:
> > [..]
> > 
> > A better approach here is to work on removing the invalid & abusing
> > usages of __FILE__ from packages, just like it was done for __DATE__.
> > 
> 
> We in fact did not do the latter because it was easier to implement
> SOURCE_DATE_EPOCH to fix the expansion of __DATE__, rather than patch
> 400 packages.

To be fair, it should be noted that various project stopped using it
altogether, as result of "reproducible builds" patches.

Also, __DATE__ expnds to a C string like "Nov 13 2017":
https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html
this means that you cannot do something like "__DATE__ + 3" you
mentioned earlier, but that requires a bit more work to parse it...
and at that point using gmtime() & friends is then way easier.

Indeed, all the usages I have seen so far are just simple display of
the build time, with messages like
  "Foo App vX.Y built at " __DATE__ " " __TIME__
Hence, changing what __DATE__ returns is almost a no-op for the 99+%
of sources using it, since there is basically almost no other usage for
it.

OTOH, as already mentioned in other replies, __FILE__ is actually
*used* for stuff, it is not a mere display string.

-- 
Pino Toscano

signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Bug#876901: QFINDTESTDATA uses __FILE__

2017-11-13 Thread Pino Toscano
In data lunedì 13 novembre 2017 20:03:00 CET, Ximin Luo ha scritto:
> Pino Toscano:
> > [..]
> > 
> > No, the solution is:
> > a) *not* break what __FILE__ means
> > b) remove the misuses of __FILE__ in packages (not the case of
> >QFINDTESTDATA)
> > 
> >> I am not "blaming the user", but pointing to the fact that __FILE__
> >> is being used in a surprising way here, which is not good for
> >> reproducible builds.
> > 
> > What I see it is happening here is: you (= people working on
> > reproducible builds) see __FILE__, and the problems that arise from its
> > abuse; to overcome this issue, you use the sledgehammer solution,
> > basically changing what __FILE__ means, and thus breaking even valid
> > use cases.  Sorry, but I do not see how this is useful.
> > 
> > A better approach here is to work on removing the invalid & abusing
> > usages of __FILE__ from packages, just like it was done for __DATE__.
> > 
> >> An analogy would be to write your program to execute something at
> >> time "__DATE__ + 30 seconds". This is obviously ridiculous, but works
> >> "by accident" if done inside a test case.
> > 
> > This is clearly a misuse, and thus it must be fixed.  OTOH, the
> > comparison with __FILE__ is not appropriate.
> > 
> Why's it not appropriate?

Because using __DATE__ as timing for a test is extremely fragile, and
it has nothing to do with reproducible builds.  It is just wrong, and
as such __DATE__ is used wrongly.

OTOH, as I already explained, at least what QFINDTESTDATA does with
__FILE__ is perfectly valid and acceptable, since it is about stuff
run during test suites at build time only.  Again, see:
http://doc.qt.io/qt-5/qtest.html#QFINDTESTDATA

> If you ever want to write tests to be runnable outside the build,
> e.g. with autopkgtest, then you're going to have to not use __FILE__
> anyway.

Sure, and it is exactly what I said in point (b) of the solution I
proposed above.  

> (Assuming you install the tests somewhere, rather than
> running the whole build again.)

This will not work with tests using QFINDTESTDATA anyway, as it works
with either the source or build directory of the source using it, or
with the Qt installation prefix.  (Hint: none of them is useful for an
installed test.)

> I can understand that breaking something that used to work is
> annoying,

This is not an annoyance, it is the crux of the problem!  __FILE__ is
something standard, with a very well defined behaviour, upon which
people rely on: you cannot trash it from one day to another like this,
saying "too bad" to all its users, even those using it appropriately.
It does not simply work.

> but the point is that the previous use does not generalise well.

To be honest: I already said multiple times that abuses of __FILE__
*must* be fixed.  However, there are valid use cases, and in this case
the only generalization is done by you, who declared __FILE__ as
"absolutely BAD".

> It's not a sledgehammer solution, it's tweakable and we can try to
> use its flexibility to un-break your tests.

No, it *is* a sledgehammer solution, since you are not even checking
what __FILE__ is used for, and thus blindly changing it without any
pre-research on its effect.  Again, this is not the correct approach
to fix the issue at hand.

> So, what do you think of this alternate solution, that I suggested:

Any of your solutions get a big, fat, and giant nope.

> If this isn't suitable to you, how else do you suppose we should try
> to make builds independent of (i.e. not embed) the build path?
> Suggestions are welcome.

I already wrote it in my email, and I repeat it again:

| No, the solution is:
| a) *not* break what __FILE__ means
| b) remove the misuses of __FILE__ in packages (not the case of
|QFINDTESTDATA)

-- 
Pino Toscano

signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Bug#876901: QFINDTESTDATA uses __FILE__

2017-11-13 Thread Pino Toscano
In data lunedì 13 novembre 2017 17:35:00 CET, Ximin Luo ha scritto:
> Adrian Bunk:
> > Control: reassign -1 qtbase5-dev
> > Control: reassign 876917 qtbase5-dev
> > Control: reassign 876933 qtbase5-dev
> > Control: forcemerge -1 876917 876933
> > Control: retitle -1 QFINDTESTDATA uses __FILE__
> > Control: severity -1 normal
> > Control: affects -1 src:karchive src:ki18n src:kcodecs src:kparts
> > thanks
> > 
> > The problem is the following implementation in
> > /usr/include/x86_64-linux-gnu/qt5/QtTest/qtestcase.h:
> > # define QFINDTESTDATA(basepath)\
> > QTest::qFindTestData(basepath, __FILE__, __LINE__)
> > 
> > With the patched gcc in the unstable reproducible builds setting
> > __FILE__ to something other value, this does no longer find the
> > test data.
> > 
> > I do not really see any reason for blaming the users here for using
> > a documented public Qt API for accesssing test data in the source
> > directory:
> >   http://doc.qt.io/qt-5/qtest.html#QFINDTESTDATA
> > 
> > I've added reproducible-builds@lists.alioth.debian.org to Cc for giving 
> > input what a reproducible and portable implementation might be for Qt.
> > 
> 
> Our patched dpkg tells GCC to map $PWD to "$srcpkg-$version" when
> expanding __FILE__. That is the source of the problem, because this
> path doesn't exist at test-time.

Exactly, this is the source of the problem.  OTOH, the problem was
created by changing __FILE__: it has a well-defined meaning:
https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macros.html
changing this behaviour is only going to create problems, because
people rely on such well-defined (and standard) behaviours.

QFINDTESTDATA is a macro used mostly (if not only) in unit tests, so
the binaries built with it are usually not installed.  QFINDTESTDATA
uses __FILE__ as one of the methods to locate files in the source
directory of the file being built.  AFAICS, this usage is *valid*, so
the reproducible changes done to __FILE__ are only a regression.

> The problem stems from the fact that we assume __FILE__ represents a
> build-time path and not a run-time path, so we rewrite it
> indiscriminately with BUILD_PATH_PREFIX_MAP.

Exactly -- but in the case of QFINDTESTDATA this is wanted.

> This assumption is broken in the specific case where you have some
> source code that uses __FILE__, that are specifically only meant to
> be run at build-time, so that they are in fact run-time paths (that
> are also build-time paths). The assumption is fine in all other cases.

So this is fine for QFINDTESTDATA.

> Therefore, the only solution to fix this problem, that also preserves
> reproducible builds, is to make those tests *not use __FILE__*. Or
> option (2), which makes the non-existent rewritten paths, into an
> actually-existing build-time path.

No, the solution is:
a) *not* break what __FILE__ means
b) remove the misuses of __FILE__ in packages (not the case of
   QFINDTESTDATA)

> I am not "blaming the user", but pointing to the fact that __FILE__
> is being used in a surprising way here, which is not good for
> reproducible builds.

What I see it is happening here is: you (= people working on
reproducible builds) see __FILE__, and the problems that arise from its
abuse; to overcome this issue, you use the sledgehammer solution,
basically changing what __FILE__ means, and thus breaking even valid
use cases.  Sorry, but I do not see how this is useful.

A better approach here is to work on removing the invalid & abusing
usages of __FILE__ from packages, just like it was done for __DATE__.

> An analogy would be to write your program to execute something at
> time "__DATE__ + 30 seconds". This is obviously ridiculous, but works
> "by accident" if done inside a test case.

This is clearly a misuse, and thus it must be fixed.  OTOH, the
comparison with __FILE__ is not appropriate.

-- 
Pino Toscano

signature.asc
Description: This is a digitally signed message part.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds