What if there's a one-byte change somewhere in a build script, or a
space added to a source file?

Don't use just SHA or MD5. There exists similarity hashing for that specific purpose.

I'm solving that question with SSDEEP and SPDX. Once a byte changes, the similarity between nearly identical files returns a value of 99% and I store a snapshot of hashes for a set of files inside an SPDX document.

I didn't found SPDX documents so easy to generate but they are really simple for humans to read. So I wrote a tool for this purpose, fairly simple. Here is a real example of what can go inside one SPDX:

-------------
## File

FileName: jfreechart-1.0.17-demo.jar
FileType: OTHER
FileChecksum: SHA1: 580d2b5c99c715b8cfee6eeb3825bee65f294558
FileChecksum: SHA256: 0ff0c95cbb4d288f541c82db7623a237e3a0f874dfcf9366c3cf5c98405e0043
FileChecksum: MD5: ae1dc5c6086ce487857ad7a6183cc725
FileChecksum: SSDEEP: 12288:8idVxJdeQwgrwt6V56ViGQ6o6LxiUR8LywpJWzlC38hK6AOtwp6vTsowx8Z:pdXJdgg2mkwSiURhIJWzlu8JXTsowxC
FileSize: 824 Kb (844423 bytes)
-------------

You can get the whole SPDX document from https://opensourceprojects.eu/p/triplecheck/code/23/tree/tool/run/products/jfreechart-1.0.17.tar.gz.spdx

SSDEEP is not perfect. There exists better but this is an established algorithm and fairly established since some years. The tool I've written for creating these snapshots is freeware, code licensed as EUPL and programmed in Java code.

With kind regards,
Nuno Brito

---
email: [email protected]
phone:  +49 615 146 03187

On 2014-03-08 12:00, [email protected] wrote:
Send legal mailing list submissions to
        [email protected]

To subscribe or unsubscribe via the World Wide Web, visit
        https://lists.gpl-violations.org/mailman/listinfo/legal
or, via email, send a message with subject or body 'help' to
        [email protected]

You can reach the person managing the list at
        [email protected]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of legal digest..."

Today's Topics:

   1. Re: Use of GNU GPL 2.0 code in Mikrotik RBXXX series and
      others (Kern Sibbald)
   2. Attitudes of violators based on company size (was Re: Use of
      GNU GPL   2.0 code in Mikrotik RBXXX series and others)
      (Bradley M. Kuhn)
   3. Re: Use of GNU GPL 2.0 code in Mikrotik RBXXX series and
      others (Bradley M. Kuhn)

Content-Transfer-Encoding: 7bit
From: Kern Sibbald <[email protected]>
Precedence: list
MIME-Version: 1.0
To: Arnt Karlsen <[email protected]>, [email protected]
References:
<[email protected]>     <[email protected]>      
<[email protected]>       <[email protected]>    <[email protected]>    
<[email protected]>    <[email protected]>    <[email protected]> <[email protected]>  
<[email protected]> <[email protected]>  <[email protected]>    <[email protected]> 
<[email protected]>
        <[email protected]>
In-Reply-To: <[email protected]>
Date: Fri, 07 Mar 2014 14:45:01 +0100
Message-ID: <[email protected]>
Content-Type: text/plain; charset=ISO-8859-1
Subject: Re: Use of GNU GPL 2.0 code in Mikrotik RBXXX series and others
Message: 1

On 03/07/2014 01:13 PM, Arnt Karlsen wrote:
On Fri, 07 Mar 2014 11:41:25 +0100, Kern wrote in message
<[email protected]>:

On 03/07/2014 01:32 AM, Arnt Karlsen wrote:
On Thu, 06 Mar 2014 13:03:04 +0100, Kern wrote in message
<[email protected]>:
Perhaps all this was obvious to you, but my main point is that from
the standpoint of an open source developer, what I am hearing on
this list about checksumming worries me that you are going in a
direction that could put more burden on developers not
just those who violate the GPL.
..having taken part in 11 years of Groklaw.net, I feel a wee bit of
short term agony on our part right now, will prevent another few
decades of frivolous litigation, once we have checksumming etc
frivolous lawsuit swatter tools in place in the courts.

..but we must make those tools available first.

..pity, you could have use this in your (secret?) lawsuit on Bareos.
Now, that is a very curious comment "secret?".  I didn't know there
was anything secret about this lawsuit.  It is documented on the
Bareos website
...where they mention how they were not properly served... ;o)
OK, now I understand your comment.
Yes, Bareos posted this comment on their website shortly after
Bacula Systems announced the lawsuit.  The facts are:
1. Bacula Systems doesn't do the formal notification,
    it is the court, and that takes time. So Bareos
    really should complain about the international judicial procedure.
2. Within a day or two Bareos hired a Swiss lawyer.
3. Within an hour of Bareos' contacting the Bacula Systems lawyer, he
    received a courtesy copy directly from the Bacula Systems'
    lawyer.
4. Bareos' lawyer with the case in hand then officially requested
    a formal notification.
5. Based on the fact that the Bareos Swiss lawyer has *everything*,
    the judge ruled that the notification was complete.
6. Bareos retains their comment on their website.


The lawsuit is about alleged theft of proprietary code and unfair
competition.  Bareos also violated the Bacula copyright license, held
by the FSFE and after working with the FSFE corrected the most
flagrant violations.  However, it is my belief that they are still
violating the copyright in two senses, one the commercial
(not putting the copyright where it should be), two violating
basic author's rights. Since I am no longer the copyright owner,
I can only act concerning part two: author's rights.  My own
personal view as an open source project manager is presented at:

http://blog.bacula.org

Warning, it is longwinded.  (this comment not made with any
negative intention).
..indeed ;o), but messing around with copyright notices is
a "Big Mistake", even without that thick Austrian accent.

..the expensive part is in your allegations of "unfair competition",
this will be hard to prove cheaply either way, "The SCO Group" also
tried that, and failed, but your alleged facts looks better founded,
and those are offtopic here, except as a litigation tactic.

Checksums would not be of any use in this case.
..read "checksums" as "tests the jurors can easily do themselves",
such easy test would be useful if we had them now.
Yes, I agree that would be useful, and I will not complain if
you succeed in doing it.

However, permit me to suggest an alternative strategy that
might prove useful.

First my assumptions that may not be valid or may be hard to do:
1. Your goal is to make sure the "vendor" doesn't have any
    additional functionality in his binary that is not present in the
    distributed source code.
2. You can require the "vendor" to release the binaries with
    debug symbols turned on or if they are stripped they must
    be done in a way that they can be re-integrated as with rpm debug
    packages.
3. You can require the *exact* build scripts that they used
to produce the binaries (i.e. which compiler, optimization options,...
4. Now at that point, one could (as a fairly big project) write a
program that
    breaks the binary up into subroutines and puts them into some human
   readable form (assembly language, or perhaps reconstructed C, C++).
   Call it the "reconstructed source".

Now you can do a fairly high level comparison of the two programs on a
subroutine by subroutine basis (object file by object file or whatever) to search and find any code that is included in the distributed binary that
is not in the distributed source.  Comparing the "reconstructed source"
from the vendor's binary and a binary that you build, could also then
show up any major discrepancies or "code that was left out of the
distributed source".

Best regards,
Kern

Best regards,
Kern

Maybe Bareos can use this in their defense and countersuit? ;o)
http://www.baculasystems.com/blog/bacula-systems-sa-files-lawsuit-against-bareos-gmbh-co-kg
http://www.bareos.org/en/news/items/lawsuite-between-bacula-systems-sa-and-bareos-gmbh-co-kg-copy.html





From: "Bradley M. Kuhn" <[email protected]>
Precedence: list
MIME-Version: 1.0
Cc: [email protected]
To: Neil Brown <[email protected]>
References: <[email protected]> <[email protected]>
        <[email protected]> <[email protected]>
        <[email protected]> <[email protected]>
        <[email protected]> <[email protected]>
        <[email protected]>
        <[email protected]>
In-Reply-To: <[email protected]>
Date: Fri, 7 Mar 2014 14:49:53 -0500
Message-ID: <[email protected]>
Content-Type: text/plain; charset=us-ascii
Subject: Attitudes of violators based on company size (was Re: Use of GNU GPL
        2.0 code in Mikrotik RBXXX series and others)
Message: 2

Neil Brown asked me:
Is it your perception that the majority of GPL violations arise from
"wealthy companies", or just that those which are most often brought to light, or perhaps those which are considered worth pursuing, are by wealthy
companies?

When I line up just about any company next to Conservancy or FSF, they are *all* wealthy compared to those orgs, so my perspective is admittedly skewed.

But, that said, the violators that give us the most trouble coming into
compliance are usually wealthy corporations who expend more resources on
skirting their responsibilities than on trying to comply.

In my experience, most violators that are small companies tend to be
regretful of their failure to comply and work quickly to come back into
compliance. Thus, they don't typically require long, drawn-out multi-year efforts (with outside legal counsel involved) to achieve GPL compliance.

By contrast, in my experience, larger, wealthier violators effectively have "money to burn" on fighting on every single compliance issue and seeing "what
they can get away with".

(There are exceptions to this, of course, the above is a generalization.)

However, what's worse is that these wealthy violators have created a
distributing cottage industry. There are now an entire group of well-known outside counsel who specialize (in part) in defending wealthy violators that
want to avoid complying with GPL.  An anecdote to exemplify that:

A few years ago, at OSCON, one of these lawyers came up to me, shook my hand, and thanked me for enforcing the GPL because it had been "so good for his business". I told him that he just made me want to puke, and I meant that literally: I felt sick to my stomach the rest of that day just thinking about what he'd said. Frankly and sadly, making a bunch of seedy lawyers rich has been an unfortunate side-effect of upholding the rights granted by the GPL.

That said, the alternative is to let the violators get away with it, which is
a much worse outcome.


I should finish by saying that there are at least a few wealthy companies
that have a good ethos and have chosen to do the right thing and are
wonderful actors on GPL compliance. I don't want to name names lest I leave any of them out, but some of their representatives even subscribe to this
mailing list and answer questions. :)

My view on that: it's really unfair to those few companies who spend so much money on getting compliance right that there are so many competitors who
would rather pay high priced outside counsel than just comply with the
license.

      -- bkuhn



From: "Bradley M. Kuhn" <[email protected]>
Precedence: list
MIME-Version: 1.0
To: [email protected]
References: <[email protected]>
        <[email protected]>
        <[email protected]>
        <[email protected]> <[email protected]>
        <[email protected]> <[email protected]>
        <[email protected]> <[email protected]>
        <[email protected]> <[email protected]>
        <[email protected]>
In-Reply-To: <[email protected]> (Jonathan Wilson's message of
        "Thu, 06 Mar 2014 22:18:37 +1000")
Date: Fri, 07 Mar 2014 16:29:16 -0500
Message-ID: <[email protected]>
Content-Type: text/plain; charset=us-ascii
Subject: Re: Use of GNU GPL 2.0 code in Mikrotik RBXXX series and others
Message: 3

Jonathan Wilson wrote at 07:18 (EST) on Thursday:

Why not simply have the makefiles for the program in question do a
shasum of the source, put that checksum into some source file and have
the file linked into the binaries in a way that can easily be found.

This is the basis of Ed's idea, but the problem becomes wasting a lot of
space.  A shasum of the entire source in aggregate, plus "scripts used
to control compilation and installation of the executable", won't be
that helpful.

Although I guess the question then becomes "what goes into the shasum"

Exactly right!

What if there's a one-byte change somewhere in a build script, or a
space added to a source file?

What you want is a more fine-grained set of shasums for each component,
possibly with pathnames somehow encoded, etc.


The idea would not just to be able to tell "does this source match" (the
answer to that is almost always "no", anyway) but "what source is
missing when it doesn't match", or "how can I quickly check that the
change to the sources since this was built is surely benign"?

Now some GPL violators will simply strip the feature out before
shipping but for those who don't, it will help in verifying that the
source they distribute matches the binaries.

Right, this idea is purely to make testing compliance of good actors
easy.


_______________________________________________
legal mailing list
[email protected]
https://lists.gpl-violations.org/mailman/listinfo/legal

Reply via email to