[Savannah-register-public] [task #8485] Submission of binary files differ

[Tong Sun]

URL:
  

 Summary: Submission of binary files differ
 Project: Savannah Administration
Submitted by: xpt
Submitted on: Saturday 08/09/2008 at 04:24
 Should Start On: Saturday 08/09/2008 at 00:00
   Should be Finished on: Tuesday 08/19/2008 at 00:00
Category: Project Approval
Priority: 5 - Normal
  Status: None
 Privacy: Public
Percent Complete: 0%
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
  Effort: 0.00

___

Details:

A new project has been registered at Savannah 
This project account will remain inactive until a site admin approves or
discards the registration.


= Registration Administration =

While this item will be useful to track the registration process, *approving
or discarding the registration must be done using the specific Group
Administration
 page*,
accessible only to site administrators, effectively *logged as site
administrators* (superuser):

* Group Administration



= Registration Details =

* Name: *binary files differ*
* System Name:  *qdiff*
* Type: non-GNU software & documentation
* License: Modified BSD License



 Description: 
qdiff - binary files differ that can detect binary
insertions/deletions/substitutions with colorful outputs.  

The differences can be printed in various ways, very colorful output
(insertions/deletions/substitutions), simple yet effective syncro engine, may
be very slow, useful and fast for large files with few differences (e.g. 
verifying correctness of grabbed cdda data). 




 Other Comments: 
Tarball info is at:
http://www.ibiblio.org/pub/Linux/utils/file/qdiff-0.8.2.lsm

The problem of the tarball is that it is very old and does not compile on
modern gcc and more. Ref:
http://thread.gmane.org/gmane.linux.distributions.grml.user/1722

I'm going to take of the code and maintain it here at nongnu.org. 




 Tarball URL: 
http://www.ibiblio.org/pub/Linux/utils/file/qdiff-0.8.2.tar.gz






___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.nongnu.org/

[Savannah-register-public] [task #8485] Submission of binary files differ

[Tong Sun]

Follow-up Comment #2, task #8485 (project administration):

yes. 

I've been waiting patiently, and hope that I can get it soon.
qdiff is the only *fast* binary files differ that's available in the FOSS
would, others don't come close to it. It should have a better home and kept
being maintained. 

thanks


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[Savannah-register-public] [task #8485] Submission of binary files differ

[Tong Sun]

Follow-up Comment #4, task #8485 (project administration):


> I've approved your project--welcome on board! :-) 

Thanks a lot!!!

> I've changed the license type in the description to 'GPLv2 or later'. 

I used to release all my programs under the BSD license. Will there be any
problem if I continue doing so with qdiff? I do saw that savannah allows some
kind of Modified BSD License (though I haven't found the exact text for it).
Please comment. 

thanks

tong


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[Savannah-register-public] [task #8485] Submission of binary files differ

[Tong Sun]

Follow-up Comment #6, task #8485 (project administration):

> but the package you've provided is already licensed under GNU GPL v2+. . .


Oh, yes, no problem. 

> Hope this is clear enough. 

Thanks


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[Savannah-register-public] [task #10138] Submission of Swiss army knife of file signature checksum tool

[Tong Sun]

URL:
  

 Summary: Submission of Swiss army knife of file signature
checksum tool
 Project: Savannah Administration
Submitted by: xpt
Submitted on: Mon 01 Feb 2010 12:56:16 PM EST
 Should Start On: Mon 01 Feb 2010 12:00:00 AM EST
   Should be Finished on: Thu 11 Feb 2010 12:00:00 AM EST
Category: Project Approval
Priority: 5 - Normal
  Status: None
 Privacy: Public
Percent Complete: 0%
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
  Effort: 0.00

___

Details:

A new project has been registered at Savannah 
This project account will remain inactive until a site admin approves or
discards the registration.


= Registration Administration =

While this item will be useful to track the registration process, *approving
or discarding the registration must be done using the specific Group
Administration
 page*,
accessible only to site administrators, effectively *logged as site
administrators* (superuser):

* Group Administration



= Registration Details =

* Name: *Swiss army knife of file signature checksum tool*
* System Name:  *checksum*
* Type: non-GNU software & documentation
* License: Modified BSD License



 Description: 
checksum is a versatile checksum creator and verifier. It support crc32 and
md5 algorithms. It can compute/verify signatures from files (with names given
from either commandline or stdin pipe), from given directory or text argument
from commandline. It understand/verify common file formats like .sfv or .md5.


Most importantly, it can remove duplicated files, i.e., identify distinct
files with identical content within the given directory using the file
signatures, and link them together to save disk space. 

programming language: C

What is special about it?

There is only one tools written in C to check and remove duplicated files --
the dupmerge, but I found its algorithm inefficient and slow:

,-
| Dupmerge works by quicksorting a list of path names, with the
| actual unlinking and relinking steps performed as side effects of
| the comparison function. The results of the sort are discarded.
`-

The algorithm is inefficient and slow because:

- File *contents* are compared *each time* in the O(n*ln(n)) comparison. 
- It doesn't provide an option to only check same-name files to speed up the
process. 
Ref:
https://sourceforge.net/projects/dupmerge/forums/forum/427960/topic/3536512
- My approach will not only omit file-contents comparison each time, but also
bring down the complexity from O(n*ln(n)) to near linear comparison, thanks to
the introduced file signature.
- The other advantage over dupmerge is that, using quicksort, the file list
has to be store in memory, which imposes a barrier to deal with a big number
of files with small memory. My checksum does not has such problem at all. 




 Other Software Required: 
MD5 message-digest algorithm, RSA Data Security, Inc.
=

 Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
rights reserved.

License to copy and use this software is granted provided that it
is identified as the "RSA Data Security, Inc. MD5 Message-Digest
Algorithm" in all material mentioning or referencing this software
or this function.

License is also granted to make and use derivative works provided
that such works are identified as "derived from the RSA Data
Security, Inc. MD5 Message-Digest Algorithm" in all material
mentioning or referencing the derived work.

RSA Data Security, Inc. makes no representations concerning either
the merchantability of this software or the suitability of this
software for any particular purpose. It is provided "as is"
without express or implied warranty of any kind.

These notices must be retained in any copies of any part of this
documentation and/or software.


32-bit CRC, ANSI X3.66
==

The computing of the 32-bit CRC is used as the frame check sequence in ADCCP
(ANSI X3.66, also known as FIPS PUB 71 and FED-STD-1003, the U.S. versions
of CCITT's X.25 link-level protocol).  The 32-bit FCS was added via the
Federal Register, 1 June 1982, p.23798.




 Other Comments: 
Half way done. No tarball to upload. Need the git to keep dev versions. The
uploaded tarball is just to satisfy the project registration. 



 Tarball URL: 
http://savannah.gnu.org/submissions_uploads/checksum.tgz






___

Reply to this item at:

  

___
  Message sent via/by Savannah
  

[Savannah-register-public] [task #10138] Submission of Swiss army knife of file signature checksum tool

[Tong Sun]

Follow-up Comment #2, task #10138 (project administration):

Thanks to your reply, Mario.

I'll use the win/md5.c file, the public domain one, as per your suggestion. 

> In GNU Savannah we host only GPL-compatible (With
any present or future version) material

I saw "Modified BSD License" is still an option. Would it be OK to release my
project as "Modified BSD License"? Reason follows. 

As for crc32.c, the zlib crc32 function *can't be used directly* since it is
using the 0x pre & post condition -- 
http://lists.xiph.org/pipermail/ogg-dev/2008-February/000803.html
However, I've found a good alternative, even faster than zlib's --
Slicing-by-8 CRC32 Implementation
http://www.slapmahfro.net/news.php?2009-11

However, there is no copyright and licensing info in source. I've written to
the author since, but haven't get reply yet:

From: Tong Sun <@cpan.org>
Date: Mon, Feb 1, 2010 at 7:15 PM
Subject: Your Slicing-by-8 CRC32 Implementation
To: brian@slapmahfro.net

Hi Brian,

I'd like to borrow your Slicing-by-8 CRC32 Implementation and use it in my
open source project, which is pending to be hosted on http://www.nongnu.org/.
(my current one there is http://www.nongnu.org/qdiff/).

I need your written permission in doing so, and your copyright and licensing
info as well.

Moreover, I don't know how much your code depends on Intel's Implementation.
I mean, I took a look at their copyright and licensing:

 * Copyright (c) 2004-2006 Intel Corporation - All Rights Reserved
 *
 * This software program is licensed subject to the BSD License,
 * available at http://www.opensource.org/licenses/bsd-license.html

and hope you release your code in BSD License as well.

Thanks


___

Reply to this item at:

  <http://savannah.gnu.org/task/?10138>

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[Savannah-register-public] [task #10138] Submission of Swiss army knife of file signature checksum tool

[Tong Sun]

Follow-up Comment #4, task #10138 (project administration):

> You must label, at least inside savannah your projects as free

Oh, thought it'd be obvious. Sure will do. 

> Do you need to use an particular polynomial for the CRC?

Yes, I need to implement the IEEE 802.3 that most CRC32 commands use, eg.
.sfv files. 

> I'm not sure, but I think MD4 is faster than CRC32 in software

Thanks for the advice, I'll add MD4 as well. I've found the OpenBSD's
implementation: 

  /*  $OpenBSD: md4.c,v 1.7 2005/08/08 08:05:35 espie Exp $   */

  /*
   * This code implements the MD4 message-digest algorithm.
   * The algorithm is due to Ron Rivest.  This code was
   * written by Colin Plumb in 1993, no copyright is claimed.
   * This code is in the public domain; do with it what you wish.
   * Todd C. Miller modified the MD5 code to do MD4 based on RFC 1186.
   *
   * Equivalent code is available from RSA Data Security, Inc.
   * This code has been tested against that, and is equivalent,
   * except that you don't need to include two pages of legalese
   * with every copy.
   *
   * To compute the message digest of a chunk of bytes, declare an
   * MD4Context structure, pass it to MD4Init, call MD4Update as
   * needed on buffers full of bytes, and then call MD4Final, which
   * will fill a supplied 16-byte array with the digest.
   */

Hope that I've clarified everything. Would my project be now
cleared for approval?

thanks


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[Savannah-register-public] [task #10138] Submission of Swiss army knife of file signature checksum tool

[Tong Sun]

Follow-up Comment #5, task #10138 (project administration):

> Additional Item Attachment, task #10138 (project administration):
> File name: dev_jackmaster-0.0.2.tar.bz2   Size:40 KB

hmmm

jackmaster - A "Master Console" for the jack-audio-connection-kit

I think it is added by mistake. 


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[Savannah-register-public] [task #10138] Submission of Swiss army knife of file signature checksum tool

[Tong Sun]

Follow-up Comment #7, task #10138 (project administration):

> >Hope that I've clarified everything. Would my project be now cleared
> >for approval?
> 
> Yes, it is, enjoy mantaining it at savannah, the url is
> http://savannah.gnu.org/p/checksum
> 
> Item status changes:
> 
> Status -> Done
> Open/Closed -> Closed

Thanks a lot, I'll start working.

___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/