Re: [Reproducible-builds] [GSoC 2016] : Application review
Hi, everyone! Jérémy Bobbio: > Thanks for your application. I much appreciated that it's done before > the deadline. I also like you being clear of your other commitments. > > I think it would have been fine application for last summer, but we've > made significant progress on several fronts since, therefore I'm not > convinced that there's much work left in the tasks your propose. I think > it would be better to aim with more precise tasks, e.g. toolchain > software you'll be improving to solve classes of issues, or at leas an > outline the features you intend to add to strip-nondeterminism or > diffoscope. > > (If you feel you're part of the reproducible builds team and disagree > with my comments, please say so!) Thanks Lunar for this valuable feedback. Yes, I am agree with you. After reading the reasons( which you mentioned) and as a part of reproducible builds team I don't think the proposed work(don't need whole summer to work on) by me will help much to reproducible builds effort too. But I think there are some issues which still needs to be fixed. There are some issues in which not even a single package have patch. I will try to look into those and will try to search for solutions. So this summer I intend to work on 1) Improvements to diffoscope: 1.1) Allow users to ignore arbitrary differences (Addition of ignore-profiles flag). 1.2) Perform fuzzy-matching across archives. 1.3) Finish parallel processing part. Above points are mentioned on GSoC wiki. And also there are more features mentioned in whishlist (https://reproducible-builds.org/events/athens2015/diffoscope-wishlist/) I will try to cover some of those too. I guess Better/smarter ELF diffing is underdevelopment (I have checked git logs and diffoscope for same) 2) Improving reproducibility of Debian packages: In this section I will be fixing Debian packages and will try to find the solutions to the issues which do not have solution yet. I am trying to enlist such issues. > If you look at packages identified as leaving timestamps in gzip > headers, you'll see that most of them already have patches, and the ones > who don't are affected with other issues > https://tests.reproducible-builds.org/issues/unstable/timestamps_in_gzip_headers_issue.html > These other issues probably deter maintainers' motivation to fix the > problems with gzip timestamps. > > Almost all packages with varying mtimes in data.tar or control.tar have > patches or have been fixed through toolchain improvements: > https://tests.reproducible-builds.org/issues/unstable/varying_mtimes_in_data_tar_gz_or_control_tar_gz_issue.html > > It feels quite suboptimal to highlight user and groups in tarballs as > separate issues as I think all are affected by other tarball related > issues. They should be fixed at the same time: > https://tests.reproducible-builds.org/issues/unstable/users_and_groups_in_tarball_issue.html > > Regarding timestamps due to C pre-processor macros, Dhole is waiting > for GCC patch window to open again—which will be in April, IIRC. > So unless you intend to work on adding support for SOURCE_DATE_EPOCH in > clang, I'm not sure there's much work left on this issue. I believe that > fixing the 400+ packages individually should not be undertaken if > we can avoid it. > https://gcc.gnu.org/ml/gcc-patches/2015-12/msg01402.html > https://wiki.debian.org/ReproducibleBuilds/TimestampsFromCPPMacros > > Emmanuel Bourg has been working and fixing almost all Java-related > issues in the course of the past year. I expect he'll probably work on > this fixing locale related javadoc issue in a near future. I guess you > could coordinate with him to write the necessary patches, though. > https://tests.reproducible-builds.org/issues/unstable/locale_in_documentation_generated_by_javadoc_issue.html > A big thanks to you, because I really didn't know about many of the above things. Its good to know that people are already working on this part. :-) > These quick evaluations leave me the feeling that your proposed schedule > is currently not adequate with actual needs of the reproducible builds > effort. > > This probably means that progress can be made on making more visible > areas that actually require work… Please let me know what you think about the work which I have proposed now. I will frame timeline accordingly. PS: I really feel that I am part of reproducible builds and I want to strengthen the bond by spending my summer working with reproducible builds ;-) ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] [GSoC 2016] : Application review
Satyam Zode: > So this summer I intend to work on > 1) Improvements to diffoscope: > 1.1) Allow users to ignore arbitrary differences (Addition of > ignore-profiles flag). > 1.2) Perform fuzzy-matching across archives. > 1.3) Finish parallel processing part. > Above points are mentioned on GSoC wiki. And also there are more > features mentioned in whishlist > (https://reproducible-builds.org/events/athens2015/diffoscope-wishlist/) > I will try to cover some of those too. Sounds good. Be aware that the first part will require some design work. Finding the right UI regarding ignores might require input from the community. I'd recommend to split this into two parts (design+experiment+survey / implementation+documentation). You probably will want to work on other things in parellel with the discussions. Could you try to come up with rough estimations on how much time all of the above would require? Fixing #818856 should not be too hard. If you could submit a patch that would make more confident that you could do all the above. > I guess Better/smarter ELF diffing is underdevelopment (I have checked > git logs and diffoscope for same) Most of the improvements we could think of have indeed been implemented since. :) -- Lunar.''`. lu...@debian.org: :Ⓐ : # apt-get install anarchism `. `'` `- signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] Raspi 3 suitable for arm64?
Hi, Karsten Merker wrote: > For the Odroid-C2 there is AFAICS no mainline support at all. There was a pull recently for the SoC used on that board (Amlogic S905): https://lkml.org/lkml/2016/3/1/1401 Does that make mainline support for the Odroid C2 any more likely? I guess substantial work is still needed yet. (Annoying that there are so many cheap boards that claim to be/do so much and yet, are of little practical use if they can only boot a vendor-supplied kernel). Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: [Reproducible-builds] [GSoC 2016] : Application review
Hi all!
Jérémy Bobbio:
> Satyam Zode:
>> So this summer I intend to work on
>> 1) Improvements to diffoscope:
>> 1.1) Allow users to ignore arbitrary differences (Addition of
>> ignore-profiles flag).
>> 1.2) Perform fuzzy-matching across archives.
>> 1.3) Finish parallel processing part.
>> Above points are mentioned on GSoC wiki. And also there are more
>> features mentioned in whishlist
>> (https://reproducible-builds.org/events/athens2015/diffoscope-wishlist/)
>> I will try to cover some of those too.
>
> Sounds good. Be aware that the first part will require some design work.
> Finding the right UI regarding ignores might require input from the
> community. I'd recommend to split this into two parts
> (design+experiment+survey / implementation+documentation). You probably
> will want to work on other things in parellel with the discussions.
>
I have started thinking on design will soon let you all know. It'd be
great if you show me some directions towards designing the proposed
features.
(I will create new thread for this).
> Could you try to come up with rough estimations on how much time all of
> the above would require?
>
Yes! But first, I need some time for that (maybe upcoming 24 hours)
because I want to think more about features proposed above(I don't
know how much complex those features could be!).
Moving to the interesting part :)
> Fixing #818856 should not be too hard. If you could submit a patch that
> would make more confident that you could do all the above.
>
Thank you so much for giving me this task :-) . Now I know how
diffoscope can be tested and build from code. (Before this I was only
familiar with codebase and diffoscope functionalities )
I have fixed this issue. I also have fixed link in the documentation.
Please find an attachment.
Here is the output which I have got:
satyam@satyamz:~/Debian/experiment/diffoscope/bin$ mkdir foo bar
satyam@satyamz:~/Debian/experiment/diffoscope/bin$ touch foo/baz
satyam@satyamz:~/Debian/experiment/diffoscope/bin$ ln -s somefile bar/baz
satyam@satyamz:~/Debian/experiment/diffoscope/bin$ ./diffoscope foo bar
--- foo
+++ bar
├── stat {}
│ @@ -1,8 +1,8 @@
│
│Size: 4096 Blocks: 8 IO Block: 4096 directory
│ Links: 2
│ Access: (0755/drwxr-xr-x) Uid: ( 1000/ satyam) Gid: ( 1000/ satyam)
│
│ -Modify: 2016-03-21 17:22:54.004395232 +
│ +Modify: 2016-03-21 17:23:20.516394537 +
│
│ Birth: -
│ --- foo/baz
├── +++ bar/baz
│ @@ -0,0 +1,2 @@
│ +000: 6465 7374 696e 6174 696f 6e3a 2073 6f6d destination: som
│ +010: 6566 696c 650a efile.
│ ├── stat {}
│ │ @@ -1,8 +1,8 @@
│ │
│ │ - Size: 0 Blocks: 0 IO Block: 4096 regular empty file
│ │ + Size: 8 Blocks: 0 IO Block: 4096 symbolic link
│ │ Links: 1
│ │ -Access: (0644/-rw-r--r--) Uid: ( 1000/ satyam) Gid: ( 1000/ satyam)
│ │ +Access: (0777/lrwxrwxrwx) Uid: ( 1000/ satyam) Gid: ( 1000/ satyam)
│ │
│ │ -Modify: 2016-03-21 17:22:54.004395232 +
│ │ +Modify: 2016-03-21 17:23:20.516394537 +
│ │
│ │ Birth: -
│ ╵
╵
satyam@satyamz:~/Debian/experiment/diffoscope/bin$
> Most of the improvements we could think of have indeed been implemented
> since. :)
Nice :)
I also want to know, whether I will be able to edit application or not
after a deadline?
Thanks again!
Satyam Zode
From 3e9aea18767099dffe62c14e7215aed54347a10f Mon Sep 17 00:00:00 2001
From: Satyam Zode
Date: Mon, 21 Mar 2016 23:12:55 +0530
Subject: [PATCH 1/2] fixed issue related to diffoscope symlinks crashing
---
diffoscope/comparators/binary.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/diffoscope/comparators/binary.py b/diffoscope/comparators/binary.py
index 9663214..5622a9c 100644
--- a/diffoscope/comparators/binary.py
+++ b/diffoscope/comparators/binary.py
@@ -183,7 +183,7 @@ class File(object, metaclass=ABCMeta):
logger.debug('%s has_same_content %s', self, other)
# try comparing small files directly first
my_size = os.path.getsize(self.path)
-other_size = os.path.getsize(other.path)
+other_size = os.lstat(other.path).st_size
if my_size == other_size and my_size <= SMALL_FILE_THRESHOLD:
if open(self.path, 'rb').read() == open(other.path, 'rb').read():
return True
--
2.1.4
From 79809c35a402f1e28f1c3f7c94985274172c0415 Mon Sep 17 00:00:00 2001
From: Satyam Zode
Date: Mon, 21 Mar 2016 23:25:27 +0530
Subject: [PATCH 2/2] Fixed link in documentation
---
README.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/README.rst b/README.rst
index fe23a37..5288e52 100644
--- a/README.rst
+++ b/README.rst
@@ -83,7 +83,7 @@ system against the diffoscope package:
Join the users and developers mailing-list:
-diffoscope website is at
