Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Dear Mo, On Tue, Apr 16, 2019 at 11:12:44AM +, Mo Zhou wrote: > ... > This time I'm really giving up all related efforts [12], and shall never > touch them again. I don't feel pity, even if these points seem to be > tightly connected to some of my Debian activities. Apart from that, I'm > still willing to provide personal opinions about related packaging > works, or machine learning datasets, pretrained neural networks, etc. Thanks a lot for the summary and all your previous work you've spent into this. As far as I understand your summary it would be even "burning" a student if we would throw theses packaging task on a student in a GSoC / outreachy project (I'm aware that we are usually not supporting packaging tasks in these projects but it could be an exception in case my suspicion would be wrong). I personally do not see any capacity I could provide myself despite it would be really helpful for Debian Med and Debian Science to cover these projects. > Well, this result looks bad. Let's hope for a sun rise. +1 > [12] My on-going works about intel-mkl / BLAS / LAPACK are unrelated. > I still have strong interest in many other aspects of Debian development. Very good to know. Please keep on your great work Andreas. -- http://fam-tille.de
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Hello, Thanks for the feedback. In the accessibility team we had interest in https://github.com/mozilla/DeepSpeech but I got quickly afraid but the mention of "TensorFlow" in dependencies, it seems I was right in not taking time to have a look, unfortunately. Samuel
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Hi Andreas, On Tue, Apr 16, 2019 at 02:29:54PM +0200, Andreas Tille wrote: > Thanks a lot for the summary and all your previous work you've spent > into this. As far as I understand your summary it would be even > "burning" a student if we would throw theses packaging task on a > student in a GSoC / outreachy project (I'm aware that we are usually > not supporting packaging tasks in these projects but it could be an > exception in case my suspicion would be wrong). For your reference, at least the following skills are required if a student wants to package a basic (CPU-only, ISA=generic) version of tensorflow: * Proficient in python, very familiar with C++. * Good at cmake. Able to read the bazel build. * Basic background in machine learning, and know how to use Tensorflow to train a neural network. So that he/she will be able to conduct smoke tests and simple benchmarks. The student may also need access to a strong build machine. My Xeon E5-2687v4 x1 [1] takes about 20 minutes to finish a build, while my laptop (I5-7440HQ) takes ~90 minutes. The student also needs to compare the existing build systems: Tensorflow has three sets of build systems: (1) bazel, the officially maintained build system. bazel packaging itself is already challenging enough. Patching the bazel build to make it policy-compliant is not a trivial work as well. This soulds like a dead end to the student; (2) cmake build. It is not officially supported, and is badly synced with the bazel build. The student could make the cmake build work again as long as he/she has enough patience to dig into the bazel build and update the cmake; (3) makefile. It is targeted on some embedded services, and only builds a very core set of C++ interface. This is not a sane choice for student. Apart from that, the src:tensorflow in experimental uses a build system written by myself, which is basically building the shared libraries manually with auto-generated ninja build. This build system doesn't download several gigabytes of dependencies, doesn't mixup outputs of 72 building processes (thanks to ninja-build). It is able to produce the full set of libraries including the shared object for python interface. However, it's eventually experimental. When all the prerequisites have been satisfied, it is still possible to make some progress. Some people, for example, the accesibility team may need the basic tensorflow package in the future. A system such as DeepSpeech, that recognizes audio with pre-trained neural networks is doing "inference", or "forward pass". That process requires much less hardware performance compared to the "training" process. A basic version of tensorflow is fine for "inference". Unfortunately my demand is "training", which makes me quite unwilling to move forward a bit because I won't use the basic version[2]. Even if the basic version had been prepared, problems about ISA baseline and non-free blobs are still there. License is not a problem for the basic version because there is no non-free stuff involved, and I've already reviewed the source package file-by-file checking the license. > Very good to know. Please keep on your great work :-) [1] It builds a defconfig linux kernel within a minute. [2] My interest in a package will drastically decay to 0 if I don't use it..
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
On Tue, Apr 16, 2019 at 02:29:54PM +0200, Andreas Tille wrote: >... > As far as I understand your summary it would be even > "burning" a student if we would throw theses packaging task on a > student in a GSoC / outreachy project (I'm aware that we are usually > not supporting packaging tasks in these projects but it could be an > exception in case my suspicion would be wrong). >... How many percent of the paid GSoC and Outreachy student workers continue unpaid afterwards and become a DM or DD? My impression is that GSoC does not have a high quota, and Outreachy is a complete failure.[1] Assuming the student would succeed in creating packages, what we would get would be some complex packaging of packages with a very volatile upstream - and the only person who understands the packaging is likely gone. The work Mo spent on the already-outdated tensorflow package in experimental was wasted if there is noone who continues maintaining it. And the same would be true for other new packages if the packager disappears afterwards. What is actually needed are one (or ideally several) people with a long-term commitment to maintain these packages. cu Adrian [1] If anyone has data to prove that I am wrong, please let me to know. -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Dear Mo, Thanks for your work! On Tue, Apr 16, 2019 at 11:53 PM Mo Zhou wrote: > > The student may also need access to a strong build machine. My Xeon > E5-2687v4 x1 [1] takes about 20 minutes to finish a build, while my > laptop (I5-7440HQ) takes ~90 minutes. Have you tried ccache? Interested to know the result. Cheers, -- Roger Shimizu, GMT +3 Sofia PGP/GPG: 4096R/6C6ACD6417B3ACB1
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Hi Adrian, On Tue, Apr 16, 2019 at 11:07:34PM +0300, Adrian Bunk wrote: > > How many percent of the paid GSoC and Outreachy student workers > continue unpaid afterwards and become a DM or DD? It's not realistic to expect a student to continue any unpaid effort after the GSoC / Outreachy participation. I guess most of them come for money and experience. Continuous Debian development is partially based on love, enthusiasm and belief. It's not realistic to expect these from a random CS student, as most CS students in the world live with micro$oft malware or the apple OS. Many of my schoolmates, members of the linux user group in my university, eventually end up embracing apple OS after graduation and abandoned their ever-beloved Arch, Gentoo, etc, due to various factors e.g. the 996.ICU working schedule. > My impression is that GSoC does not have a high quota, > and Outreachy is a complete failure.[1] It's just Debian's proposals aren't attractive enough. Here is a successful example of GSoC hosting organization: https://github.com/opencv/opencv/wiki/ChangeLog#version410 Search for "GSoC" and 25 matches show up there. And I somehow submitted a GSoC proposal to Gentoo this year. > Assuming the student would succeed in creating packages, > what we would get would be some complex packaging of > packages with a very volatile upstream - and the only > person who understands the packaging is likely gone. Some ideas and proposals in the Debian community, including tensorflow packaging or DUPR, aren't quite proper for GSoC students unless they have strong motivation to do so. Maybe people could provide some more attractive and interesting ideas for GSoC students. Or the students won't understand "what on earth is this hosting organization doing". > The work Mo spent on the already-outdated tensorflow package in > experimental was wasted if there is noone who continues maintaining it. > And the same would be true for other new packages if the packager > disappears afterwards. Debian should have already got used to the fact that volunteers are scarce resource. > What is actually needed are one (or ideally several) people with > a long-term commitment to maintain these packages. I think most of such long-term + high-workload maintaining works are actually backed by salary.
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Adrian Bunk: > Outreachy is a complete failure.[1] > […] > [1] If anyone has data to prove that I am wrong, please let me to know. I only have one single data point: the only person I've been an Outreachy mentor for is now an active, uploading DD. Cheers, -- intrigeri
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Adrian Bunk wrote: > How many percent of the paid GSoC and Outreachy student workers > continue unpaid afterwards and become a DM or DD? > > My impression is that GSoC does not have a high quota, > and Outreachy is a complete failure. Curious that you have that perception. I don't have hard data but I can immediately think of at least five folks. And that's not counting myself. Regards, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org 🍥 chris-lamb.co.uk `-
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
The only single person I’ve been mentoring in GSoC did as little as possible to pass the half-time, cashed the money and stopped responding to emails. So, experience might wildly vary... Ondrej -- Ondřej Surý ond...@sury.org > On 17 Apr 2019, at 07:17, intrigeri wrote: > > Adrian Bunk: >> Outreachy is a complete failure.[1] >> […] >> [1] If anyone has data to prove that I am wrong, please let me to know. > > I only have one single data point: the only person I've been an > Outreachy mentor for is now an active, uploading DD. > > Cheers, > -- > intrigeri >
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
On Wed, Apr 17, 2019 at 07:08:53AM -0400, Chris Lamb wrote: > Adrian Bunk wrote: > > > How many percent of the paid GSoC and Outreachy student workers > > continue unpaid afterwards and become a DM or DD? > > > > My impression is that GSoC does not have a high quota, > > and Outreachy is a complete failure. > > Curious that you have that perception. I don't have hard data but I > can immediately think of at least five folks. > > And that's not counting myself. GSoC or Outreachy? And 6 out of how many? Where can I find a list of all past Debian GSoC and Outreachy participants? Based on that it would be easy to get hard data. > Regards, cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Hi Ondřej, On Wed, Apr 17, 2019 at 01:12:12PM +0200, Ondřej Surý wrote: > The only single person I’ve been mentoring in GSoC did as little as > possible to pass the half-time, cashed the money and stopped > responding to emails. > > So, experience might wildly vary... Sorry to hear that. It must be a hurtful experience working with someone who would rob the money and flee.
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
> On 17 Apr 2019, at 13:58, Mo Zhou wrote: > > Hi Ondřej, > > On Wed, Apr 17, 2019 at 01:12:12PM +0200, Ondřej Surý wrote: >> The only single person I’ve been mentoring in GSoC did as little as >> possible to pass the half-time, cashed the money and stopped >> responding to emails. >> >> So, experience might wildly vary... > > Sorry to hear that. It must be a hurtful experience working with > someone who would rob the money and flee. Well, actually, it was a good lesson. I should have failed him much sooner, so at least I did gain some experience in that regard. Also his work wasn’t complete failure - while he didn’t do much, he had some some ideas and it finally pushed me to finish the work on coinstallable PHP packages. Ondrej -- Ondřej Surý ond...@sury.org
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Hi Chris On 2019/04/17 13:08, Chris Lamb wrote: >> How many percent of the paid GSoC and Outreachy student workers >> continue unpaid afterwards and become a DM or DD? >> >> My impression is that GSoC does not have a high quota, >> and Outreachy is a complete failure. > > Curious that you have that perception. I don't have hard data but I > can immediately think of at least five folks. > > And that's not counting myself. Well, the Outreachy site is running, and since they have plans ahead they clearly seem to be continuing their work. So if Adrian wants to call it a complete failure then the burden lies on him to justify that statement, and not on everyone else to invalidate it. -Jonathan -- ⢀⣴⠾⠻⢶⣦⠀ Jonathan Carter (highvoltage) ⣾⠁⢠⠒⠀⣿⡁ Debian Developer - https://wiki.debian.org/highvoltage ⢿⡄⠘⠷⠚⠋ https://debian.org | https://jonathancarter.org ⠈⠳⣄ Be Bold. Be brave. Debian has got your back.
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
On Wed, Apr 17, 2019 at 01:38:22AM +, Mo Zhou wrote: > Hi Adrian, Hi Mo, > On Tue, Apr 16, 2019 at 11:07:34PM +0300, Adrian Bunk wrote: >... > > The work Mo spent on the already-outdated tensorflow package in > > experimental was wasted if there is noone who continues maintaining it. > > And the same would be true for other new packages if the packager > > disappears afterwards. > > Debian should have already got used to the fact that volunteers are > scarce resource. Are they scarce or scared away? I had some points in [1] regarding the problems of volunteers trying to get contributions in Debian. > > What is actually needed are one (or ideally several) people with > > a long-term commitment to maintain these packages. > > I think most of such long-term + high-workload maintaining works are > actually backed by salary. >From what I am aware, I would say it is pretty much an exception when maintaining packages in unstable is paid work. Especially long-term, apart from Canonical there are not many companies that have an interest in Debian that does not go away after their next round of restructure+layoffs. cu Adrian [1] https://lists.debian.org/debian-vote/2019/03/msg00199.html -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Hi, On 4/16/19 10:07 PM, Adrian Bunk wrote: > On Tue, Apr 16, 2019 at 02:29:54PM +0200, Andreas Tille wrote: >> ... >> As far as I understand your summary it would be even >> "burning" a student if we would throw theses packaging task on a >> student in a GSoC / outreachy project (I'm aware that we are usually >> not supporting packaging tasks in these projects but it could be an >> exception in case my suspicion would be wrong). >> ... > > How many percent of the paid GSoC and Outreachy student workers > continue unpaid afterwards and become a DM or DD? > > My impression is that GSoC does not have a high quota, > and Outreachy is a complete failure.[1] Should the success of (Debians paricipation in) GSoC and Outreachy really be measured on the participants becoming DM or DD afterwards? I think its also a win if a student decides to join some upstream project or contributes to another distribution or even just knows about Debian and uses it for further work. cheers, Birger
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
On Wed, Apr 17, 2019 at 03:04:12PM +0200, Jonathan Carter wrote: > On 2019/04/17 13:08, Chris Lamb wrote: > >> How many percent of the paid GSoC and Outreachy student workers > >> continue unpaid afterwards and become a DM or DD? > >> > >> My impression is that GSoC does not have a high quota, > >> and Outreachy is a complete failure. > > > > Curious that you have that perception. I don't have hard data but I > > can immediately think of at least five folks. > > > > And that's not counting myself. > > Well, the Outreachy site is running, and since they have plans ahead > they clearly seem to be continuing their work. So if Adrian wants to > call it a complete failure then the burden lies on him to justify that > statement, and not on everyone else to invalidate it. I said it was my impression, and was asking if anyone has data to prove me wrong. IMHO anyone spending a 5 digit US Dollar amount of Debian money annually should also have the burden of proof that the money is well-spent. Personally I would also consider it questionable that Debian Outreachy does "expressly invite"[1] every black man born with the privilege of US citizenship to participate, while black heterosexual men in South Africa are excluded and heterosexual men from South America are only invited if living in the US - this conveys a not very subtle message what kind of diversity and inclusivity in Debian is wanted and what kind is not... I am sure this all makes sense for people in the US, but for a global project it is a WTF. It might even happen one day that a female student born into a middle-class German family gets paid 5k from Debian plus mentorship from a DD in South America - who is too shy to ask Debian for 1k travel sponsorship to a Debian event in Europe he cannot afford himself.[2] > -Jonathan cu Adrian [1] https://wiki.debian.org/Outreachy/Round17 [2] See the "we know it could be used for 4 or 5 other developers inside Europe to travel around there" in [3]. [3] https://lists.debian.org/debian-vote/2019/04/msg00011.html -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
On Wed, Apr 17, 2019 at 08:05:17PM +0300, Adrian Bunk wrote: > I am sure this all makes sense for people in the US, > but for a global project it is a WTF. not commenting on your main point (the above), but... > [2] See the "we know it could be used for 4 or 5 other developers > inside Europe to travel around there" in [3]. > [3] https://lists.debian.org/debian-vote/2019/04/msg00011.html I'm very glad this particular developer has asked the DPL for travel funds and will be attending the mini debconf in hamburg! and so should you! ;p https://wiki.debian.org/DebianEvents/de/2019/MiniDebConfHamburg -- tschau, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C Dance like no one's watching. Encrypt like everyone is. signature.asc Description: PGP signature
Re: Bits from /me: Difficulties in Deep Learning Framework Packaging
Adrian Bunk wrote: > > Curious that you have that perception. I don't have hard data but I > > can immediately think of at least five folks. > > > > And that's not counting myself. > > GSoC or Outreachy? I don't really track in my head the origin outreach programme of these new friends of mine, sorry. > And 6 out of how many? No idea. As others have pointed out, it feels like the ball is not in my court to provide hard data to disprove your impressions. To blindly continue in a purely anecdotal manner, since I posted I could recall at least two more and was even told IRL about another two in an entirely-spontaneous conversation last night. Furthermore, the silent implication that this is the primary or even the sole metric for these initiatives success does not sit right to me, not least because a number you might find unacceptably low might say more about Debian's attitude or welcoming atmosphere rather than a judgement of the programmes themselves... As an aside, you may need to reassure or correct me but I must say I find myself somewhat uneasy at the tenor and manner in which you are "just asking questions" in this thread. Regards, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org 🍥 chris-lamb.co.uk `-
