Re: Concern for: A humble draft policy on "deep learning v.s. freedom"

[Yao Wei (魏銘廷)]

Hi,

>> With a labeling like "ToxicCandy Model" for the situation, it makes bad
>> impression on people and I am afraid people may not be make rational
>> decision.  Is this characterization correct and sane one?  At least,
>> it looks to me that this is changing status-quo of our policy and
>> practice severely.  So it is worth evaluating idea without labeling.
> 
> My motivation for the naming "ToxicCandy" is pure: to warn developers
> about this special case as it may lead to very difficult copyright
> or software freedom questions. I admit that this name looks not
> quite friendly. Maybe "SemiFree" look better?

About the term ToxicCandy it makes me reminded of an existing
term "Tainted" which also used in Linux kernel to describe kernel
running with non-free module.

So... how about "Tainted Model"?

Just 2 cents,
Yao Wei

(This email is sent from a phone; sorry for HTML email if it happens.)

Re: Concern for: A humble draft policy on "deep learning v.s. freedom"

[Mo Zhou]

Hi Osamu,

On 2019-06-08 18:43, Osamu Aoki wrote:
>> This draft is conservative and overkilling, and currently
>> only focus on software freedom. That's exactly where we
>> start, right?
> 
> OK but it can't be where we end-up-with.

That's why I said the two words "conservative" and "overkilling".
In my blueprint we can actually loosen these restrictions bit
by bit with further case study.

> Before scientific "deep learning" data, we already have practical "deep
> learning" data in our archive.

Thanks for pointing them out. They are good case study
for me to revise the DL-Policy.

> Please note one of the most popular Japanese input method mozc will be
> kicked out from main as a starter if we start enforcing this new
> guideline.

I'm in no position of irresponsibly enforcing an experimental
policy without having finished enough case study.

>> Specifically, I defined 3 types of pre-trained machine
>> learning models / deep learning models:
>>
>>   Free Model, ToxicCandy Model. Non-free Model
>>
>> Developers who'd like to touch DL software should be
>> cautious to the "ToxicCandy" models. Details can be
>> found in my draft.
> 
> With a labeling like "ToxicCandy Model" for the situation, it makes bad
> impression on people and I am afraid people may not be make rational
> decision.  Is this characterization correct and sane one?  At least,
> it looks to me that this is changing status-quo of our policy and
> practice severely.  So it is worth evaluating idea without labeling.

My motivation for the naming "ToxicCandy" is pure: to warn developers
about this special case as it may lead to very difficult copyright
or software freedom questions. I admit that this name looks not
quite friendly. Maybe "SemiFree" look better?

> As long as the "data" comes in the form which allows us to modify it and
> re-train it to make it better with a set of free software tools to do it,
> we shouldn't make it non-free, for sure.  That is my position and I
> think this was what we operated as the project.  We never asked how they
> are originally made.  The touchy question is how easy it should be to
> modify and re-train, etc.
>
> Let's list analogy cases.  We allow a photo of something on our archive
> as wallpaper etc.  We don't ask object of photo or tool used to make it
> to be FREE.  Debian logo is one example which was created by Photoshop
> as I understand.  Another analogy to consider is how we allow
> independent copyright and license for the dictionary like data which
> must have processed previous copyrighted (possibly non-free) texts by
> human brain and maybe with some script processing.  Packages such as
> opendict, *spell-*, dict-freedict-all, ... are in main.
> 
> I agree it is nice to have base data in the package.  If you can, please
> include the training data if it is a FREE set.  But it may become
> unrealistic for Debian to getting into business of distributing many GB
> of training data for this purpose.  You may be talking data size being over
> 10s of GB.  This is another thing you should realize -- So mandating its
> inclusion is unpractical since it is not the focus point on which Debian
> needs to spend its resource.
>
> Let's talk about actual cases in main.
> 
> "mecab" is free a tool for Japanese text morphological analysis which
> can create CRF optimized parameters from the marked-up training data.
> 
> (This is also the base of mozc which uses such data to create desirable
> typing output in normal Japanese text input from the keyboard.)
> 
> One of the dictionary for mecab is 800MB compressed deb in main:
> unidic-mecab which is 2.2GB data in text format containing CRF optimized
> parameters and other text data obtained by training. These text and
> parameters are triple licensed BSD/LGPL/GPL. Re-training this is very
> straight forward application of mecab tool with additional data only.
> So this is FREE as it can be in current practice and we have it in main.
>   https://unidic.ninjal.ac.jp/
> 
> When these CRF parameters were initially made, it used non-free data
> (Japanese Government funded) available in multiple DVDs with hefty price
> and restriction on its use and its redistribution.  This base data for
> training is as NON-FREE as it can be so we don't distribute.
>   https://pj.ninjal.ac.jp/corpus_center/bccwj/dvd-index.html
> 
> In case of MOZC, the original training data is only available in Google
> and not published by them.  Actually, tweaking data is possible but
> consistently retraining this data in MOZC may not be a trivial
> application of mecab tool.  We are placing this in main now, anyway
> since its data (CRF optimized parameters and other text data ) are
> licensed under BSD-3-clause and we have MOZC in main.

Thank you Osamu. These cases inspired me on finding a better
balance point for DL-Policy. I'll add these cases to the case
study section, and I'm going to add the following points to DL-Policy:

1. Free datasets used to train FreeModel are not required t

Re: speeding up installs

[Paul Wise]

On Sun, Jun 9, 2019 at 3:44 AM Adam Borowski wrote:

> So if you took current d-i and planted it into a regular live image

IIRC debian-live used to have that but it was too buggy so it got
removed. Later Calamares replaced it.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: ZFS in Buster

[Jeremy Stanley]

[No need to Cc an extra copy, I've been a d-d subscriber since...
the 1990s?]

On 2019-06-08 13:00:02 -0400 (-0400), Sam Hartman wrote:
> Jeremy Stanley  writes:
> > Your earlier message also implied the motives behind
> > Conservancy's recommendations to be something other than a
> > desire to protect projects relying on free/libre open source
> > software licenses from making costly mistakes. Suggesting that
> > their interpretation of these licenses is driven by an ulterior
> > motive strikes me as a gross mischaracterization, particularly
> > in light of the ways in which they (individually and
> > collectively) have demonstrated a dedication to core values of
> > software freedom over the years.
> 
> I actually think that the SFC has a strong motive to defend
> copyleft. Certainly their leaders have taken strong pro-copyleft
> positions.
> 
> I don't think it is negative to them to describe them that way at
> all.

I'll concede, I may have been reading too much into Aron Xu's
accounting of the in-person debate, but it seemed like the
implication was that Conservancy was working in their own self
interest in this matter, and not simply out of a desire to protect
the (possibly tenuous, as you also indicate) legal grounds
underpinning the GPL and by extension those projects who have chosen
the GPL to represent their needs.

> I think there are times when the desire to establish a strong
> copyleft is in conflict with the desire to accurately articulate
> the legal risks associated with something Debian might do.
[...]

I wholeheartedly concur, which is why I contested the grounds for
the suggestion that Debian was coerced into its current position
regarding ZFS integration with the Linux kernel (I was not there for
the face-to-face discussions, so all I have to go on is the
established reputations of the parties involved).

> And the fact is we don't know what would happen in the courts in a
> lot of situations. I attended a session at Libreplanet this year
> where a lawyer talked about what actual rulings the courts have
> made about the GPL and copyleft. It's not clear. Cases like Oracle
> V Google and the German Vmware case both surprised us in various
> ways.

Yes, I see the VMware result as a sort of victory. While it's
unfortunate that the German courts effectively dismissed the appeal
on technicalities without weighing in on the issue at stake, at
least the vendor has agreed to cease future flagrant violation of
license terms by removing Linux components from their proprietary
software (even if this does nothing to address their past violation
the Linux contrbutors' legally-documented wishes with regard to
their copyright). But point taken, the choice to retroactively
underpin the free/open software movement (I have trouble seeing the
two in isolation from one another as some do) with copyright law is
akin to building a house on a foundation of sand.

> And there are cases where courts will listen to what happens in
> practice. If everyone or a lot of people are interpreting a
> license a particular way, that can potentially influence courts.
> Case law matters and case law is influenced by what companies
> actually do and how copyright holders and licensees actually use
> software.
> 
> > To be clear, I seriously doubt Conservancy (or more precisely,
> > the fine individuals within Conservancy involved in debating
> > this topic over the years) would have been "upset" if Debian
> > chose to act counter to their opinions. I'm quite sure they know
> > that they don't control the choices of the Debian community, and
> > are therefore not responsible for any additional risk that
> > Debian knowingly takes on itself or passes on to its users.
> 
> It's not the risk that Debian passes on to its users. It's what
> Debian says and what happens if that becomes part of an argument
> in a court case.
> 
> Also, even if it never comes up in court, Debian's actions could
> influence others. If Debian takes a strong position it makes it
> easier to argue that in order to get your software actually used,
> you need to interpret the GPL strictly. If Debian takes a weaker
> position then as a practical matter you can commercially succeed
> by releasing CDDL works that are combined with GPL works, at least
> until a court says you cannot.

While I respect that concern for case law and precedent may be at
the forefront of Conservancy's arguments and preferences in this
matter, I prefer to look at the underlying reasons for this. I
highly doubt they chose this stance for their own sake, but rather
for the good of software relying on the licenses with which they are
most closely affiliated. I've had plenty of discussions and been on
the sidelines for plenty more where these individuals left me with
no impression that they were motivated by anything other than the
proliferation and preservation of software freedom.

> I am quite certain that factors like this were considered by
> Debian. It was more than just the lega

Re: Suspending Offer to Reimburse Expenses for Attending Future Bug Squashing Parties

[Dominik George]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Hi,

> I'm hurt reading your message and Holger's.
> I wrote to the project saying that the current situation sucked for me.
> I got some great offers of help and I think this issue will be easy to
> resolve.
> 
> But then I got a message from holger and agreement from you that
> basically said "if you did things this way it wouldn't suck."  Neither
> of you took the time to understand what my concerns are.
> You just assumed you knew what my frustration was without asking and
> jumped to a solution without actually considering the things that are
> important to me.

I do not think that this was the intention, at least not as I read it.

While the way it was written sure can be interpreted that way, I
somehow can see indications that the tone was i nno way related to the
topic. Maybe to the bar in the MiniDebConf basement; at least right
before I read that mail I saw a very tired Holger stumble by ;).

In any case, I think the point here should be - and this, I support
very much - is that we should make the process more open for non-DDs,
that is, contributors new to Debian (but of course that can be somehow
verified as having the true intention to contribute something, and in
a way that, for starters, "getting to know some othe rcontributors"
can be valued as a contribution in itself, to grow and strengthen the
community).

tl;dr: let's take the chance to make the community as open as possible
(ideally, providing good mentoring at the same time).

That, of course, does not rule out any other ideas and concerns on top
of that.

Cheers,
Nik
-BEGIN PGP SIGNATURE-

iQJlBAEBCgBPFiEEPJ1UpHV1wCb7F/0mt5o8FqDE8pYFAlz8I7UxGmh0dHBzOi8v
d3d3LmRvbWluaWstZ2VvcmdlLmRlL2dwZy1wb2xpY3kudHh0LmFzYwAKCRC3mjwW
oMTyluppD/0dFc3TvSmXbhWfV78ugKkTTLNo/0MWTpLx2qvk2V0KNm98+MxL+vBZ
hsSKsc2zYcg1Gl9gkf/9QYtxHAmz/VLm97AfMzHC1KEU+i6LDJ2ltU8EXMjC++zh
Dn5EwZHCES0zM3qRCm7J6d0HT6EX9aarqqfqzwuaCmHT3zkrRn30oaxZ2IAWQZ3A
kLyG88f+9oyIr6EfFv7+9q/9gcNfphqY621CxLeFAXDG5m5w+uzu3taLsoOMDVWM
hSiVrHPBeUIe7hv0dAmbD0CyaaqWqBD2bekOUNJCumsW4z7c1Kt1WYHTIMGqCJ9i
vh7eRsEd7Nv8BrqZEgPwUe8Xtp2ca6SOGcsjIZcnGuIydZobwEyd6as/ns1h
Vq+o+hoZAdo1WK2NTKx+nCkCNnlSaZb9UXVBdo5oAFySA9mQXw/OAhPGflcas0/p
4lUl/0WEgGe/cfs2a1R/8QGvEg1wLv7d1T6RuT2JqhpVS9uDnl3PnjAhNORh+aeD
rYPbXxwEW8gsuqk8wx1l3ejXY++woRy2KsJbIDI19vCquSdkcpwjJ0fCAk9bs0Xr
S42OC7JUtT9R+g3qiGvdPt17xwN7Tgkqez+Ki+vBZJjmGUgAnxOdKxHzCHCIyPMQ
LaDzp0NZBFGIpvN3aXzcPl+TmHFKIeSP+0E22JzxyyTxbWVI2sV85g==
=aDax
-END PGP SIGNATURE-

Re: speeding up installs

[Jonathan Carter]

On 2019/06/08 21:43, Adam Borowski wrote:
> I like this.  The d-i is nothing but a glorified live image already.  On
> every not completely non-eventful install I find myself wishing this live
> system wasn't crippled for space reasons.  This crippling was intentional --
> d-i was designed in the days of boot floppies when a couple of megabytes was
> a lot -- but today, no one gives a damn about ten or fifty megs extra to
> have a real shell, less, man pages and what not.

I did some tests before, using squashfs on an entire image is more
efficient than using udebs. If you install all of d-i's dependencies in
a chroot and squashfs it with xz, the resulting system ends up being
smaller (by a small margin but still), than the current d-i images.
Unfortunately, it's not trivial to just build normal debs of all the d-i
modules and install them in a normal live system and use them like that.
In a pinch, Calamares can also run from a framebuffer which might be
nice for some environments but the version in buster doesn't support
advanced partitioning yet, things like using RAID or ZFS.

> So if you took current d-i and planted it into a regular live image, that'd
> be great.  It probably has too much functionality to be easily rewriteable,
> but changing the base system under it sounds like a good idea.

Yeah there's been a lot of discussions about this, it needs someone to
spend a significant amount of time working with the d-i maintainers to
make that an option, which involves going through a whole lot of shell
scripts that isn't all that debianny anymore and modernising them.

> On the other hand, do we have a list of what d-i can do that Calamares or
> whatever you are using, can't?

Calamares doesn't:

 * Have a cli installer (which is still useful and probably will be for
a long time) (although as I mentioned above, it can run on a
framebuffer, which is kind of cool).
 * Install on RAID or ZFS setups or SAN devices, although kpmcore, it's
partitioning library, has recently been nearly rewritten and now has
some RAID support.
 * Have any kind of pre-seed/kickstart or method to preconfigur it,
although I've been in discussion with upstream and they are interested
in implementing it.

kpmcore has also dropped dependencies on all the kde/qt stuff. I'm often
tempted to create another installer based on kpmcore becuase besides
partitioning, most of the rest is reasonably simple. Calamares has some
good ideas (I like how super simple it is to create extensions for it,
which seems hard in d-i), but personally I'm not a fan of it being
written in C++, developers spend way too much time trying to fix crashes
which would better be spent adding new features. I also don't like its
current dependency on Qt either. Ideally I think an installer should be
more of a library than something that's tied to a front end, and then
the world can go ahead and smack on any UI (or none at all) to it that
they like. I think that would be even better than the script + configure
UI approach that things like Propellor and Subiquity does.

Calamares does a pretty good job of providing a distro-agnostic
installer that works for the large majority of desktop end-users out
there, and it gets a lot of things right that others don't, but I think
if that concept could just go a little bit more further it could be
possible to have a great installer that works for basically all the
popular linux (and non-linux) distributions and fix all the problems
with the current ones. Personally I don't think Calamares will ever be
able to replace d-i, I think eventually something will come along that
will replace both.

-Jonathan

-- 
  ⢀⣴⠾⠻⢶⣦⠀  Jonathan Carter (highvoltage) 
  ⣾⠁⢠⠒⠀⣿⡁  Debian Developer - https://wiki.debian.org/highvoltage
  ⢿⡄⠘⠷⠚⠋   https://debian.org | https://jonathancarter.org
  ⠈⠳⣄  Be Bold. Be brave. Debian has got your back.

Re: speeding up installs

[Adam Borowski]

On Sat, Jun 08, 2019 at 07:45:50PM +0200, Jonathan Carter wrote:
> It might be useful to some that we'll have a 'standard' image for Debian
> Live Buster. This is a base image with Debian standard that doesn't
> contain any desktop environment. It ships with d-i with a module that
> just unsquashes the squashfs image (just like the other live images), so
> it's a really quick and convenient way to get a Debian system installed
> on bare metal.

I like this.  The d-i is nothing but a glorified live image already.  On
every not completely non-eventful install I find myself wishing this live
system wasn't crippled for space reasons.  This crippling was intentional --
d-i was designed in the days of boot floppies when a couple of megabytes was
a lot -- but today, no one gives a damn about ten or fifty megs extra to
have a real shell, less, man pages and what not.

So if you took current d-i and planted it into a regular live image, that'd
be great.  It probably has too much functionality to be easily rewriteable,
but changing the base system under it sounds like a good idea.

On the other hand, do we have a list of what d-i can do that Calamares or
whatever you are using, can't?

Another improvement would be to kill that squashfs and replace it with a
regular minimized filesystem that expands to the whole USB stick (CDs are
long long gone).  This would avoid redownloading the same .debs if you
install from the same stick again, yet not download .debs you don't want
even once.  And it'd get us an user-friendly way of generating preseeds:
you install manually once, then use saved settings.  And heck, a CD would
simply not cache downloads.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢰⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ At least spammers get it right: "Hello beautiful!".
⠈⠳⣄

Re: @debian.org mail

[Florian Reitmeir]

Am 06.06.2019 um 12:49 schrieb Bjørn Mork:

Daniel Lange  writes:


We have more people registered for DebConf ("the Debian Developers'
conference") with @gmail.com than @debian.org addresses.

You can't fix @gmail.com.  It is deliberately broken for commercial
reasons, and that won't stop with SPF and DKIM.  Anti-spam is just the
current selling excuse for moving users to a closed, commercially
controlled, messaging service.

Document that @gmail.com doesn't work and ask anyone subscribed with
such an address to resubscribe using an Internet email service.

You might want to make a press announcement out of it, to prevent other
service providers from making the same mistake Google has made.

You can also stop sending e-mails if it doesn't matter that they arrive 
anyway.


greetings

Re: speeding up installs

[Adam Borowski]

On Sat, Jun 08, 2019 at 06:36:30PM +0100, Simon McVittie wrote:
> On Fri, 07 Jun 2019 at 19:29:49 +0200, Adam Borowski wrote:
> > A desktop install can writeout
> > while the user answers installer questions
> 
> I can't help wondering whether the right answer for a finite number
> of relatively predictable installations (a minimal system, or all of
> Priority: standard, or all of [insert favourite desktop environment
> here]) is to build a pre-prepared installation image

I want more customizability, not less!

Individual .debs are modular and can be cherry-picked.

> unpack it onto
> disk as one big blob (some compressed archive, or an unpacked tree on
> the installation media, or OSTree, or OCI, or whatever other format is
> most appropriate)

On modern hardware, parallelizing I/O makes it a lot faster.  A blob is
supposed to be read linearly.  You can split it into chunks to uncompress
individually but we already have that as individual packages.

Splitting too much has a compression ratio cost, but we'd then need some way
to combine packages into blocks compressed together.  Which would be ok for
install .isos but not as hot for unstable repos that mutate often.

> and do the configuration steps during the first boot
> into the installed system.

I'd say it's better to ask the questions _during_ install.  As soon as the
partitioner is done, we can start fetching+unpacking, and ask stuff like
root password, username or language while the network+CPU+disk are working.

Currently we have:
* a lot of questions before anything starts
* long wait: base install
* popcon
* short wait
* tasksel
* very long wait
* grub

I propose asking the questions in one non-interrupted series.  Then the
user can go grab some tea, and so on.  Or, if I'm successful, not have to
wait at all as by the time you're asked for root password, the install is
done.

And as my approach happens to require knowing the list of packages to
install beforehand, two stones with one bird...

> Smaller customizations like whether to install sshd or not could be done
> during first-boot; larger customizations like "I want to install both
> GNOME and KDE" are less suitable for this, but is that common enough to
> be optimizing for?

Why not?

I despise Red Hat's approach of installing everything but having it
disabled.

> It's also very suitable for preinstalled hardware, where it's sometimes
> referred to as the "out of the box experience": the hardware vendor does
> the partitioning and the "unpack installation image" step

That's not what I'm optimizing for.  My changes would make building the
image faster.  What you do with it is another matter.

> If the pre-prepared installation image is in a suitable format (for
> example an unpacked tree, squashfs or OSTree, but not a tarball or an
> OCI image), then there's no reason it couldn't also be bootable as a
> stateless live system.

Meh, let's install the system faster than it would take to read it from the
disk :)  (2.5GB unpacked vs 750MB packed).

> > * almost all Pre-Depends and preinsts care only about upgrades
> > * dpkg-diverts can be a problem but going yolo seems to work for me so far
> 
> These seem fine for a prototype, but also like the sort of assumption
> that is likely to come back to haunt you when moving from prototype to
> production, unless supported by Policy requiring some identifiable class
> of bad things not to happen.

Yeah.  Such assumptions can be tested on current but not future packages;
the latter would require a Policy entry or updating installer data.

What I have in mind is having the installer data ship as a purely
declarative package.  Besides keeping a list of packages which must have
their postinst run serially, and a list of tasks, being a real package means
it can have versioned Conflicts declared upon it.  This way, you know
that a given .deb won't work with this installer.

Of course, erroring during install (but not producing a broken system) is
non-ideal, but that's only a last resort fallback for unforeseen problems. 
If what I'm working on moves beyond mockups and vapourware, we can ask for
declarative diverts.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Remember, the S in "IoT" stands for Security, while P stands
⢿⡄⠘⠷⠚⠋⠀ for Privacy.
⠈⠳⣄

Concern for: A humble draft policy on "deep learning v.s. freedom"

[Osamu Aoki]

Hi,

On Tue, May 21, 2019 at 12:11:14AM -0700, Mo Zhou wrote:
> Hi people,

I see your good intention but this is basically changing status-quo for
the main requirement.

>   https://salsa.debian.org/lumin/deeplearning-policy
>   (issue tracker is enabled)

I read it ;-)

> This draft is conservative and overkilling, and currently
> only focus on software freedom. That's exactly where we
> start, right?

OK but it can't be where we end-up-with.

Before scientific "deep learning" data, we already have practical "deep
learning" data in our archive.

Please note one of the most popular Japanese input method mozc will be
kicked out from main as a starter if we start enforcing this new
guideline.

> Specifically, I defined 3 types of pre-trained machine
> learning models / deep learning models:
> 
>   Free Model, ToxicCandy Model. Non-free Model
> 
> Developers who'd like to touch DL software should be
> cautious to the "ToxicCandy" models. Details can be
> found in my draft.

With a labeling like "ToxicCandy Model" for the situation, it makes bad
impression on people and I am afraid people may not be make rational
decision.  Is this characterization correct and sane one?  At least,
it looks to me that this is changing status-quo of our policy and
practice severely.  So it is worth evaluating idea without labeling.

As long as the "data" comes in the form which allows us to modify it and
re-train it to make it better with a set of free software tools to do it,
we shouldn't make it non-free, for sure.  That is my position and I
think this was what we operated as the project.  We never asked how they
are originally made.  The touchy question is how easy it should be to
modify and re-train, etc.

Let's list analogy cases.  We allow a photo of something on our archive
as wallpaper etc.  We don't ask object of photo or tool used to make it
to be FREE.  Debian logo is one example which was created by Photoshop
as I understand.  Another analogy to consider is how we allow
independent copyright and license for the dictionary like data which
must have processed previous copyrighted (possibly non-free) texts by
human brain and maybe with some script processing.  Packages such as
opendict, *spell-*, dict-freedict-all, ... are in main.

I agree it is nice to have base data in the package.  If you can, please
include the training data if it is a FREE set.  But it may become
unrealistic for Debian to getting into business of distributing many GB
of training data for this purpose.  You may be talking data size being over
10s of GB.  This is another thing you should realize -- So mandating its
inclusion is unpractical since it is not the focus point on which Debian
needs to spend its resource.

Let's talk about actual cases in main.

"mecab" is free a tool for Japanese text morphological analysis which
can create CRF optimized parameters from the marked-up training data.

(This is also the base of mozc which uses such data to create desirable
typing output in normal Japanese text input from the keyboard.)

One of the dictionary for mecab is 800MB compressed deb in main:
unidic-mecab which is 2.2GB data in text format containing CRF optimized
parameters and other text data obtained by training. These text and
parameters are triple licensed BSD/LGPL/GPL. Re-training this is very
straight forward application of mecab tool with additional data only.
So this is FREE as it can be in current practice and we have it in main.
  https://unidic.ninjal.ac.jp/

When these CRF parameters were initially made, it used non-free data
(Japanese Government funded) available in multiple DVDs with hefty price
and restriction on its use and its redistribution.  This base data for
training is as NON-FREE as it can be so we don't distribute.
  https://pj.ninjal.ac.jp/corpus_center/bccwj/dvd-index.html

In case of MOZC, the original training data is only available in Google
and not published by them.  Actually, tweaking data is possible but
consistently retraining this data in MOZC may not be a trivial
application of mecab tool.  We are placing this in main now, anyway
since its data (CRF optimized parameters and other text data ) are
licensed under BSD-3-clause and we have MOZC in main.

Regards,

Osamu

Re: Debian, so ugly and unwieldy!

[Adam Borowski]

On Sat, Jun 08, 2019 at 10:21:22AM +0100, Jonathan Dowland wrote:
> On Fri, Jun 07, 2019 at 05:24:17PM +0200, Adam Borowski wrote:
> > In general: could we please do something to appearance beyond choosing a
> > wallpaper once a release?
> 
> I fully support more effort on improving the usability, consistency etc. 
> across
> the GUI stuff we provide. Timing-wise it's best done at a different point in
> the release cycle to now.

Good point -- but, as I said let's discuss before filing bugs.  Let's
pretend I meant that this discussion is timed in a way so once it settles
the phase of the release cycle should improve. :)

> In the past, this has been worked on via the debian-desktop@ list.

Hmm right.  Not a place I haunt, though.


Meow!
-- 
Autotools hint: to do a zx-spectrum build on a pdp11 host, type:
  ./configure --host=zx-spectrum --build=pdp11

Re: speeding up installs

[Jonathan Carter]

On 2019/06/08 19:36, Simon McVittie wrote:
> If the pre-prepared installation image is in a suitable format (for
> example an unpacked tree, squashfs or OSTree, but not a tarball or an
> OCI image), then there's no reason it couldn't also be bootable as a
> stateless live system.

It might be useful to some that we'll have a 'standard' image for Debian
Live Buster. This is a base image with Debian standard that doesn't
contain any desktop environment. It ships with d-i with a module that
just unsquashes the squashfs image (just like the other live images), so
it's a really quick and convenient way to get a Debian system installed
on bare metal.

-Jonathan

-- 
  ⢀⣴⠾⠻⢶⣦⠀  Jonathan Carter (highvoltage) 
  ⣾⠁⢠⠒⠀⣿⡁  Debian Developer - https://wiki.debian.org/highvoltage
  ⢿⡄⠘⠷⠚⠋   https://debian.org | https://jonathancarter.org
  ⠈⠳⣄  Be Bold. Be brave. Debian has got your back.

Re: speeding up installs

[Simon McVittie]

On Fri, 07 Jun 2019 at 19:29:49 +0200, Adam Borowski wrote:
> A desktop install can writeout
> while the user answers installer questions

I can't help wondering whether the right answer for a finite number
of relatively predictable installations (a minimal system, or all of
Priority: standard, or all of [insert favourite desktop environment
here]) is to build a pre-prepared installation image, unpack it onto
disk as one big blob (some compressed archive, or an unpacked tree on
the installation media, or OSTree, or OCI, or whatever other format is
most appropriate), and do the configuration steps during the first boot
into the installed system.

Smaller customizations like whether to install sshd or not could be done
during first-boot; larger customizations like "I want to install both
GNOME and KDE" are less suitable for this, but is that common enough to
be optimizing for?

This is the approach assumed by gnome-initial-setup, which we don't
really support in Debian yet (it's automatically run by gdm if there are
no users in the "ordinary user" uid range, but d-i never sets up that
situation), and also one of the modes the cross-desktop/cross-distro
Calamares installer can work in.

It's also very suitable for preinstalled hardware, where it's sometimes
referred to as the "out of the box experience": the hardware vendor does
the partitioning and the "unpack installation image" step (in practice
via a disk image), then ships the device, and the buyer starts from the
first boot into the installed system (which could start by expanding
the disk image from its minimal size to a disk-filling partition, so
one disk image can work for many disk sizes).

If the pre-prepared installation image is in a suitable format (for
example an unpacked tree, squashfs or OSTree, but not a tarball or an
OCI image), then there's no reason it couldn't also be bootable as a
stateless live system.

> * almost all Pre-Depends and preinsts care only about upgrades
> * dpkg-diverts can be a problem but going yolo seems to work for me so far

These seem fine for a prototype, but also like the sort of assumption
that is likely to come back to haunt you when moving from prototype to
production, unless supported by Policy requiring some identifiable class
of bad things not to happen.

smcv

Re: ZFS in Buster

[Sam Hartman]

> "Jeremy" == Jeremy Stanley  writes:

Jeremy> Your earlier message also implied the motives behind
Jeremy> Conservancy's recommendations to be something other than a
Jeremy> desire to protect projects relying on free/libre open source
Jeremy> software licenses from making costly mistakes. Suggesting
Jeremy> that their interpretation of these licenses is driven by an
Jeremy> ulterior motive strikes me as a gross mischaracterization,
Jeremy> particularly in light of the ways in which they
Jeremy> (individually and collectively) have demonstrated a
Jeremy> dedication to core values of software freedom over the
Jeremy> years.

I actually think that the SFC has a strong motive to defend copyleft.
Certainly their leaders have taken strong pro-copyleft positions.

I don't think it is negative to them to describe them that way at all.


I think there are times when the desire to establish a strong copyleft
is in conflict with the desire to accurately articulate the legal risks
associated with something Debian might do.

I'd totally turn to the conservancy if I wanted advice on how to best
defend copyleft.  Similarly i'd totally turn to them if I wanted help
with community GPL enforcement.


But the individuals in the SFC (if not the organization) have a vested
interest in the strong interpretation of the GPL.
So do many parts of Debian.  But not all: BSD and LGPL are also free
software licenses and you can comply with the social contract without
supporting copyleft at all.

And the fact is we don't know what would happen in the courts in a lot
of situations.  I attended a session at Libreplanet this year where a
lawyer talked about what actual rulings the courts have made about the
GPL and copyleft.  It's not clear.  Cases like Oracle V Google and the
German Vmware case both surprised us in various ways.

And there are cases where courts will listen to what happens in
practice.  If everyone or a lot of people are interpreting a license a
particular way, that can potentially influence courts..
Case law matters and case law is influenced by what companies actually
do and how copyright holders and licensees actually use software.



Jeremy> To be clear, I seriously doubt Conservancy (or more
Jeremy> precisely, the fine individuals within Conservancy involved
Jeremy> in debating this topic over the years) would have been
Jeremy> "upset" if Debian chose to act counter to their
Jeremy> opinions. I'm quite sure they know that they don't control
Jeremy> the choices of the Debian community, and are therefore not
Jeremy> responsible for any additional risk that Debian knowingly
Jeremy> takes on itself or passes on to its users.

It's not the risk that Debian passes on to its users.
It's what Debian says and what happens if that becomes part of an
argument in a court case.

Also, even if it never comes up in court, Debian's actions could
influence others.
If Debian takes a strong position it makes it easier to argue that in
order to get your software actually used, you need to interpret the GPL
strictly.
If Debian takes a weaker position then as a practical matter you can
commercially succeed by releasing CDDL works that are combined with GPL
works, at least until a court says you cannot.

I am quite certain that factors like this were considered by Debian.  It
was more than just the legal risks that were considered.  I have not
talked to the conservancy yet (although they did reach out to me when
this issue came up again).  I have read some of the correspondance
between the DPL at the time and the FSF and I can assure you that
choosing an interpretation of the GPL that defended copyleft was
something the FSF cared about.

I will be completely unsurprised to learn that the conservancy also
cared about interpreting the GPL in a manner that preserves copyleft.

And again, this is a matter of interpretation.  Vmware has taken a
different interpretation and so far they've won their court cases.

Re: Suspending Offer to Reimburse Expenses for Attending Future Bug Squashing Parties

[Sam Hartman]

> "Tobias" == Tobias Frost  writes:

Tobias> On Fri, Jun 07, 2019 at 11:10:12PM +, Holger Levsen wrote:
>> On Wed, Jun 05, 2019 at 09:19:03AM -0400, Sam Hartman wrote:
>> > Handling reimbursements for BSPs has significantly crossed my
>> threshhold > for not being fun with our current procedures.  We
>> absolutely should > reimburse developers attending BSPs
>> 
>> yes! why don't you just say 'yes' to all requests written with
>> somewhat proper grammar^w^w^w^wfrom people with a bit of
>> searchable debian activity?! that would cost, what 10k a year,
>> and would just be yay.
>> 
>> i'd be very surprised if it would raise to more than 20k even if
>> we announced this *everywhere allthetime*. and if/when that
>> happens, more yay, we increased contributors and contributions!
>> 
>> i'd be in favor. go BSPs!

Tobias> +1

I'm hurt reading your message and Holger's.
I wrote to the project saying that the current situation sucked for me.
I got some great offers of help and I think this issue will be easy to
resolve.

But then I got a message from holger and agreement from you that
basically said "if you did things this way it wouldn't suck."  Neither
of you took the time to understand what my concerns are.
You just assumed you knew what my frustration was without asking and
jumped to a solution without actually considering the things that are
important to me.

I do not feel valued in this interaction.
When I bring up an issue i'd hope that people working on trying to solve
that issue would take the time to understand what my concern is.
I hope we'd offer that to everyone in our community.

--Sam

Bug#930223: ITP: node-dagre-d3-renderer -- D3-based renderer for Dagre

[Pirate Praveen]

Package: wnpp
Severity: wishlist
Owner: Pirate Praveen 
X-Debbugs-CC: debian-devel@lists.debian.org

* Package name : node-dagre-d3-renderer
 Version : 0.5.8
 Upstream Author : Liad Yosef
* URL : https://github.com/tylingsoft/dagre-d3-renderer#readme
* License : Expat
 Programming Lang: JavaScript
 Description : D3-based renderer for Dagre
This library is an out-of-box replacement for dagre-d3 and it is based 
on the

original dagre-d3 project.
.
Dagre is a JavaScript library that makes it easy to lay out directed 
graphs on
the client-side. The dagre-d3 library acts as a front-end to dagre, 
providing

actual rendering using D3.
.
Node.js is an event-based server-side JavaScript engine.

This library is a dependency of gitlab. Since it uses babel and webpack 
to generate ES5 code, it is not suitable for embedding.

Re: ZFS in Buster

[Jeremy Stanley]

On 2019-06-08 18:11:28 +0800 (+0800), Aron Xu wrote:
> On Sat, Jun 8, 2019 at 5:33 PM Ondřej Surý  wrote:
> > On 8 Jun 2019, at 10:56, Aron Xu  wrote:
> > > Even further, it's distributed in the form of dkms source and
> > > theoretically not in Debian (contrib) to save people of
> > > Software Freedom Conservancy from being upset about losing
> > > their position of Linux GPL enforcement.
> >
> > I don’t care much about the rest, I said what I wanted to say,
> > but I strongly disagree with this statement. It’s both untrue
> > and very rude.
> >
> > We (the Debian community) care about the software freedom and
> > associated licensing because we care “the software freedom”, not
> > to please or displease some people that deeply care about the
> > same/similar thing.
> 
> There's no point to be rude from me because I was on the table
> when the face to face discussion happened. We care about software
> freedom and we have agreed that we choose to keep SFC's position
> for the good of free software ecology, so in turn we agreed to
> ship ZFS in contrib (which is not an official part of Debian) for
> the best of our users and community.
[...]

Your earlier message also implied the motives behind Conservancy's
recommendations to be something other than a desire to protect
projects relying on free/libre open source software licenses from
making costly mistakes. Suggesting that their interpretation of
these licenses is driven by an ulterior motive strikes me as a gross
mischaracterization, particularly in light of the ways in which they
(individually and collectively) have demonstrated a dedication to
core values of software freedom over the years.

To be clear, I seriously doubt Conservancy (or more precisely, the
fine individuals within Conservancy involved in debating this topic
over the years) would have been "upset" if Debian chose to act
counter to their opinions. I'm quite sure they know that they don't
control the choices of the Debian community, and are therefore not
responsible for any additional risk that Debian knowingly takes on
itself or passes on to its users. If they're going to be upset by
anything, I expect it's projects unknowingly making poor choices
which might otherwise have been avoided if only someone had given
them some guidance. That they are passionate and steadfast in
arguing their points is laudable, and does not itself indicate a
lack of good judgement.

The phrasing in your latest reply continues to paint the decision to
ship ZFS in contrib as a compromise between Debian and Conservancy,
rather than Debian making an informed choice based on advice from
Conservancy (and others). Your apparent disagreement with the result
comes across as though you're implying an adversarial relationship
between Debian and Conservancy which I sincerely hope does not
reflect the feelings of the community as a whole. As Harry Tuttle
once said, "we're all in it together."
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Bug#930219: ITP: node-dagre-layout -- Graph layout for JavaScript

[Pirate Praveen]

Package: wnpp
Severity: wishlist
Owner: Pirate Praveen 
X-Debbugs-CC: debian-devel@lists.debian.org

* Package name : node-dagre-layout
 Version : 0.8.8
 Upstream Author : Tyler Long 
* URL : https://github.com/tylingsoft/dagre-layout#readme
* License : Expat
 Programming Lang: JavaScript
 Description : Graph layout for JavaScript
This library is an out-of-box replacement for dagre and it is based on
original dagre.
.
Dagre is a JavaScript library that makes it easy to lay out directed 
graphs on

the client-side.

This library is a dependency of gitlab. Since it needs babel and 
webpack to build es5 code, it is not suitable for embedding.

Re: Suspending Offer to Reimburse Expenses for Attending Future Bug Squashing Parties

[Tobias Frost]

On Fri, Jun 07, 2019 at 11:10:12PM +, Holger Levsen wrote:
> On Wed, Jun 05, 2019 at 09:19:03AM -0400, Sam Hartman wrote:
> > Handling reimbursements for BSPs has significantly crossed my threshhold
> > for not being fun with our current procedures.  We absolutely should
> > reimburse developers attending BSPs
> 
> yes! why don't you just say 'yes' to all requests written with somewhat
> proper grammar^w^w^w^wfrom people with a bit of searchable debian
> activity?! that would cost, what 10k a year, and would just be yay.
> 
> i'd be very surprised if it would raise to more than 20k even if we
> announced this *everywhere allthetime*. and if/when that happens, more
> yay, we increased contributors and contributions!
> 
> i'd be in favor. go BSPs!

+1

I think also that BSPs are good to "aquire" new contributors or get them
wetted enough to get even more involved into Debian.

(I think that is also something we need to take care off, I have the
impression that Debian is shrinking in people, not growing -- disclaimer that
could be also a MIA-team-member-view, offseted by an subjective
impression you get when you do MIA.)

-- 
tobi


> 
> -- 
> tschau,
>   Holger
> 
> ---
>holger@(debian|reproducible-builds|layer-acht).org
>PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C




signature.asc
Description: PGP signature

Bug#930216: ITP: python-noise -- Perlin noise for image generation

[Steffen Moeller]

Package: wnpp
Severity: wishlist
Owner: Steffen Moeller 

* Package name: python-noise
  Version : 1.2.3
* URL : https://github.com/casemane/noise
* License : MIT
  Programming Lang: Python
  Description : Perlin noise for image generation

To be team maintained with the Debian Python Modules Team
on https://salsa.debian.org/python-team/modules/python-noise

Re: Debian, so ugly and unwieldy!

[Paul Wise]

On Fri, Jun 7, 2019 at 11:24 PM Adam Borowski wrote:

> In general: could we please do something to appearance beyond choosing a
> wallpaper once a release?

Please note that modifying the appearance of apps can annoy upstreams:

https://stopthemingmy.app/
https://www.omgubuntu.co.uk/2019/05/open-letter-stop-gtk-theming-distros

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

dMagnetic - A magnetic Scrolls Interpreter just had its release 0.15

[dettus]

Hello.


I couple of days ago I submitted my project "dMagnetic" to your WNPP list.
Today I released Version 0.15.


Just wanted to keep this bugreport updated.


Thomas

Re: ZFS in Buster

[Aron Xu]

On Sat, Jun 8, 2019 at 5:33 PM Ondřej Surý  wrote:
>
>
> > On 8 Jun 2019, at 10:56, Aron Xu  wrote:
> >
> > Even further, it's distributed in
> > the form of dkms source and theoretically not in Debian (contrib) to
> > save people of Software Freedom Conservancy from being upset about
> > losing their position of Linux GPL enforcement.
>
> I don’t care much about the rest, I said what I wanted to say, but I strongly 
> disagree with this statement. It’s both untrue and very rude.
>
> We (the Debian community) care about the software freedom and associated 
> licensing because we care “the software freedom”, not to please or displease 
> some people that deeply care about the same/similar thing.
>

There's no point to be rude from me because I was on the table when
the face to face discussion happened. We care about software freedom
and we have agreed that we choose to keep SFC's position for the good
of free software ecology, so in turn we agreed to ship ZFS in contrib
(which is not an official part of Debian) for the best of our users
and community.

"Our priorities are our users and free software", and the current
status quo is what we can do to make sure both our users and free
software are not disregarded from being our priorities. I believe this
is in no way untrue.

Regards,
Aron

Re: Question about Debian build infrastructure

[Simon McVittie]

On Sat, 08 Jun 2019 at 02:25:44 +0200, Matthias Klumpp wrote:
> Am Do., 6. Juni 2019 um 20:33 Uhr schrieb Kyle Edwards
> :
> > 2. According to https://wiki.debian.org/BuilddSetup, there seems to be
> > a distinction between the build broker (wanna-build) and the build
> > workers (buildd). Do either of these roles have their own OS images one
> > can download?
> 
> AFAIK there are no OS images, but I would bet that the buildd machines
> have an Ansible recipe or something similar somewhere, as those are
> continuously updated and refreshed.

I think the Debian sysadmin team (DSA) use Puppet for this.
 is a public mirror.

> Launchpad and the Open Build Service[4] may also be very interesting
> options. The Open Build Service comes with a few of its own problems
> (by being designed for RPM in the first place), but in general it is
> very neat and may be exactly what you want for this usecase - it's
> definitely a solution to look into, and I know some people use it to
> manage repositories for internal deployments.

My employer (Collabora) uses OBS for Debian derivatives, but we've had
to make some changes to make it work better with .dsc/.deb. I'm not sure
how many of our changes have gone upstream already and how many are still
delta in the Debian packages; at one point we also had downstream delta
beyond what's in Debian, but I hope that has all been fixed now.

smcv

Re: ZFS in Buster

[Ondřej Surý]

> On 8 Jun 2019, at 10:56, Aron Xu  wrote:
> 
> Even further, it's distributed in
> the form of dkms source and theoretically not in Debian (contrib) to
> save people of Software Freedom Conservancy from being upset about
> losing their position of Linux GPL enforcement.

I don’t care much about the rest, I said what I wanted to say, but I strongly 
disagree with this statement. It’s both untrue and very rude.

We (the Debian community) care about the software freedom and associated 
licensing because we care “the software freedom”, not to please or displease 
some people that deeply care about the same/similar thing.

Ondrej
--
Ondřej Surý 

Re: Debian, so ugly and unwieldy!

[Jonathan Dowland]

On Fri, Jun 07, 2019 at 05:24:17PM +0200, Adam Borowski wrote:

In general: could we please do something to appearance beyond choosing a
wallpaper once a release?


I fully support more effort on improving the usability, consistency etc. across
the GUI stuff we provide. Timing-wise it's best done at a different point in
the release cycle to now.

In the past, this has been worked on via the debian-desktop@ list.

--

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Jonathan Dowland
⢿⡄⠘⠷⠚⠋⠀ https://jmtd.net
⠈⠳⣄ Please do not CC me, I am subscribed to the list.

Re: ZFS in Buster

[Aron Xu]

On Fri, Jun 7, 2019 at 4:16 PM Philipp Kern  wrote:
>
> On 6/6/2019 8:09 PM, Aron Xu wrote:
> > Key interest in the thread is getting some insights about how to deal
> > with the awkward situation that affects ZFS experience dramatically -
> > Linux will remove those symbols in LTS kernel release, although
> > in-kernel symbols are never promised to be stable. I've been in touch
> > with ZoL upstream to listen to their plans and wishes, so that we
> > (Debian ZoL people) can take actions that serve best for our users and
> > community.
>
> I will note that in terms of prior art Debian has in the past always
> prioritized freeness over performance. Whenever there are binary blobs
> involved to improve performance, we opted not to ship them unless they
> could be reproduced using free software and have their source included.
>
> Of course in that case people were still free to install the binary
> blobs from non-free, assuming that the blob was in fact distributable.
> This would not be the case here. But crippling the performance would be
> indeed an option, even though this would make Debian much less relevant
> for ZFS deployments and people would just go and use Ubuntu instead.
>

I agree that we usually prefer free-ness and universal-ness over
performance, you know Mo Zhou in this thread is actively pushing deep
learning stuff into Debian and he is discovering a lot of way to
exploit better performance while not being too invasive to existing
culture.

Also I'd like to mention, it could be a widespread misunderstanding
that "when something is incompatible with GPL/Linux then it's likely
non-free". ZFS itself is free and open source software, I don't think
it's comparable to non-free blobs. Even further, it's distributed in
the form of dkms source and theoretically not in Debian (contrib) to
save people of Software Freedom Conservancy from being upset about
losing their position of Linux GPL enforcement.

> Still, crippling performance would still provide a lever and motivation
> for upstream to come up with a way to disable the FPU on every supported
> architecture one by one (if only on the most important one), even if
> it's brittle and hacky. I personally wonder why a kernel which provides
> a module interface does not provide a way to save FPU state, but alas,
> they made their decision.
>

It is hard to interpret why Greg KH would like to disallow access to
hardware features and I don't think I'm at the position to comment.
There is another way around at implementation side though, when a
kernel thread is going to use FPU, preempt is usually (if not always)
disabled, so it's still possible to make use of those registers
without messing up with all the kernel states. But it's still weeks to
go before being able to have an PR upstream. We are confident to have
a version of ZFS that could make use of vector instructions in several
months, but it could be hardly possible to merge back to buster stable
updates, so that we'll have to recommend people using
stretch-backports which is fine.

> In the great scheme of things doing things the slow way has forced
> certain progress to happen in the past when it was painful enough. The
> question I wonder is if we are relevant enough here to push the needle
> or if essentially all other distributions (say Ubuntu) will dare not to
> follow upstream here and carry a patch forever.
>

IIRC all major distributions have a series of downstream maintained
patches being carried for very long time, like the aufs4 patchset in
Debian kernel. But this is not the case for the FPU one because it's
not technically complex but people tend to avoid something supposed to
lead to heated discussion in some regard.

Regards,
Aron

Re: Debian, so ugly and unwieldy!

[intrigeri]

Hi Adam,

Adam Borowski:
> I also hate with a passion so-called "UX designers".

I would love if we tried to make Debian more welcoming to folks with
skills that our project lacks, such as UX design and graphics, rather
than the opposite.

> They work from a Mac while not having to actually use what
> they produce.

FYI, a UX designer using themselves the product they've designed is
only one tool in the UX toolbox for validating such work, and by far
not the best.

This being said, I'm glad you care about the appearance and user
experience of the Debian desktop :)

Cheers,
-- 
intrigeri