Re: Proposing a New App Developer Upload Process

[Iain Lane]

Hi,

On Mon, Sep 03, 2012 at 07:59:15PM +0200, David Planella wrote:
> Hi everyone,
> 
> […] 
> As we continue building upon these foundations, we also keep on refining
> our processes to identify and improve areas in which we can provide a
> better experience for app developers. While doing this, we've been
> gathering metrics from different sources –the current queue of apps
> pending review and the Showdown results being the main ones. They all
> provide clear evidence: the current approach to submit third-party apps
> in the Extras repository has already reached the limit in terms of human
> review capability and application throughput.
> 
> Despite our best intentions and the Ubuntu App Review Board's epic
> efforts, we're currently putting a strain on reviewers (who cannot keep
> up with the incoming stream of apps) and providing an unsatisfactory
> experience for app authors (who have to endure long delays to be able to
> upload their apps).

I am trying to leave my opinions aside for now (until UDS), but I feel
compelled provide some food for thought. (I don't imagine that I will
reply again to this thread)

There are a lot of existing relationships both within Ubuntu and between
Ubuntu (developers) and (developers of) other projects. I'm mainly
thinking about Debian. When making any proposal to change the way we
deliver software to our users, we should think about the effect that it
will have on these relationships with and the morale of (likely not an
exhaustive list):

  - Our MOTU and MOTU hopefuls, who have been trying to fight a meme
that they were to be disbanded.
  - Debian which, if the tie is cut (due to universe ceasing to be the
distribution point for new software), will be less important to
Ubuntu. (Why package my software for Debian if most users will not
see it due to upstream's package taking priority?)
  - Technical users and other developers who could be disempowered to
fix software if the distribution's role is reduced to that of a
conduit for "non-platform" software.

Creating two almost entirely separate tiers of software distribution is
something that requires a very large amount of thought and careful
deliberation. We absolutely must not rush into anything.

Thanks for reading,

-- 
Iain Lane  [ i...@orangesquash.org.uk ]
Debian Developer   [ la...@debian.org ]
Ubuntu Developer   [ la...@ubuntu.com ]


signature.asc
Description: Digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Stefano Rivera]

Hi Loïc (2012.09.07_15:48:53_+0200)
> e.g. can an application rely on libgtk-x11-2.0.so.0 to be there or
> should it bundle it?

It wouldn't make any sense to bundle anything like that. The library's
ABI isn't going to change over the life of the stable release.

However, (although this is the wrong thread for this, sorry) apps *are*
going to need to bundle new third party libraries that aren't in Ubuntu.
And then we run into the problem of the third party libraries not being
authored by the app author.

Even if there isn't a bundled library, there is quite likely to be
bundled artwork not authored by the app author.

It seems that we have to allow apps to incorporate works with copyright
held by third parties, assuming it's released under an appropriate
license.

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  H: +27 21 461 1230 C: +27 72 419 8559

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/07/2012 09:48 AM, Loïc Minier wrote:
> On Fri, Sep 07, 2012, Matthew Paul Thomas wrote:
>> What kind of sandboxing, specifically, do you think would be necessary
>> for hundreds of thousands of Ubuntu applications not to interfere with
>> each other? It seems to me there are four possible points of contention:
>> 1.  package names (versus the OS archive, and versus each other)
>> 2.  installed files
>> 3.  saved documents and settings
>> 4.  resource use (memory, CPU, network, peripherals) while running.
> 
> Sandboxing might also involve enforcing the app / system interface; e.g.
> not expose any other shared library than the ones application can rely
> on being "always there" for a particular version of the interface.
> 
> e.g. can an application rely on libgtk-x11-2.0.so.0 to be there or
> should it bundle it?  If we encourage apps to be self-contained, we are
> lowering the overall security experience of the system by expecting all
> application developers to update a lot of embedded libraries; if we make
> them rely on system libraries, we're stuck with deps on them "forever".
> 

There is currently other work going on to define the "platform" that
Ubuntu offers to application developers, and things like what Gtk
version and it's APIs will be part of that.  App developers will be
encouraged to use the packages already provided in our repositories,
even if they're not installed, by using Depends in their package
definition.  Part of the spec says that they can't depend on a maximum
version, so they can't say it *has* to be version 2.0.0 ( = 2.0.0), they
can only say it has to be *at* *least* version 2.0.0 ( >= 2.0.0), so
that we can update it with security fixes.

> 
> Another constraints for sandboxing is integration between apps and
> integration of apps with the system.  There are various levels at which
> we expect apps will integrate with the system such as notification area
> icon, a background service, gadgets, but integration between apps is
> also important and isn't very developed in Android / iOs.  Sure, there
> are some "Share" buttons or "Open with" intents in iOS and Android and
> even Nautilus has a "Send to...", but I feel this is a very limited
> level of integration.  Will we allow detecting the presence of another
> app?  How do I embed this or that image viewer or music player into this
> or that cloud file sharing app?
>   Also, we want application sandboxing but are we going to allow
> replacing system services in apps?  Would we allow an app to act as an
> interactive desktop background?  Are sandboxed apps always fullscreen
> like on Android and iOS, or may they have resizeable windows?
> 

While this would be awesome to have, it's not part of the upload
process.  I'd really like to see a "desktop intents" framework for
Linux, and I'm aware of at least a couple projects that have started on
one.  But that is something for a different spec.

> 
> [ 2/ (installed files) above seems like a non-problem if we have unique
> app names though ]
> 

Michael Hall
mhall...@ubuntu.com

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[David Planella]

Al 07/09/12 15:11, En/na Scott Kitterman ha escrit:
> On Friday, September 07, 2012 11:01:55 AM Matthew Paul Thomas wrote:
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>
>> Scott Kitterman wrote on 06/09/12 22:02:
>>> On Thursday, September 06, 2012 09:36:05 PM Matthew Paul Thomas
>>> wrote:
>>>
>>> ...
>>>
 Scott Kitterman wrote on 06/09/12 14:22:
>>> ...
>>>
> To pick just one example, rolling delivery of applications and
> offering multiple versions of the same package (which is what
> the BSD Unixes do, although not in a way I've personally found
> at all satisfying as a user) implies huge changes in package
> management. Not the least of which is the ability to support
> downgrades, including migrating per user settings back to
> previous versions.

 Windows, OS X, iOS, and Android all let an author issue a new or
 updated application within a week or so of wanting to, which I
 guess is what you mean by "rolling delivery". This whole
 discussion is predicated on that being a requirement for a
 mass-market OS.
>>>
>>> On iOS and Android there's rather more extreme sandboxing than is
>>> being proposed here that make potential interactions between
>>> applications less of an issue.  The term DLL hell was invented for
>>> Windows.  I don't think it's at all obvious that replicating the
>>> existing systems is a path to success.
>>
>> DLL hell or no, Windows and (lesserly) Mac OS were both popular for
>> decades before they both introduced sandboxing this year. But let's
>> leave aside the definition of "success".
>>
>> What kind of sandboxing, specifically, do you think would be necessary
>> for hundreds of thousands of Ubuntu applications not to interfere with
>> each other? It seems to me there are four possible points of contention:
>> 1.  package names (versus the OS archive, and versus each other)
>> 2.  installed files
>> 3.  saved documents and settings
>> 4.  resource use (memory, CPU, network, peripherals) while running.
>>
>> All four might have benefits, but for this particular goal I think
>> it's sufficient to have technical measures for #1 and #2. Do you at
>> least agree on that?
> 
> For this particular goal, #1 and #2 are essential.  Some elements of #3 and 
> #4 
> are important as well to meet the security goals associated with the 
> proposal, 
> which is a critical element of reducing review/allowing third parties more 
> direct control, so I agree we have to have #1 and #2.  It may even be 
> sufficient 
> to start with #1 and #2 designed and work out the details on #3 and #4 as the 
> project matures.
> 
>>> There is an assumption here that there's a vast user base that
>>> awaits if only it weren't so hard to run the latest versions of
>>> things.
>>
>> That's a misunderstanding. There are, indeed, users who get annoyed
>> when they can't play a game like Hedgewars online because the packaged
>> version of the game is too old for the server. (Raises hand.)
> 
> We have mechanisms in place already to fix this problem, but that's a 
> separate 
> issue.
> 
>> But the main motivation is that one of the things we need for a vast
>> user base is an order of magnitude more and better applications. And
>> one of the things we need to attract those application developers is
>> the ability for them to issue new and updated applications at will.
>> It's not sufficient, but it is necessary.
> 
> OK.  I guess that's the crux of the argument.  We need better/more 
> applications (personally, I think better is far more important than more, but 
> I guess that's a detail) and you believe more users will attract the 
> developers to achieve that.  I suspect that helps more with more than better, 
> but who knows.  For commercial vendors, I can definitely see the more users 
> == 
> more software, but for FOSS developers is there a significant body of 
> developers for whom Ubuntu isn't already big enough to be important?
> 
>>> ...
>>>
 None of those OSes offer application downgrades (though
 individual applications might). None of them migrate user
 settings to previous versions. ("The file “iTunes Library” cannot
 be read because it was created by a newer version of iTunes.
 Would you like to download iTunes now?") And absent that settings
 problem, OS X is the only one that makes multiple application
 versions moderately easy ("portable apps" on Windows being the
 exception rather than the rule). That tells me that while they
 might be desirable, none of those three things is a requirement
 for a mass-market OS.
>>>
>>> It is also quite normal for things to get screwed up on other
>>> operating systems and people do a full reinstall to fix it.  One
>>> of the fundamental design principles behind package management in
>>> Debian and by extension Ubuntu is that once installed, systems can
>>> just be upgraded.  Reinstalls should not be needed.  I think this
>>> is an important feature that sho

Re: Proposing a New App Developer Upload Process

[Loïc Minier]

On Fri, Sep 07, 2012, Matthew Paul Thomas wrote:
> What kind of sandboxing, specifically, do you think would be necessary
> for hundreds of thousands of Ubuntu applications not to interfere with
> each other? It seems to me there are four possible points of contention:
> 1.  package names (versus the OS archive, and versus each other)
> 2.  installed files
> 3.  saved documents and settings
> 4.  resource use (memory, CPU, network, peripherals) while running.

Sandboxing might also involve enforcing the app / system interface; e.g.
not expose any other shared library than the ones application can rely
on being "always there" for a particular version of the interface.

e.g. can an application rely on libgtk-x11-2.0.so.0 to be there or
should it bundle it?  If we encourage apps to be self-contained, we are
lowering the overall security experience of the system by expecting all
application developers to update a lot of embedded libraries; if we make
them rely on system libraries, we're stuck with deps on them "forever".


Another constraints for sandboxing is integration between apps and
integration of apps with the system.  There are various levels at which
we expect apps will integrate with the system such as notification area
icon, a background service, gadgets, but integration between apps is
also important and isn't very developed in Android / iOs.  Sure, there
are some "Share" buttons or "Open with" intents in iOS and Android and
even Nautilus has a "Send to...", but I feel this is a very limited
level of integration.  Will we allow detecting the presence of another
app?  How do I embed this or that image viewer or music player into this
or that cloud file sharing app?
  Also, we want application sandboxing but are we going to allow
replacing system services in apps?  Would we allow an app to act as an
interactive desktop background?  Are sandboxed apps always fullscreen
like on Android and iOS, or may they have resizeable windows?


[ 2/ (installed files) above seems like a non-problem if we have unique
app names though ]

-- 
Loïc Minier

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Friday, September 07, 2012 11:01:55 AM Matthew Paul Thomas wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Scott Kitterman wrote on 06/09/12 22:02:
> > On Thursday, September 06, 2012 09:36:05 PM Matthew Paul Thomas
> > wrote:
> > 
> > ...
> > 
> >> Scott Kitterman wrote on 06/09/12 14:22:
> > ...
> > 
> >>> To pick just one example, rolling delivery of applications and
> >>> offering multiple versions of the same package (which is what
> >>> the BSD Unixes do, although not in a way I've personally found
> >>> at all satisfying as a user) implies huge changes in package
> >>> management. Not the least of which is the ability to support
> >>> downgrades, including migrating per user settings back to
> >>> previous versions.
> >> 
> >> Windows, OS X, iOS, and Android all let an author issue a new or
> >> updated application within a week or so of wanting to, which I
> >> guess is what you mean by "rolling delivery". This whole
> >> discussion is predicated on that being a requirement for a
> >> mass-market OS.
> > 
> > On iOS and Android there's rather more extreme sandboxing than is
> > being proposed here that make potential interactions between
> > applications less of an issue.  The term DLL hell was invented for
> > Windows.  I don't think it's at all obvious that replicating the
> > existing systems is a path to success.
> 
> DLL hell or no, Windows and (lesserly) Mac OS were both popular for
> decades before they both introduced sandboxing this year. But let's
> leave aside the definition of "success".
> 
> What kind of sandboxing, specifically, do you think would be necessary
> for hundreds of thousands of Ubuntu applications not to interfere with
> each other? It seems to me there are four possible points of contention:
> 1.  package names (versus the OS archive, and versus each other)
> 2.  installed files
> 3.  saved documents and settings
> 4.  resource use (memory, CPU, network, peripherals) while running.
> 
> All four might have benefits, but for this particular goal I think
> it's sufficient to have technical measures for #1 and #2. Do you at
> least agree on that?

For this particular goal, #1 and #2 are essential.  Some elements of #3 and #4 
are important as well to meet the security goals associated with the proposal, 
which is a critical element of reducing review/allowing third parties more 
direct control, so I agree we have to have #1 and #2.  It may even be 
sufficient 
to start with #1 and #2 designed and work out the details on #3 and #4 as the 
project matures.

> > There is an assumption here that there's a vast user base that
> > awaits if only it weren't so hard to run the latest versions of
> > things.
> 
> That's a misunderstanding. There are, indeed, users who get annoyed
> when they can't play a game like Hedgewars online because the packaged
> version of the game is too old for the server. (Raises hand.)

We have mechanisms in place already to fix this problem, but that's a separate 
issue.

> But the main motivation is that one of the things we need for a vast
> user base is an order of magnitude more and better applications. And
> one of the things we need to attract those application developers is
> the ability for them to issue new and updated applications at will.
> It's not sufficient, but it is necessary.

OK.  I guess that's the crux of the argument.  We need better/more 
applications (personally, I think better is far more important than more, but 
I guess that's a detail) and you believe more users will attract the 
developers to achieve that.  I suspect that helps more with more than better, 
but who knows.  For commercial vendors, I can definitely see the more users == 
more software, but for FOSS developers is there a significant body of 
developers for whom Ubuntu isn't already big enough to be important?

> > ...
> > 
> >> None of those OSes offer application downgrades (though
> >> individual applications might). None of them migrate user
> >> settings to previous versions. ("The file “iTunes Library” cannot
> >> be read because it was created by a newer version of iTunes.
> >> Would you like to download iTunes now?") And absent that settings
> >> problem, OS X is the only one that makes multiple application
> >> versions moderately easy ("portable apps" on Windows being the
> >> exception rather than the rule). That tells me that while they
> >> might be desirable, none of those three things is a requirement
> >> for a mass-market OS.
> > 
> > It is also quite normal for things to get screwed up on other
> > operating systems and people do a full reinstall to fix it.  One
> > of the fundamental design principles behind package management in
> > Debian and by extension Ubuntu is that once installed, systems can
> > just be upgraded.  Reinstalls should not be needed.  I think this
> > is an important feature that should not be lightly abandoned.  I
> > think the implication of that is you've got to find a way to
> > support downgrades.
> 
> 

Re: Proposing a New App Developer Upload Process

[Matthew Paul Thomas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Scott Kitterman wrote on 06/09/12 22:02:
> 
> On Thursday, September 06, 2012 09:36:05 PM Matthew Paul Thomas 
> wrote:
> 
> ...
>> Scott Kitterman wrote on 06/09/12 14:22:
> ...
>>> To pick just one example, rolling delivery of applications and 
>>> offering multiple versions of the same package (which is what 
>>> the BSD Unixes do, although not in a way I've personally found 
>>> at all satisfying as a user) implies huge changes in package 
>>> management. Not the least of which is the ability to support 
>>> downgrades, including migrating per user settings back to 
>>> previous versions.
>> 
>> Windows, OS X, iOS, and Android all let an author issue a new or 
>> updated application within a week or so of wanting to, which I 
>> guess is what you mean by "rolling delivery". This whole 
>> discussion is predicated on that being a requirement for a 
>> mass-market OS.
> 
> On iOS and Android there's rather more extreme sandboxing than is 
> being proposed here that make potential interactions between 
> applications less of an issue.  The term DLL hell was invented for 
> Windows.  I don't think it's at all obvious that replicating the 
> existing systems is a path to success.

DLL hell or no, Windows and (lesserly) Mac OS were both popular for
decades before they both introduced sandboxing this year. But let's
leave aside the definition of "success".

What kind of sandboxing, specifically, do you think would be necessary
for hundreds of thousands of Ubuntu applications not to interfere with
each other? It seems to me there are four possible points of contention:
1.  package names (versus the OS archive, and versus each other)
2.  installed files
3.  saved documents and settings
4.  resource use (memory, CPU, network, peripherals) while running.

All four might have benefits, but for this particular goal I think
it's sufficient to have technical measures for #1 and #2. Do you at
least agree on that?

> There is an assumption here that there's a vast user base that 
> awaits if only it weren't so hard to run the latest versions of 
> things.

That's a misunderstanding. There are, indeed, users who get annoyed
when they can't play a game like Hedgewars online because the packaged
version of the game is too old for the server. (Raises hand.)

But the main motivation is that one of the things we need for a vast
user base is an order of magnitude more and better applications. And
one of the things we need to attract those application developers is
the ability for them to issue new and updated applications at will.
It's not sufficient, but it is necessary.

> ...
> 
>> None of those OSes offer application downgrades (though 
>> individual applications might). None of them migrate user 
>> settings to previous versions. ("The file “iTunes Library” cannot
>> be read because it was created by a newer version of iTunes.
>> Would you like to download iTunes now?") And absent that settings
>> problem, OS X is the only one that makes multiple application
>> versions moderately easy ("portable apps" on Windows being the
>> exception rather than the rule). That tells me that while they
>> might be desirable, none of those three things is a requirement
>> for a mass-market OS.
> 
> It is also quite normal for things to get screwed up on other 
> operating systems and people do a full reinstall to fix it.  One
> of the fundamental design principles behind package management in 
> Debian and by extension Ubuntu is that once installed, systems can 
> just be upgraded.  Reinstalls should not be needed.  I think this 
> is an important feature that should not be lightly abandoned.  I 
> think the implication of that is you've got to find a way to 
> support downgrades.

It is not an implication of that, because Debian and Ubuntu have never
allowed downgrades. Upgradeability may be a design principle behind
package management in Debian and Ubuntu, but downgradeability is not.

Lack of rollback in general is a big problem: woe betide you if your
battery runs out during an update. But again, that doesn't mean that
problem is best solved together with the app developer upload problem.
If you think it should be, now is the time to explain why.

> I think it's feasible, perhaps using something like etckeeper, but 
> it would take work.
> 
> Allowing areas where we beat the proprietary competition in 
> quality/ capability to degrade to their level is not a recipe for 
> success.

Beat them how? Windows Installer allows rollback to a known-good
state. dpkg does not.

> ...
> 
> The current ARB process and this new proposal are focused on a 
> specific class of applications that make use of some specific 
> restricted features of the operating system.  The current proposal 
> is geared towards that.  If the real goal though, is to deliver
> all applications this way, I think this proposal is not very
> useful because I don't think it's extensible in the ways it would
> need to be to gro

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Thursday, September 06, 2012 09:36:05 PM Matthew Paul Thomas wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Scott Kitterman wrote on 06/09/12 14:22:
> > Matthew Paul Thomas  wrote:
> > 
> > ...
> > 
> >> Scott Kitterman wrote on 05/09/12 18:54:
> > ...
> > 
> >>> - In this brave new world, what is the definition of "Ubuntu
> >>> itself"? If the application developers are unshackled then it
> >>> must be some subset of what we consider Ubuntu to be today.
> >> 
> >> The shell and everything that makes it work (down to the kernel
> >> and init system), the toolchain, official developer APIs and the
> >> software that implements them, and whichever apps are installed
> >> by default.
> > 
> > ...
> > 
> > I suspect not everyone shares your definition of what Ubuntu
> > is/will be.  Even the I favor a more traditional scope myself, I'm
> > very surprised you didn't include the desktop environment (e.g.
> > Unity, Plasma, etc.).
> 
> That's what I meant by "the shell".

Yes.  Someone else pointed that out already.

> > I think it's critical that there be some shared vision of where we
> > are going and even though there won't be resources to do
> > everything at once, a broad outline of the chunks of work needed to
> > get there.
> > 
> > To pick just one example, rolling delivery of applications and
> > offering multiple versions of the same package (which is what the
> > BSD Unixes do, although not in a way I've personally found at all
> > satisfying as a user) implies huge changes in package management.
> > Not the least of which is the ability to support downgrades,
> > including migrating per user settings back to previous versions.
> 
> Windows, OS X, iOS, and Android all let an author issue a new or
> updated application within a week or so of wanting to, which I guess
> is what you mean by "rolling delivery". This whole discussion is
> predicated on that being a requirement for a mass-market OS.

On iOS and Android there's rather more extreme sandboxing than is being 
proposed here that make potential interactions between applications less of an 
issue.  The term DLL hell was invented for Windows.  I don't think it's at all 
obvious that replicating the existing systems is a path to success.

There is an assumption here that there's a vast user base that awaits if only 
it weren't so hard to run the latest versions of things.  While there are 
certainly many such people in the geek community, the vast majority of non-
technical end users neither know nor care about what version of anything they 
are running.  The class of users that will tell their friends "I just bought 
Facebook" to mean the bought a computer isn't interested in such things.  To 
give a specific example, it was only about 5 years ago I got my father off of 
Office 95 and that took pressure.  He saw no reason to change something that 
was 
doing what he needed.

If Ubuntu wants to hit the mass market, it needs to find ways to be better, not 
ways to be the same, but cheaper.  Personally, while I know that application 
developers get grumpy about it, I think that the stable model with periodic 
upgrades if desired is something most users would like better than what they 
experience with other O/S's.  It's still good to find ways to make the flow of 
fixing actual bugs work better, but I don't think there's a huge market for 
getting every single application update and working through new sets of 
issues.

> None of those OSes offer application downgrades (though individual
> applications might). None of them migrate user settings to previous
> versions. ("The file “iTunes Library” cannot be read because it was
> created by a newer version of iTunes. Would you like to download
> iTunes now?") And absent that settings problem, OS X is the only one
> that makes multiple application versions moderately easy ("portable
> apps" on Windows being the exception rather than the rule). That tells
> me that while they might be desirable, none of those three things is a
> requirement for a mass-market OS.

It is also quite normal for things to get screwed up on other operating 
systems and people do a full reinstall to fix it.  One of the fundamental 
design principles behind package management in Debian and by extension Ubuntu 
is that once installed, systems can just be upgraded.  Reinstalls should not 
be needed.  I think this is an important feature that should not be lightly 
abandoned.  I think the implication of that is you've got to find a way to 
support downgrades.  I think it's feasible, perhaps using something like 
etckeeper, but it would take work.

Allowing areas where we beat the proprietary competition in quality/ 
capability to degrade to their level is not a recipe for success.

> > It doesn't give the user much to give them the ability to choose
> > when to upgrade a package if you don't also give them the ability
> > to change there mind if the experience is poor with the new
> > version.
> > 
> > ...
> 
> Even

Re: Proposing a New App Developer Upload Process

[Matthew Paul Thomas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Scott Kitterman wrote on 06/09/12 14:22:
> 
> Matthew Paul Thomas  wrote:
> 
> ...
>> Scott Kitterman wrote on 05/09/12 18:54:
> ...
>>> 
>>> - In this brave new world, what is the definition of "Ubuntu 
>>> itself"? If the application developers are unshackled then it 
>>> must be some subset of what we consider Ubuntu to be today.
>> 
>> The shell and everything that makes it work (down to the kernel 
>> and init system), the toolchain, official developer APIs and the 
>> software that implements them, and whichever apps are installed 
>> by default.
> ...
> 
> I suspect not everyone shares your definition of what Ubuntu 
> is/will be.  Even the I favor a more traditional scope myself, I'm 
> very surprised you didn't include the desktop environment (e.g. 
> Unity, Plasma, etc.).

That's what I meant by "the shell".

> I think it's critical that there be some shared vision of where we 
> are going and even though there won't be resources to do
> everything at once, a broad outline of the chunks of work needed to
> get there.
> 
> To pick just one example, rolling delivery of applications and 
> offering multiple versions of the same package (which is what the 
> BSD Unixes do, although not in a way I've personally found at all 
> satisfying as a user) implies huge changes in package management. 
> Not the least of which is the ability to support downgrades, 
> including migrating per user settings back to previous versions.

Windows, OS X, iOS, and Android all let an author issue a new or
updated application within a week or so of wanting to, which I guess
is what you mean by "rolling delivery". This whole discussion is
predicated on that being a requirement for a mass-market OS.

None of those OSes offer application downgrades (though individual
applications might). None of them migrate user settings to previous
versions. ("The file “iTunes Library” cannot be read because it was
created by a newer version of iTunes. Would you like to download
iTunes now?") And absent that settings problem, OS X is the only one
that makes multiple application versions moderately easy ("portable
apps" on Windows being the exception rather than the rule). That tells
me that while they might be desirable, none of those three things is a
requirement for a mass-market OS.

> It doesn't give the user much to give them the ability to choose 
> when to upgrade a package if you don't also give them the ability 
> to change there mind if the experience is poor with the new 
> version.
> 
> ...

Even if that's a problem worth solving, that doesn't necessarily mean
it's best solved at the same time as the app developer upload problem,
or that it even affects the solution for that problem. If you think
either is true, perhaps you could explain why, and describe how the
solutions might coexist.

- -- 
mpt
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlBJCTUACgkQ6PUxNfU6ecrpjwCfWGsn4/++Veu53znGt4Bh7kFp
PhsAnj+SRqJFv0z6dKqg030KuJ8jEaHF
=7jqA
-END PGP SIGNATURE-

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Thursday, September 06, 2012 11:31:45 AM Steve Langasek wrote:
> On Thu, Sep 06, 2012 at 01:38:09PM -0400, Scott Kitterman wrote:
> > You'll need to be at least somewhat careful about FHS reservations in
> > /opt.
> > /etc/opt is defined and subdirectories in it can be used, but directories
> > such as /opt/bin, /opt/lib, and /opt/man cannot.  Also, it's probably
> > worth mentioning that last I checked, extras.ubuntu.com was not LANANA
> > registered. Someone should do that.
> 
>   "Alternatives: Instead of applying for an LSB Provider Name, it is also
>   possible to use your fully qualified Internet domain name (FQDN in lower
>   case) to prefix your package and script names as described in the Software
> Installation and System Initialization chapters of the LSB Core
>   specification."
> 
> http://www.lanana.org/lsbreg/providers/index.html

Ah.  Good to know.  Thanks.  I missed that when I was looking into it.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Steve Langasek]

On Thu, Sep 06, 2012 at 01:38:09PM -0400, Scott Kitterman wrote:
> You'll need to be at least somewhat careful about FHS reservations in /opt.  
> /etc/opt is defined and subdirectories in it can be used, but directories 
> such 
> as /opt/bin, /opt/lib, and /opt/man cannot.  Also, it's probably worth 
> mentioning that last I checked, extras.ubuntu.com was not LANANA registered.  
> Someone should do that.

  "Alternatives: Instead of applying for an LSB Provider Name, it is also
  possible to use your fully qualified Internet domain name (FQDN in lower
  case) to prefix your package and script names as described in the Software
  Installation and System Initialization chapters of the LSB Core
  specification."

http://www.lanana.org/lsbreg/providers/index.html

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Thursday, September 06, 2012 01:29:45 PM Stéphane Graber wrote:
> On 12-09-06 12:12 PM, Steve Langasek wrote:
> > On Thu, Sep 06, 2012 at 03:35:50PM +0200, Stefano Rivera wrote:
> >> Hi ubuntu-devel (2012.09.06_15:31:14_+0200)
> >> 
> >>> The hooks just run a script provided by another package (in the
> >>> archive). It makes the decisions on how to collate things.
> >> 
> >> A (hopefully) clearer attempt to articulate this:
> >> 
> >> We make the extras packages entirely self-contained and namespaced.
> >> 
> >> Then, we provide some machinery outside them that handles collates
> >> things across extras packages. If there's some kind of conflict here,
> >> (although it should be avoidable), it's not an issue. It just results in
> >> a broken extras package. Not broken in a way that stops apt from
> >> working. And it doesn't break anything in the Ubuntu archive, only the
> >> conflicting extras packages.
> > 
> > There's no reason that any of this should be done in a postinst hook.  If
> > we already have a scheme to make the extras packages properly namespaced
> > with no conflicts, the same class of namespacing should be used as well
> > for the integration points (the shared directories), and the files should
> > all be shipped in the package.  "If there's a conflict" means it's
> > designed wrong, because this should be done in a way that there's never a
> > conflict.
> > 
> > It's completely achievable to have our packaging helper create the correct
> > symlinks automatically.  Compared to the work of getting the package
> > installed correctly in /opt/extras.u.c/$pkg, it's a piece of cake, even.
> 
> I agree that we shouldn't generate the symlinks from the maintainer
> scripts, instead we should just be enforcing the same filename
> requirement as the ARB is currently using for files outside of /opt.
> 
> That's prefixing any file outside of /opt/extras.ubuntu.com/ by
> _. As dpkg is ensuring we can't have two binary packages
> installed with the same name on the system, there's no potential
> conflict possible.
> 
> My opinion would be to have the following:
>  - All files in /opt/extras.ubuntu.com/
>  - Have a reserved directory in /opt/extras.ubuntu.com or directly in /opt/
>  - Use /opt/extras.ubuntu.com// or /opt// to contain
> symlinks to the desktop files, dbus services, unity lenses/scopes, ...
> adding that path at the end of the XDG_DATA_DIRS list. Any file in that
> path needs to be prefixed by "_".

You'll need to be at least somewhat careful about FHS reservations in /opt.  
/etc/opt is defined and subdirectories in it can be used, but directories such 
as /opt/bin, /opt/lib, and /opt/man cannot.  Also, it's probably worth 
mentioning that last I checked, extras.ubuntu.com was not LANANA registered.  
Someone should do that.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Stéphane Graber]

On 12-09-06 12:12 PM, Steve Langasek wrote:
> On Thu, Sep 06, 2012 at 03:35:50PM +0200, Stefano Rivera wrote:
>> Hi ubuntu-devel (2012.09.06_15:31:14_+0200)
>>> The hooks just run a script provided by another package (in the
>>> archive). It makes the decisions on how to collate things.
> 
>> A (hopefully) clearer attempt to articulate this:
> 
>> We make the extras packages entirely self-contained and namespaced.
> 
>> Then, we provide some machinery outside them that handles collates
>> things across extras packages. If there's some kind of conflict here,
>> (although it should be avoidable), it's not an issue. It just results in
>> a broken extras package. Not broken in a way that stops apt from
>> working. And it doesn't break anything in the Ubuntu archive, only the
>> conflicting extras packages.
> 
> There's no reason that any of this should be done in a postinst hook.  If we
> already have a scheme to make the extras packages properly namespaced with
> no conflicts, the same class of namespacing should be used as well for the
> integration points (the shared directories), and the files should all be
> shipped in the package.  "If there's a conflict" means it's designed wrong,
> because this should be done in a way that there's never a conflict.
> 
> It's completely achievable to have our packaging helper create the correct
> symlinks automatically.  Compared to the work of getting the package
> installed correctly in /opt/extras.u.c/$pkg, it's a piece of cake, even.


I agree that we shouldn't generate the symlinks from the maintainer
scripts, instead we should just be enforcing the same filename
requirement as the ARB is currently using for files outside of /opt.

That's prefixing any file outside of /opt/extras.ubuntu.com/ by
_. As dpkg is ensuring we can't have two binary packages
installed with the same name on the system, there's no potential
conflict possible.

My opinion would be to have the following:
 - All files in /opt/extras.ubuntu.com/
 - Have a reserved directory in /opt/extras.ubuntu.com or directly in /opt/
 - Use /opt/extras.ubuntu.com// or /opt// to contain
symlinks to the desktop files, dbus services, unity lenses/scopes, ...
adding that path at the end of the XDG_DATA_DIRS list. Any file in that
path needs to be prefixed by "_".

-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com



signature.asc
Description: OpenPGP digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Steve Langasek]

On Thu, Sep 06, 2012 at 03:35:50PM +0200, Stefano Rivera wrote:
> Hi ubuntu-devel (2012.09.06_15:31:14_+0200)
> > The hooks just run a script provided by another package (in the
> > archive). It makes the decisions on how to collate things.

> A (hopefully) clearer attempt to articulate this:

> We make the extras packages entirely self-contained and namespaced.

> Then, we provide some machinery outside them that handles collates
> things across extras packages. If there's some kind of conflict here,
> (although it should be avoidable), it's not an issue. It just results in
> a broken extras package. Not broken in a way that stops apt from
> working. And it doesn't break anything in the Ubuntu archive, only the
> conflicting extras packages.

There's no reason that any of this should be done in a postinst hook.  If we
already have a scheme to make the extras packages properly namespaced with
no conflicts, the same class of namespacing should be used as well for the
integration points (the shared directories), and the files should all be
shipped in the package.  "If there's a conflict" means it's designed wrong,
because this should be done in a way that there's never a conflict.

It's completely achievable to have our packaging helper create the correct
symlinks automatically.  Compared to the work of getting the package
installed correctly in /opt/extras.u.c/$pkg, it's a piece of cake, even.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Thursday, September 06, 2012 09:28:58 AM Michael Hall wrote:
> On 09/06/2012 09:22 AM, Scott Kitterman wrote:
> > Matthew Paul Thomas  wrote:
> >> -BEGIN PGP SIGNED MESSAGE-
> >> Hash: SHA1
> >> 
> >> Scott Kitterman wrote on 05/09/12 18:54:
> >>> Matthew Paul Thomas  wrote: ...
> >>> 
>  That's a near-tautology. "Distributions" are named after the
>  assumption that selecting and packaging other people's software
>  is a way to produce a useful operating system.
>  
>  That may work for a few hundred thousand or even a few million
>  notebook/desktop users, but it has failed to grow beyond that.
>  The distro model discourages application developers, slows
>  application updates (making the installed base less reliable and
>  less secure), and distracts Ubuntu developers from improving
>  Ubuntu itself. Eventually the time comes to say "enough, let's
>  try something else".
> >>> 
> >>> This though is a much bigger step than just providing an easy
> >>> entry point for additional apps.
> >> 
> >> Sure. It's necessary but not sufficient.
> >> 
> >>> It brings in many fundamental questions that need to be answered
> >>> before we can really start to proceed:
> >>> 
> >>> - In this brave new world, what is the definition of "Ubuntu
> >>> itself"? If the application developers are unshackled then it must
> >>> be some subset of what we consider Ubuntu to be today.
> >> 
> >> The shell and everything that makes it work (down to the kernel and
> >> init system), the toolchain, official developer APIs and the software
> >> that implements them, and whichever apps are installed by default.
> >> 
> >>> - How do we provide stability for users that are more interested
> >>> in using what they have than the latest upstream crack that was
> >>> pushed out the door at 2am last night?
> >> 
> >> By presenting application updates as opt-in rather than opt-out.
> >> 
> >> 
> >>> - How do we deal with library transitions?
> >> 
> >> Others on this list can answer that far better than I can.
> >> 
> >>> - If application author s are getting unleashed, why can't library
> >>> authors get unleashed too?
> >> 
> >> Because those are mutually exclusive (at least for libraries that are
> >> part of Ubuntu itself), and on a successful platform application
> >> authors are far more numerous and distributed.
> >> 
> >> A library that wasn't part of Ubuntu itself could be unleashed in the
> >> same way, but it would be application authors' responsibility to judge
> >> how serious the library author was about backward compatibility.
> >> 
> >>> - What about packages that provide both applications and
> >>> libraries?
> >> 
> >> They would have to play by library rules: preserve backward
> >> compatibility, and/or not be part of the official APIs.
> >> 
> >>> - What does it mean to be a distribution?
> >> 
> >> A distribution is to an operating system as a runway is to a flight.
> >> It's the expensive but vital buildup of momentum before takeoff.
> >> 
> >>> ...
> >>> 
>  There are a lot of developers involved in packaging, compared to
>  what? Two years ago there were 160 MOTUs. Today there are 149.
>  That isn't how you scale to an order of magnitude more
>  applications.
> >>> 
> >>> Certainly, but that's also the result of a determined effort to
> >>> kill off MOTU from which that community has never recovered.  There
> >>> is some good work going on now to bring in new blood, so I expect
> >>> the numbers to start to improve.
> >> 
> >> I'm surprised to read of "a determined effort to kill off MOTU", but
> >> if those numbers increase then good. MOTUs will be vital for years to
> >> come.
> >> 
> >>> I do agree that we need something different to scale to an order
> >>> of magnitude more applications.  I don't agree that doing that
> >>> particularly solves any problems we're having.  I can't remember
> >>> the last time I wanted a tool to do a job and there wasn't one
> >>> handy, with the exception of cases where a free software solution
> >>> wasn't legally possible.
> >> 
> >> That's great, but not particularly telling, because if it wasn't true
> >> perhaps you'd no longer be here to discuss this.
> >> 
>  Maybe the current proposal isn't the best way to solve the
>  problem. But the first step is to recognize the problem.
> >>> 
> >>> I understand the problem statement.  To the extent there are real
> >>> problems (e.g. key applications out of date), this proposal is
> >>> barely the tip of the iceberg of what would need to be addressed to
> >>> make the transition to a model where we outsource that to
> >>> application developers.
> >>> 
> >>> ...
> >> 
> >> So, assume that we can't do everything at once, but we want to act as
> >> soon as possible. What do you suggest we do as well, or instead?
> > 
> > I suspect not everyone shares your definition of what Ubuntu is/will be. 
> > Even the I favor

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/06/2012 09:22 AM, Scott Kitterman wrote:
> 
> 
> Matthew Paul Thomas  wrote:
> 
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>
>> Scott Kitterman wrote on 05/09/12 18:54:
>>>
>>> Matthew Paul Thomas  wrote: ...

 That's a near-tautology. "Distributions" are named after the 
 assumption that selecting and packaging other people's software
 is a way to produce a useful operating system.

 That may work for a few hundred thousand or even a few million 
 notebook/desktop users, but it has failed to grow beyond that.
 The distro model discourages application developers, slows
 application updates (making the installed base less reliable and
 less secure), and distracts Ubuntu developers from improving
 Ubuntu itself. Eventually the time comes to say "enough, let's
 try something else".
>>>
>>> This though is a much bigger step than just providing an easy
>>> entry point for additional apps.
>>
>> Sure. It's necessary but not sufficient.
>>
>>> It brings in many fundamental questions that need to be answered 
>>> before we can really start to proceed:
>>>
>>> - In this brave new world, what is the definition of "Ubuntu
>>> itself"? If the application developers are unshackled then it must
>>> be some subset of what we consider Ubuntu to be today.
>>
>> The shell and everything that makes it work (down to the kernel and
>> init system), the toolchain, official developer APIs and the software
>> that implements them, and whichever apps are installed by default.
>>
>>> - How do we provide stability for users that are more interested
>>> in using what they have than the latest upstream crack that was
>>> pushed out the door at 2am last night?
>>
>> By presenting application updates as opt-in rather than opt-out.
>> 
>>
>>> - How do we deal with library transitions?
>>
>> Others on this list can answer that far better than I can.
>>
>>> - If application author s are getting unleashed, why can't library 
>>> authors get unleashed too?
>>
>> Because those are mutually exclusive (at least for libraries that are
>> part of Ubuntu itself), and on a successful platform application
>> authors are far more numerous and distributed.
>>
>> A library that wasn't part of Ubuntu itself could be unleashed in the
>> same way, but it would be application authors' responsibility to judge
>> how serious the library author was about backward compatibility.
>>
>>> - What about packages that provide both applications and
>>> libraries?
>>
>> They would have to play by library rules: preserve backward
>> compatibility, and/or not be part of the official APIs.
>>
>>> - What does it mean to be a distribution?
>>
>> A distribution is to an operating system as a runway is to a flight.
>> It's the expensive but vital buildup of momentum before takeoff.
>>
>>> ...

 There are a lot of developers involved in packaging, compared to 
 what? Two years ago there were 160 MOTUs. Today there are 149.
 That isn't how you scale to an order of magnitude more
 applications.
>>>
>>> Certainly, but that's also the result of a determined effort to
>>> kill off MOTU from which that community has never recovered.  There
>>> is some good work going on now to bring in new blood, so I expect
>>> the numbers to start to improve.
>>
>> I'm surprised to read of "a determined effort to kill off MOTU", but
>> if those numbers increase then good. MOTUs will be vital for years to
>> come.
>>
>>> I do agree that we need something different to scale to an order
>>> of magnitude more applications.  I don't agree that doing that 
>>> particularly solves any problems we're having.  I can't remember
>>> the last time I wanted a tool to do a job and there wasn't one
>>> handy, with the exception of cases where a free software solution
>>> wasn't legally possible.
>>
>> That's great, but not particularly telling, because if it wasn't true
>> perhaps you'd no longer be here to discuss this.
>>
 Maybe the current proposal isn't the best way to solve the
 problem. But the first step is to recognize the problem.
>>>
>>> I understand the problem statement.  To the extent there are real 
>>> problems (e.g. key applications out of date), this proposal is
>>> barely the tip of the iceberg of what would need to be addressed to
>>> make the transition to a model where we outsource that to
>>> application developers.
>>>
>>> ...
>>
>> So, assume that we can't do everything at once, but we want to act as
>> soon as possible. What do you suggest we do as well, or instead?
> 
> I suspect not everyone shares your definition of what Ubuntu is/will be.  
> Even the I favor a more traditional scope myself, I'm very surprised you 
> didn't include the desktop environment (e.g. Unity, Plasma, etc.).
> 
> I think it's critical that there be some shared vision of where we are going 
> and even though there won't be resources to do everything at once, a broad 
> o

Re: Proposing a New App Developer Upload Process

[Stefano Rivera]

Hi ubuntu-devel (2012.09.06_15:31:14_+0200)
> The hooks just run a script provided by another package (in the
> archive). It makes the decisions on how to collate things.

A (hopefully) clearer attempt to articulate this:

We make the extras packages entirely self-contained and namespaced.

Then, we provide some machinery outside them that handles collates
things across extras packages. If there's some kind of conflict here,
(although it should be avoidable), it's not an issue. It just results in
a broken extras package. Not broken in a way that stops apt from
working. And it doesn't break anything in the Ubuntu archive, only the
conflicting extras packages.

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  H: +27 21 461 1230 C: +27 72 419 8559

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Stefano Rivera]

Hi Michael (2012.09.06_15:12:40_+0200)
> No, but if you have two packages with post-install hooks trying to put a
> symlink in the same place, one of them is going to lose.

The hooks just run a script provided by another package (in the
archive). It makes the decisions on how to collate things.
This is a standard practice amongst groups of packages that need similar
postinst actions.

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  H: +27 21 461 1230 C: +27 72 419 8559

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

Matthew Paul Thomas  wrote:

>-BEGIN PGP SIGNED MESSAGE-
>Hash: SHA1
>
>Scott Kitterman wrote on 05/09/12 18:54:
>> 
>> Matthew Paul Thomas  wrote: ...
>>> 
>>> That's a near-tautology. "Distributions" are named after the 
>>> assumption that selecting and packaging other people's software
>>> is a way to produce a useful operating system.
>>> 
>>> That may work for a few hundred thousand or even a few million 
>>> notebook/desktop users, but it has failed to grow beyond that.
>>> The distro model discourages application developers, slows
>>> application updates (making the installed base less reliable and
>>> less secure), and distracts Ubuntu developers from improving
>>> Ubuntu itself. Eventually the time comes to say "enough, let's
>>> try something else".
>> 
>> This though is a much bigger step than just providing an easy
>> entry point for additional apps.
>
>Sure. It's necessary but not sufficient.
>
>> It brings in many fundamental questions that need to be answered 
>> before we can really start to proceed:
>> 
>> - In this brave new world, what is the definition of "Ubuntu
>> itself"? If the application developers are unshackled then it must
>> be some subset of what we consider Ubuntu to be today.
>
>The shell and everything that makes it work (down to the kernel and
>init system), the toolchain, official developer APIs and the software
>that implements them, and whichever apps are installed by default.
>
>> - How do we provide stability for users that are more interested
>> in using what they have than the latest upstream crack that was
>> pushed out the door at 2am last night?
>
>By presenting application updates as opt-in rather than opt-out.
>
>
>> - How do we deal with library transitions?
>
>Others on this list can answer that far better than I can.
>
>> - If application author s are getting unleashed, why can't library 
>> authors get unleashed too?
>
>Because those are mutually exclusive (at least for libraries that are
>part of Ubuntu itself), and on a successful platform application
>authors are far more numerous and distributed.
>
>A library that wasn't part of Ubuntu itself could be unleashed in the
>same way, but it would be application authors' responsibility to judge
>how serious the library author was about backward compatibility.
>
>> - What about packages that provide both applications and
>> libraries?
>
>They would have to play by library rules: preserve backward
>compatibility, and/or not be part of the official APIs.
>
>> - What does it mean to be a distribution?
>
>A distribution is to an operating system as a runway is to a flight.
>It's the expensive but vital buildup of momentum before takeoff.
>
>> ...
>>> 
>>> There are a lot of developers involved in packaging, compared to 
>>> what? Two years ago there were 160 MOTUs. Today there are 149.
>>> That isn't how you scale to an order of magnitude more
>>> applications.
>> 
>> Certainly, but that's also the result of a determined effort to
>> kill off MOTU from which that community has never recovered.  There
>> is some good work going on now to bring in new blood, so I expect
>> the numbers to start to improve.
>
>I'm surprised to read of "a determined effort to kill off MOTU", but
>if those numbers increase then good. MOTUs will be vital for years to
>come.
>
>> I do agree that we need something different to scale to an order
>> of magnitude more applications.  I don't agree that doing that 
>> particularly solves any problems we're having.  I can't remember
>> the last time I wanted a tool to do a job and there wasn't one
>> handy, with the exception of cases where a free software solution
>> wasn't legally possible.
>
>That's great, but not particularly telling, because if it wasn't true
>perhaps you'd no longer be here to discuss this.
>
>>> Maybe the current proposal isn't the best way to solve the
>>> problem. But the first step is to recognize the problem.
>> 
>> I understand the problem statement.  To the extent there are real 
>> problems (e.g. key applications out of date), this proposal is
>> barely the tip of the iceberg of what would need to be addressed to
>> make the transition to a model where we outsource that to
>> application developers.
>> 
>> ...
>
>So, assume that we can't do everything at once, but we want to act as
>soon as possible. What do you suggest we do as well, or instead?

I suspect not everyone shares your definition of what Ubuntu is/will be.  Even 
the I favor a more traditional scope myself, I'm very surprised you didn't 
include the desktop environment (e.g. Unity, Plasma, etc.).

I think it's critical that there be some shared vision of where we are going 
and even though there won't be resources to do everything at once, a broad 
outline of the chunks of work needed to get there.

To pick just one example, rolling delivery of applications and offering 
multiple versions of the same package (which is what the BSD Unixes do, 

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/06/2012 04:33 AM, Stefano Rivera wrote:
> Hi Michael (2012.09.05_23:56:04_+0200)
>> Yes, or we could just allow the direct installation into
>> /opt/extras.ubuntu.com/share/applications/.  But in doing so, we bring
>> back the possibility for file conflicts because we're not isolating
>> Extras apps from each other.
> 
> They aren't conceptually the same. If the file isn't in the package,
> dpkg isn't going to see it as a conflict, and our collation tool can be
> smart about it.
> 
> SR
> 

No, but if you have two packages with post-install hooks trying to put a
symlink in the same place, one of them is going to lose.

Michael Hall
mhall...@ubuntu.com

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Matthew Paul Thomas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Scott Kitterman wrote on 05/09/12 18:54:
> 
> Matthew Paul Thomas  wrote: ...
>> 
>> That's a near-tautology. "Distributions" are named after the 
>> assumption that selecting and packaging other people's software
>> is a way to produce a useful operating system.
>> 
>> That may work for a few hundred thousand or even a few million 
>> notebook/desktop users, but it has failed to grow beyond that.
>> The distro model discourages application developers, slows
>> application updates (making the installed base less reliable and
>> less secure), and distracts Ubuntu developers from improving
>> Ubuntu itself. Eventually the time comes to say "enough, let's
>> try something else".
> 
> This though is a much bigger step than just providing an easy
> entry point for additional apps.

Sure. It's necessary but not sufficient.

> It brings in many fundamental questions that need to be answered 
> before we can really start to proceed:
> 
> - In this brave new world, what is the definition of "Ubuntu
> itself"? If the application developers are unshackled then it must
> be some subset of what we consider Ubuntu to be today.

The shell and everything that makes it work (down to the kernel and
init system), the toolchain, official developer APIs and the software
that implements them, and whichever apps are installed by default.

> - How do we provide stability for users that are more interested
> in using what they have than the latest upstream crack that was
> pushed out the door at 2am last night?

By presenting application updates as opt-in rather than opt-out.


> - How do we deal with library transitions?

Others on this list can answer that far better than I can.

> - If application author s are getting unleashed, why can't library 
> authors get unleashed too?

Because those are mutually exclusive (at least for libraries that are
part of Ubuntu itself), and on a successful platform application
authors are far more numerous and distributed.

A library that wasn't part of Ubuntu itself could be unleashed in the
same way, but it would be application authors' responsibility to judge
how serious the library author was about backward compatibility.

> - What about packages that provide both applications and
> libraries?

They would have to play by library rules: preserve backward
compatibility, and/or not be part of the official APIs.

> - What does it mean to be a distribution?

A distribution is to an operating system as a runway is to a flight.
It's the expensive but vital buildup of momentum before takeoff.

> ...
>> 
>> There are a lot of developers involved in packaging, compared to 
>> what? Two years ago there were 160 MOTUs. Today there are 149.
>> That isn't how you scale to an order of magnitude more
>> applications.
> 
> Certainly, but that's also the result of a determined effort to
> kill off MOTU from which that community has never recovered.  There
> is some good work going on now to bring in new blood, so I expect
> the numbers to start to improve.

I'm surprised to read of "a determined effort to kill off MOTU", but
if those numbers increase then good. MOTUs will be vital for years to
come.

> I do agree that we need something different to scale to an order
> of magnitude more applications.  I don't agree that doing that 
> particularly solves any problems we're having.  I can't remember
> the last time I wanted a tool to do a job and there wasn't one
> handy, with the exception of cases where a free software solution
> wasn't legally possible.

That's great, but not particularly telling, because if it wasn't true
perhaps you'd no longer be here to discuss this.

>> Maybe the current proposal isn't the best way to solve the
>> problem. But the first step is to recognize the problem.
> 
> I understand the problem statement.  To the extent there are real 
> problems (e.g. key applications out of date), this proposal is
> barely the tip of the iceberg of what would need to be addressed to
> make the transition to a model where we outsource that to
> application developers.
> 
> ...

So, assume that we can't do everything at once, but we want to act as
soon as possible. What do you suggest we do as well, or instead?

- -- 
mpt

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlBIgEMACgkQ6PUxNfU6ecpzrgCfYKFQU8BFj8bqAGZIVcKtalOV
lOIAn2Ravn4zOBwxmDQY9SuxVeQVvzv/
=IY8E
-END PGP SIGNATURE-

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Didier Roche wrote:
> Le 06/09/2012 10:20, Emmet Hikory a écrit :
> > Given the current shape of the package, and the belief that packaging
> >of this package could mostly safely be completely automated, would you
> >expect that future uploads would be broken in some way so that these
> >developers would be incapable of performing the maintenance?
> I think they should be able if I teach them about debian/changelog,
> dpkg-buildpackage, dput, gpg keys, signing the CoC… So even after
> the first upload, if they have PPU access, the bar to upload a new
> revision will still be high, even if there is no packaging changes
> to do.

This makes sense, although I think that we'd want to ask folk to sign
the CoC anyway, if for nothing else than to have their documented agreement
to participate in our dispute resolution processes, for those exceptional
cases where a problem occurs.  Despite my agreement, I suspect that there
is significant scope for improvement of this hurdle through automation
without also forcing all participants to be subject to significant
limitations on the functionality they are permitted to provide, and believe
that as we develop the automation to support scalable automatic acception
of applications, we should do so in a way that permits us to use what
portions of that automation might reduce our educational requirements
independently of our automatic acceptance system.

> In addition to that, as you told, the packaging is now done, but it
> took months for me to take some spare time to get to their package
> because of the queue of people asking me to package a new software
> is high, and I have other work to do for my direct work/life… So if
> relying to a human review or human first packaging from upstream
> will always take a lot of time (despite heroic effort on REVU, on
> the ARB, sponsoring queue…) and will give them a suboptimal first
> experience to deliver their application to ubuntu.

Absolutely.  My apologies if I gave the impression that this should
be the typical model of inclusion - my intention was to use it as an
example of a package where there had been time for human review (which
we should only expect to be the case for a tiny minority of pacakges if
we wish to scale broadly).

> [F]or applications, I think we shouldn't make everyone paying
> the high learning curve to developers. If they want to focus on
> packaging, we can direct them to this process, but if they don't, we
> need an easy way which have different tools, processes and
> reactivity. Those would of course, not cover the whole set of cases
> we can into the distro, but will deliver a snappy experience, which
> as much automation as possible on both packaging and server-side
> checking, in addition to insulation to protect our users.

My concern here is that by maintaining entirely parallel processes,
we create a situation where those whose software is unsuitable for the
streamlined process must begin again, essentially from scratch, in order
to deliver their software to our users.  I'd much prefer that in these
exceptional cases we had an integrated mechanism to assist the relevant
developers with the delivery of their package - to be able to provide
human review (resources permitting) and relax insulation restrictions
without ever needing to give the developers any message that could be
interpreted as rejection.  Telling an application developer that their
application is special, and that we apologize, but they will either have
to wait for human review or limit the functionality of their software is
a positive and reinforcing message.  Telling the same application developer
that their software cannot be accepted by this method, and they should
go chat someone up on IRC, or repackage the software and resubmit it to
some entirely other queue for which we won't hold their hand is less so,
and may well cause those very developers whose applications would most
enhance Ubuntu to discredit the process.

Similarly, I would hope that as tools are developed to further automate
the packaging process, these tools could be made available also to our
developers, who might use them in producing packages intended for the
current distribution mechanisms (where that distribution developer is
presumably willing to provide human review for their own pacakge, but does
appreciate the automation of the intial work).

> I strongly think that different process and scopes will need different
> tools and so, delivery mechanism. In addition to that, the tools
> will evolute quite quickly at the beginning, and we need to be able
> to change those without risking/impacting the distro and ubuntu
> developers.

While I agree that we will need additional tools and an additional
delivery mechanism to support arbitrary application developers interacting
with an automated system to provide insulated pacakges to our users, I
do not believe we need a "different process", rather I submit that we
should rather be extending

Re: Proposing a New App Developer Upload Process

[Didier Roche]

Le 06/09/2012 10:20, Emmet Hikory a écrit :

Didier Roche wrote:

Le 05/09/2012 20:20, Emmet Hikory a écrit :

Didier Roche wrote:

I don't agree it's only for low quality apps. More than once, people
asked me to package their apps into ubuntu. This is particularly
annoying when I have no interest at all in the package itself. The
last case is pyromaths, which turned out to be quite popular in
schools. It's definitively a quality apps and I kind of regret to
upload it to ubuntu proper instead of letting them deal with myapps
for now as I have to admit I won't do a good job tracking them.

 In this case, do you see any reason that the developers of this
package should not be able to be granted PPU access to keep their
software well maintained in the archive now that it has been packaged,
rather than being subjected to arbitrary restrictions on what the
software may be allowed to do?

Because those developers are not interested at all at learning
packaging. Today, the package is IMHO in a good shape as I've done
the first pass on it.
However, their application is simple enough that this is typically
the kind of application which can be package by python-mkdebian and
I guess pkgme?).

Those won't produce perfect package of course (which is the standard
we even have for universe), but workable upstream component that I'm
totally confortable with (with insulation if the all process is
automated without double checking).

 Given the current shape of the package, and the belief that packaging
of this package could mostly safely be completely automated, would you
expect that future uploads would be broken in some way so that these
developers would be incapable of performing the maintenance?
I think they should be able if I teach them about debian/changelog, 
dpkg-buildpackage, dput, gpg keys, signing the CoC… So even after the 
first upload, if they have PPU access, the bar to upload a new revision 
will still be high, even if there is no packaging changes to do.


In addition to that, as you told, the packaging is now done, but it took 
months for me to take some spare time to get to their package because of 
the queue of people asking me to package a new software is high, and I 
have other work to do for my direct work/life… So if relying to a human 
review or human first packaging from upstream will always take a lot of 
time (despite heroic effort on REVU, on the ARB, sponsoring queue…) and 
will give them a suboptimal first experience to deliver their 
application to ubuntu.



   Not being
perfect is fine (we have *lots* of packaging bugs; while we want all
the packages to be perfect, this is a matter for ongoing effort, rather
than a hard goal that must be met prior to distribution - the perception
that it must be perfet to distribute likely stems from efforts by various
reviewers to address as many packaging bugs as possible before distribution
in the hopes of reducing later bugfix effort).


 If so, what do you think would be required for you to feel
comfortable endorsing such a grant of access?  If not, why should
any Ubuntu Developer refrain from such an endorsement if they
consider the application well-packaged (either as a result of
their collaboration or review of the packaging)?

 I still don't understand what you believe would be required for you
to feel comfortable with the original developers maintaining their tool
in our repositories.  Is it that you would want them to learn packaging
in a general sense, or am I overinterpreting your comments above?
We have multiple tools, process for uploading packages to the distro. 
Let's agree that most of them are suboptimal and takes time to grap them 
(I'm always amazed when I explain basics of packaging to newcomers 
either at Canonical or to mates how we succeeded to make things that 
ought to be simple really complicated). And I think that's ok: debian 
inheritage showed and benefited us with years of experience, tackling 
all the corner cases to deliver a strong a coherent distribution, and so 
we need that complexity, especially for low level plumbing and librairies.


However, for applications, I think we shouldn't make everyone paying the 
high learning curve to developers. If they want to focus on packaging, 
we can direct them to this process, but if they don't, we need an easy 
way which have different tools, processes and reactivity. Those would of 
course, not cover the whole set of cases we can into the distro, but 
will deliver a snappy experience, which as much automation as possible 
on both packaging and server-side checking, in addition to insulation to 
protect our users.





The MyApps story is to avoid those 2 pitfalls to occur:
* having ubuntu developers working on what they want to work on and
focus their effort on that, following the components they selected
closely and be able to help them.
* having application developers being able to have the control where
they should have: their own applications and decide whe

Re: Proposing a New App Developer Upload Process

[Stefano Rivera]

Hi Michael (2012.09.05_23:56:04_+0200)
> Yes, or we could just allow the direct installation into
> /opt/extras.ubuntu.com/share/applications/.  But in doing so, we bring
> back the possibility for file conflicts because we're not isolating
> Extras apps from each other.

They aren't conceptually the same. If the file isn't in the package,
dpkg isn't going to see it as a conflict, and our collation tool can be
smart about it.

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  H: +27 21 461 1230 C: +27 72 419 8559

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Didier Roche wrote:
> Le 05/09/2012 20:20, Emmet Hikory a écrit :
> >Didier Roche wrote:
> >>I don't agree it's only for low quality apps. More than once, people
> >>asked me to package their apps into ubuntu. This is particularly
> >>annoying when I have no interest at all in the package itself. The
> >>last case is pyromaths, which turned out to be quite popular in
> >>schools. It's definitively a quality apps and I kind of regret to
> >>upload it to ubuntu proper instead of letting them deal with myapps
> >>for now as I have to admit I won't do a good job tracking them.
> > In this case, do you see any reason that the developers of this
> >package should not be able to be granted PPU access to keep their
> >software well maintained in the archive now that it has been packaged,
> >rather than being subjected to arbitrary restrictions on what the
> >software may be allowed to do?
> 
> Because those developers are not interested at all at learning
> packaging. Today, the package is IMHO in a good shape as I've done
> the first pass on it.
> However, their application is simple enough that this is typically
> the kind of application which can be package by python-mkdebian and
> I guess pkgme?).
> 
> Those won't produce perfect package of course (which is the standard
> we even have for universe), but workable upstream component that I'm
> totally confortable with (with insulation if the all process is
> automated without double checking).

Given the current shape of the package, and the belief that packaging
of this package could mostly safely be completely automated, would you
expect that future uploads would be broken in some way so that these
developers would be incapable of performing the maintenance?  Not being
perfect is fine (we have *lots* of packaging bugs; while we want all
the packages to be perfect, this is a matter for ongoing effort, rather
than a hard goal that must be met prior to distribution - the perception
that it must be perfet to distribute likely stems from efforts by various
reviewers to address as many packaging bugs as possible before distribution
in the hopes of reducing later bugfix effort).

> > If so, what do you think would be required for you to feel
> >comfortable endorsing such a grant of access?  If not, why should
> >any Ubuntu Developer refrain from such an endorsement if they
> >consider the application well-packaged (either as a result of
> >their collaboration or review of the packaging)?

I still don't understand what you believe would be required for you
to feel comfortable with the original developers maintaining their tool
in our repositories.  Is it that you would want them to learn packaging
in a general sense, or am I overinterpreting your comments above?

> >>The MyApps story is to avoid those 2 pitfalls to occur:
> >>* having ubuntu developers working on what they want to work on and
> >>focus their effort on that, following the components they selected
> >>closely and be able to help them.
> >>* having application developers being able to have the control where
> >>they should have: their own applications and decide when they
> >>update. We can enable them to update as long as they do no harm to
> >>the platform (like in the file conflict case, and many other use
> >>cases requiring insulation). The ultimate goal is that all the
> >>packaging part is helped/assisted to them: it's not because I want
> >>to upload my application to ubuntu that I have to become a packager
> >>and know every detail of the debian policy. I just want to deliver
> >>my application to users.
> > While these are both laudable goals, I don't understand why there
> >needs to be such a firm separation between "packages that are in Ubuntu"
> >and "packages that upstream authors may update".  While I am in favor
> >of taking care to insulate our users from potentially malicious packages
> >in the presence of a completely automated acceptance process, I expect
> >that the vast majority of software authors would also be perfectly happy
> >to be able to upload their software directly after receiving some manual
> >review or assistance from someone knowledgeable about packaging, and I
> >believe that we both can and should attempt to enable as many as we feel
> >we can trust to do just this, rather than relegating them to some more
> >limited packaging area just because we don't want to be personally
> >responsible for the package.
> >
> I don't think we want right now touching soyuz and other distro
> components to introduce new services like automatic packaging, web
> frontend and other parts that myapps provides. I don't think we
> should have the upstream developer having to generate a source
> package on their machine, then knowing about dput to push it into
> the distro, our freezes, and other processes.

Absolutely.  I entirely agree.  This in no way helps me understand why
there needs to be the separation.  For those packages for which we can spare
initial huma

Re: Proposing a New App Developer Upload Process

[Didier Roche]

Le 05/09/2012 20:19, Micah Gersten a écrit :

On 09/05/2012 11:46 AM, Didier Roche wrote:

Le 05/09/2012 16:26, David Planella a écrit :

Al 05/09/12 14:29, En/na Scott Kitterman ha escrit:

David Planella  wrote:


* File name conflicts: there I would suggest exploring Daniel's
proposal
of relying on a conflict checker that works across all archives, so
that
before an upload is accepted this service checks for any potential
clashes and informs the uploader to fix the package before they can do
the next submission. The uploader would either be an Ubuntu developer
(through the main archive) or an app developer (through extras, via
MyApps). This would not only benefit the app developer process, but
also
fix the existing issue in the regular Ubuntu upload process.

This would be useful, but insufficient.


Could you elaborate on why you think it would be insufficient and what
alternative you believe would be a solution for file name conflicts in
this context?


* Namespace ownership: even with conflict checking there is the issue
of
who gets to own a particular file name or namespace. E.g. would "Mad
Feathered Creatures" (/usr/bin/birds, from Extras) have priority in
owning the binary's name if it had been submitted before "Jolly Flying
Animals" (also /usr/bin/birds, from Universe)? I think if we want to
make apps first-class citizens, even if not part of the distro, a
simple
suggestion would just be to do it on a first-come-first-serve basis.

I think it is fundamentally incorrect to give something built on
Ubuntu namespace priority over Ubuntu itself. Additionally, if this
service proves popular, this approach would drive a permanent
namespace wedge between Debian and Ubuntu that might, over time,
significanly change the nature of the relationship between the two
distributions.


I can see the point of the namespace wedge if we become immensely
popular. What do you think the alternatives are?


What are your thoughts on these?

Finally, I believe we do need to provision for those cases, but my
understanding is that the potential amount of occurrences would be
rather low. So do you think they justify additional Engineering work,
or
rather they could be dealt manually on a case-by-case basis?


You've got to consider it now or it won't scale.  IIRC the original
presentations on this topic at UDS Orlando(?), the intent was to be
able to scale out to hit numbers of applications equal to or greater
than the Apple Appstore/Google Play.  If you hit that, then MyApps
ends up being several times the size of the Ubuntu archive.


And that's what we're doing right now. My only concern is not to block
on a situation that will concern just a fraction of the uploads, even at
a higher scale. That's what I'm trying to get a feel for.

For all those 3 namespacing/files issues, maybe we can think of a
simple solution.

I really like Daniel 's idea of a conflict-check-before-publish
service. One of the case that was raised on the thread is about "you
can't predict the future". What about that example taking back the
"Mad feathered creatures" shipping /usr/bin/birds
- in precise, only the apps from extras is there
- in quantal, we sync from debian, or upload directly to universe
"Jolly Flying Animals" which ships the same file (new package or update)
-> nothing happens at this stage (remember that extras is not opened)
- a little bit (like 3 weeks?) before opening extras, we run (and then
continuously run) the conflict-check-before-publish service. This one
will detect the new conflict between the two packages, and:
1. add a Conflicts: in "Mad feathered creatures" debian/control file
in extras against the package in universe.
2. will send an email to the app developer telling "hey, maybe not all
ubuntu user will be able to use your apps as there is this excellent
new application <…> into the archive"

At the extreme, if the component in conflict is a core component, as
the ubuntu archive have an higher priority than the extra one
(right?), then the core component will be preferred on dist-upgrade.
This has the advantage of:
- pushing the burden to the app developer, not to ubuntu developers
- avoid having to do conflicts/replaces on our side and so diverging
from debian
- by pushing the burden to the app developer, still having a automatic
update solution integrated into myapps, but mailing them, we ensure to
have people committed to their application in ubuntu

I think this solution would fit for what will really be and stay,
IMHO, a corner case. I doubt with all the precautions taken into the
naming and namespace that will happen with every cycles and the few
developers in that case will be warned and have time to react before
the extras opening. In case they don't react, we have the automatic
metadata addition in conflicts: which enables apt to deal with it.


Worrying about extras when extras opens for the release is not
sufficient.  People upgrading in the mean time will have file conflicts
between packages.  Also, once 

Re: Proposing a New App Developer Upload Process

[Didier Roche]

Le 05/09/2012 20:20, Emmet Hikory a écrit :

Didier Roche wrote:

I don't agree it's only for low quality apps. More than once, people
asked me to package their apps into ubuntu. This is particularly
annoying when I have no interest at all in the package itself. The
last case is pyromaths, which turned out to be quite popular in
schools. It's definitively a quality apps and I kind of regret to
upload it to ubuntu proper instead of letting them deal with myapps
for now as I have to admit I won't do a good job tracking them.

 In this case, do you see any reason that the developers of this
package should not be able to be granted PPU access to keep their
software well maintained in the archive now that it has been packaged,
rather than being subjected to arbitrary restrictions on what the
software may be allowed to do?


Because those developers are not interested at all at learning 
packaging. Today, the package is IMHO in a good shape as I've done the 
first pass on it.
However, their application is simple enough that this is typically the 
kind of application which can be package by python-mkdebian and I guess 
pkgme?).


Those won't produce perfect package of course (which is the standard we 
even have for universe), but workable upstream component that I'm 
totally confortable with (with insulation if the all process is 
automated without double checking).

 If so, what do you think would be required for you to feel
comfortable endorsing such a grant of access?  If not, why should
any Ubuntu Developer refrain from such an endorsement if they
consider the application well-packaged (either as a result of
their collaboration or review of the packaging)?


The MyApps story is to avoid those 2 pitfalls to occur:
* having ubuntu developers working on what they want to work on and
focus their effort on that, following the components they selected
closely and be able to help them.
* having application developers being able to have the control where
they should have: their own applications and decide when they
update. We can enable them to update as long as they do no harm to
the platform (like in the file conflict case, and many other use
cases requiring insulation). The ultimate goal is that all the
packaging part is helped/assisted to them: it's not because I want
to upload my application to ubuntu that I have to become a packager
and know every detail of the debian policy. I just want to deliver
my application to users.

 While these are both laudable goals, I don't understand why there
needs to be such a firm separation between "packages that are in Ubuntu"
and "packages that upstream authors may update".  While I am in favor
of taking care to insulate our users from potentially malicious packages
in the presence of a completely automated acceptance process, I expect
that the vast majority of software authors would also be perfectly happy
to be able to upload their software directly after receiving some manual
review or assistance from someone knowledgeable about packaging, and I
believe that we both can and should attempt to enable as many as we feel
we can trust to do just this, rather than relegating them to some more
limited packaging area just because we don't want to be personally
responsible for the package.

I don't think we want right now touching soyuz and other distro 
components to introduce new services like automatic packaging, web 
frontend and other parts that myapps provides. I don't think we should 
have the upstream developer having to generate a source package on their 
machine, then knowing about dput to push it into the distro, our 
freezes, and other processes.


Also, there is this file conflict checker and restrictions for namespace 
needed if we want to automatically accept those packages. That's why for 
those kind of applications developers, we should segregate their apps 
into a different silo than the main distro one, as they have different 
process and requirements.


Didier

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/05/2012 05:51 PM, Stefano Rivera wrote:
>> It's important to remember that when we say "support /opt/ properly"
>> what we really mean is "support /opt/extras.ubuntu.com/* properly".  We
>> aren't just using /opt/ as an install prefix, we making a different
>> install prefix for every single application, and that means that every
>> tool is going to need to consider every install prefix for every
>> application that might be installed.
> 
> Can this not be solved with a post-install-hooked script (added by
> dh_extras) that collates symlinks to everything in
> /opt/extras.ubuntu.com/{appname}/share/applications/{appname}.desktop
> in /opt/extras.ubuntu.com/share/applications?
> 
> Suddenly we only have one new path to look at. Not n paths.
> 
> SR
> 

Yes, or we could just allow the direct installation into
/opt/extras.ubuntu.com/share/applications/.  But in doing so, we bring
back the possibility for file conflicts because we're not isolating
Extras apps from each other.

Michael Hall
mhall...@ubuntu.com

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Allison Randal]

On 09/05/2012 10:12 AM, David Planella wrote:
> Al 05/09/12 17:49, En/na Allison Randal ha escrit:
>>
>> - Extras is a place to experiment with how the training wheels should
>> work, without disrupting the main archives.
>>
>> - Eventually Extras should go away, once we've extracted the best value
>> from the experiment.
> 
> That said, there is one part where I disagree: in considering Extras (if
> we identify Extras with the App Developer Process) an experiment or
> training ground for Ubuntu platform development, for which we already
> have a process in place.
> 
> I consider the App Developer Process simply a way to enable application
> authors to seamlessly and securely publish their apps. Human review and
> packaging policies have proven to be the main bottlenecks for this
> happening, and these (plus security) are the main points the proposal is
> trying to address.

Oh, yeah, I'm not saying we should chuck out App Developers. What I'm
saying is that if they have unique needs, over the long-term we need to
address those needs as part of the overall Ubuntu Developer strategy,
and not segregated off to the side. Similarly, if it turns out that some
type of lighter PPU rights with required sandboxing are valuable, then
maybe they belong in the wider Ubuntu Developer context too. (Last I
checked, the DMB was considering some kind of PPU without Membership,
which is another sort of lighter rights.)

I wouldn't say we've proven either idea yet. All Extras has really
proven so far is that packaging and reviewing packages is hard work,
even for simple apps. And, we pushed it farther than REVU, because we
did several rapid iterations on interfaces/tooling for manual reviews,
and still found it was way too much work and couldn't scale to handle a
rapid influx. That's a valuable lesson, and worth the experience.

I am really interested in the potential for automated sandboxing, and
right now Extras is the best place to experiment with it. And, if it
works well, then it could very well benefit other parts of Ubuntu, or
Debian, or any Debian-based distro. (I don't imagine the tools will be
general enough to handle other package formats. Not a requirement.)

Allison

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Scott Kitterman wrote:
> On Thursday, September 06, 2012 05:05:41 AM Emmet Hikory wrote:
> > Independently of the conflict resolution model, I still don't understand
> > why we would need to consider such a collection of software "external",
> > beyond the historical precedent: is there a reason we want to prevent an
> > interested Ubuntu Developer from fixing a bug in one of these packages? Do
> > we feel that having two clearly documented competing different processes
> > for the inclusion of software will help our users to be able to run the
> > software they prefer?  Do we want to prevent upstream developers from being
> > able to maintain their software as part of Ubuntu?  Should flavours be
> > restricted from seeding packages submitted by the original developers
> > unless they are willing to override the upstream submission?
> 
> This alternate submission model will have it's own tools, processes and 
> procedures.  I doubt that all Ubuntu developers will be sufficiently aware of 
> these that the ability to upload into the alternate process should be allowed 
> by default.

While I can accept this argument in terms of Ubuntu Developer access to
insulated packages and conversely for access by developers of insulated
packages to the Ubuntu archive, I believe you to be unnecessarily conflating
separable concepts by claiming it to be true for "[t]his alternate submission
model".  I would hope firstly that we don't end up arbitrarily rejecting
applications submitted through MyApps simply because they should instead be
submitted as system packages, and secondly that we could provide sufficient
education to our developers of the the nature and restrictions of some
collection of insulated software which we consider to be part of Ubuntu, but
not (or perhaps not yet) given the human review required to be uninsulated,
that it becomes safe to apply our regular upload permissions model.

If we do continue to consider it an external third-party repository,
albeit one we permit to be enabled by default during installation, then I
suggest we should take extra care to make sure that we have a well-discussed
mechanism to adopt these packages into the distribution itself (whether
insulated or not), as some of them may be ideal for seeding by some flavour,
and we most definitely do not want to create anything that could give the
impression that we are only willing to grant developers this sort of access
for applications that we believe to be unsuitable or unusable by default.

> It does raise an interesting question about the need for a group of extras 
> 'gardeners' (in the same sense MOTU are Ubuntu archive 'gardeners').  Outside 
> the ARB we don't have such a thing now, AFAIK, and we might want to get such 
> a 
> group if there are people interested.

Is this something the ARB could do independently, delegating upload
rights to extras to some group of interested folk who might volunteer to
assist upstream developers keep their applications well groomed, or would
it also require the involvement of other approval bodies?  If the former,
I would suggest that the ARB consider such a thing (and policies surrounding
it), and make a call for volunteers.  If the latter, I'd suggest that anyone
with interest in doing such a thing draft some guidelines for how it would
work, garner the review and approval of the ARB, and help get it approved
by whatever necessary hierarchy of authorities - even if we initially have
no members of such a group beyond the ARB, if we expect to scale to very
large numbers of applications, we will likely need more available hands
to assist with the various exception cases and I suspect we would all be
happier with random folk trusted by the ARB being granted these rights than
with the members of the ARB feeling overwhelmed with such administrative
work to the degree that they don't feel they have time for accepting new
packages or overseeing transitions between releases.

-- 
Emmet HIKORY

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

Michael Hall  wrote:

>
>On 09/05/2012 02:08 PM, Emmet Hikory wrote:
>> Regardless of our final decision with regard to the
>implementation
>> limiting the impact on users of potentially malicious automatically
>> reviewed applications, I think we ought continue to make efforts so
>> that our tools support /opt properly.  There are many existing
>popular
>> and successful free and commercial applications using /opt today for
>> other unices, and we can only benefit by reducing the porting effort
>> for this software to Ubuntu.
>> 
>
>It's important to remember that when we say "support /opt/ properly"
>what we really mean is "support /opt/extras.ubuntu.com/* properly".  We
>aren't just using /opt/ as an install prefix, we making a different
>install prefix for every single application, and that means that every
>tool is going to need to consider every install prefix for every
>application that might be installed.

That's normal for /opt.  The FHS standard layout for /opt is 
/opt/$VENDOR/$PACKAGE.  That's why /opt gives us filename namespace separation.

Scott K




-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Michael Hall wrote:
> On 09/05/2012 02:08 PM, Emmet Hikory wrote:
> > Regardless of our final decision with regard to the implementation
> > limiting the impact on users of potentially malicious automatically
> > reviewed applications, I think we ought continue to make efforts so
> > that our tools support /opt properly.  There are many existing popular
> > and successful free and commercial applications using /opt today for
> > other unices, and we can only benefit by reducing the porting effort
> > for this software to Ubuntu.
> > 
> 
> It's important to remember that when we say "support /opt/ properly"
> what we really mean is "support /opt/extras.ubuntu.com/* properly".  We
> aren't just using /opt/ as an install prefix, we making a different
> install prefix for every single application, and that means that every
> tool is going to need to consider every install prefix for every
> application that might be installed.

Actually, in the above context, I mean to add support for /opt/foo,
where foo represents both arbitrary values and arbitrary levels of
nesting.  This can easily be used to support /opt/e.u.c/bar/, but also
allows the use of /opt/getdeb.net/baz or /opt/autodesk.com/autocad/,
or any other arbitrarily identified third-party install location that
some software distributor wants to present to their users.

In terms of runtime tool support, rather than attempting to identify
all possible values of foo in applications that perform discovery, we
would do better to either investigate, improve, and use tools like
xdg-icon-resource to manage third-party application support, allow such
packages to add paths to some consolidated virtual filesystem via
some well-defined API, or other similar solution.  (Yes, this is another
potential source of namespace collisions, but one separated from the
base filesystem, and thereby in letter (if perhaps not entirely in
spirit), less likely to result in debian policy violations.

> But at the same time we're not currently making those app-specific
> install prefixes actual alternatives to /usr/, they don't contain
> /opt/extras.ubuntu.com/{appname}/share/applications/{appname}.desktop,
> or /opt/extras.ubuntu.com/{appname}/share/pixmaps/{appname}.png

Right, which I would argue is an incomplete implementation of the
FHS-compliant use of the /opt hierarchy.  I consider our incomplete
adoption and support of this to be a set of bugs (although I make no
claim that these are important or even significant bugs, depending on
our final determination on the implementation of application insulation).

> So even if we did make all of our tools use $XDG_DATA_DIRS, and we did
> include every installed Extras app in $XDG_DATA_DIRS, they still
> wouldn't work because we're not making those directories behave like an
> XDG_DATA_DIR should.

If we took the first two steps, I do not understand why we would not
then take the third, and I certainly don't understand why our failure to
take the third should be interpreted as a failure of the proposition,
rather than a failure of our resolve.  There do exist bugs for which the
simple use of XDG_DATA_DIRS is insufficient, the identification and
solution to which may cause us to decide that we do not wish to include
the restriction to /opt in our application insulation, but if we do
decide to use /opt as part of our solution, we should by all means expect
that applications built with an install path including /opt will not
require different configuration of the content, simply as a result of
the install location.  ${INSTALL_PATH}/share/quux should have the same
content in both situations, such that, for example, we may expect that
/opt/extras.ubuntu.com/feathered-friends/share/applications/ would
contain .desktop files that could be selectively processed by any
compliant xdg-menu implementation, and, similarly, .../icons/ would
contain either bitmap icons in common formats or compliant themes
(or partial themes) to support those .desktop files.

-- 
Emmet HIKORY

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Stefano Rivera]

> It's important to remember that when we say "support /opt/ properly"
> what we really mean is "support /opt/extras.ubuntu.com/* properly".  We
> aren't just using /opt/ as an install prefix, we making a different
> install prefix for every single application, and that means that every
> tool is going to need to consider every install prefix for every
> application that might be installed.

Can this not be solved with a post-install-hooked script (added by
dh_extras) that collates symlinks to everything in
/opt/extras.ubuntu.com/{appname}/share/applications/{appname}.desktop
in /opt/extras.ubuntu.com/share/applications?

Suddenly we only have one new path to look at. Not n paths.

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  H: +27 21 461 1230 C: +27 72 419 8559

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/05/2012 02:08 PM, Emmet Hikory wrote:
> Regardless of our final decision with regard to the implementation
> limiting the impact on users of potentially malicious automatically
> reviewed applications, I think we ought continue to make efforts so
> that our tools support /opt properly.  There are many existing popular
> and successful free and commercial applications using /opt today for
> other unices, and we can only benefit by reducing the porting effort
> for this software to Ubuntu.
> 

It's important to remember that when we say "support /opt/ properly"
what we really mean is "support /opt/extras.ubuntu.com/* properly".  We
aren't just using /opt/ as an install prefix, we making a different
install prefix for every single application, and that means that every
tool is going to need to consider every install prefix for every
application that might be installed.

But at the same time we're not currently making those app-specific
install prefixes actual alternatives to /usr/, they don't contain
/opt/extras.ubuntu.com/{appname}/share/applications/{appname}.desktop,
or /opt/extras.ubuntu.com/{appname}/share/pixmaps/{appname}.png

So even if we did make all of our tools use $XDG_DATA_DIRS, and we did
include every installed Extras app in $XDG_DATA_DIRS, they still
wouldn't work because we're not making those directories behave like an
XDG_DATA_DIR should.

Michael Hall
mhall...@ubuntu.com

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Wednesday, September 05, 2012 01:29:48 PM Steve Langasek wrote:
> On Wed, Sep 05, 2012 at 01:04:53PM -0400, Scott Kitterman wrote:
> > >App developers don't want to be distro maintainers.  That's why so many
> > >never attempt to get their apps into Universe or Debian.
> > >
> > >Right now PPAs are the easiest way for app developers to deliver their
> > >apps, so that's what they're using.  It makes things worse (and less
> > >safe) for users, but when they are faced with the choice between
> > >installing yet another PPA from someone they don't know, or not getting
> > >the app they want, they have been installing the PPA.
> > 
> > Certainly, but the current ARB level of effort could have gotten packages
> > into the distro via backports with almost certainly less effort on the
> > part of ARB members and no more on the part of app developers.  Absent a
> > highly automated system, the technical rationale for extras is very weak.
> 
> Not sure if something was lost in translation here, but the new proposal is
> precisely about implementing that highly automated system.

I understand that, but I think the current proposal is insufficient to reach 
that goal.  Part of my confusion is that to date the way extras has been run 
has been largely counter to it's stated goals and it's been done so at the 
expense of more efficient alternatives to achieve what's been achieved so far 
(this is not post-facto hindsight speaking - better alternatives were 
discussed at the time and rejected).  I am afraid the same thing will happen 
again.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Thursday, September 06, 2012 05:05:41 AM Emmet Hikory wrote:
> Scott Kitterman wrote:
> > On Thursday, September 06, 2012 04:14:40 AM Emmet Hikory wrote:
> > > To avoid conflicts of interest, I suggest that agreeing that the
> > > 
> > > content of the repository to be populated through MyApps is to be
> > > treated
> > > as part of Ubuntu is a prerequisite: otherwise there is too much
> > > potential
> > > for those Ubuntu Developers who have decided to entirely ignore the
> > > presence of a third-party external repository called "extras" to
> > > complain that their upload is unreasonably blocked, and no sensible
> > > conflict resolution path due to the lack of a common authority other
> > > than sabdfl.
> > 
> > That's not the only way to solve the procedural problem.  It's been
> > established that the Ubuntu technical board has the authority to set
> > policy
> > for the Canonical Partner archive.  I have assumed that they do for extras
> > as well.  If they don't, that can be fixed without redefining what is
> > part of Ubuntu and what is external to it but related (for those
> > unfamiliar, Partner is served from a different archive
> > (archive.canonical.com) and not formally part of Ubuntu).
> 
> Ah, if the partner precedent is extensible also to extras (and it may
> well be given that the ARB has often referred to the Tech Board to take
> decisions), then that also satisfies the prerequisite, so that in the event
> of conflict, there is a means to request external resolution (either from
> the Tech Board, or, if this becomes a significant burden, from some
> delegated authority), which we might reasonably expect to be binding for
> Ubuntu Developers.
> 
> Independently of the conflict resolution model, I still don't understand
> why we would need to consider such a collection of software "external",
> beyond the historical precedent: is there a reason we want to prevent an
> interested Ubuntu Developer from fixing a bug in one of these packages? Do
> we feel that having two clearly documented competing different processes
> for the inclusion of software will help our users to be able to run the
> software they prefer?  Do we want to prevent upstream developers from being
> able to maintain their software as part of Ubuntu?  Should flavours be
> restricted from seeding packages submitted by the original developers
> unless they are willing to override the upstream submission?

This alternate submission model will have it's own tools, processes and 
procedures.  I doubt that all Ubuntu developers will be sufficiently aware of 
these that the ability to upload into the alternate process should be allowed 
by default.

It does raise an interesting question about the need for a group of extras 
'gardeners' (in the same sense MOTU are Ubuntu archive 'gardeners').  Outside 
the ARB we don't have such a thing now, AFAIK, and we might want to get such a 
group if there are people interested.

> Yes, there are trust issues, especially if the submission is the first
> time we encounter some developer, but these can be handled by limiting
> what their software is permitted to do, without necessarily resorting to
> banishing the software from Ubuntu, and doing so in such a way that also
> delivers it to users to that later inclusion in Ubuntu may require even
> more discussion and coordination than has historically been required,
> rather than just a shift between different parts of Ubuntu as the
> necessary trust is established.

I think use of a container based approach would make this easier since MyApps 
packages would be operating as if they were part of the standard system and so 
it should be just a matter of adding breaks/replaces/transitional package the 
MyApps pacakgename and uploading.

> > FWIW, I think it's entirely appropriate for Ubuntu developers to not be
> > overly worried about third party repositories.  Any solution that
> > requires them to be concerned will drive Ubuntu to a forked namespace
> > with Debian and that will be a fundamental change in the way the
> > distribution works that I don't think we want.
> 
> My apologies if I created this impression: while I agree that we should
> impose no requirement that Ubuntu Developers be aware of the content of
> arbitrary third-party repositories, I believe it is in our interest to both
> have a clear means in place to resolve potential conflicts resulting from
> those third-party repositories we have agreed to present to users as
> potential defaults, and to follow namespace claims for what we might expect
> to be promoted as "the way" to request that software be made available to
> our users by the developers of that software.  Of course, if the inclusion
> requirements for such a third-party repository are such that it prevents
> any namespace claims, then there is no need to be concerned, but from the
> discussion so far, I don't believe we will reach this during the quantal
> cycle.

I don't think any substant

Re: Proposing a New App Developer Upload Process

[Steve Langasek]

On Wed, Sep 05, 2012 at 01:04:53PM -0400, Scott Kitterman wrote:
> >App developers don't want to be distro maintainers.  That's why so many
> >never attempt to get their apps into Universe or Debian.

> >Right now PPAs are the easiest way for app developers to deliver their
> >apps, so that's what they're using.  It makes things worse (and less
> >safe) for users, but when they are faced with the choice between
> >installing yet another PPA from someone they don't know, or not getting
> >the app they want, they have been installing the PPA.

> Certainly, but the current ARB level of effort could have gotten packages
> into the distro via backports with almost certainly less effort on the
> part of ARB members and no more on the part of app developers.  Absent a
> highly automated system, the technical rationale for extras is very weak.

Not sure if something was lost in translation here, but the new proposal is
precisely about implementing that highly automated system.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Scott Kitterman wrote:
> On Thursday, September 06, 2012 04:14:40 AM Emmet Hikory wrote:
> > To avoid conflicts of interest, I suggest that agreeing that the
> > content of the repository to be populated through MyApps is to be treated
> > as part of Ubuntu is a prerequisite: otherwise there is too much potential
> > for those Ubuntu Developers who have decided to entirely ignore the presence
> > of a third-party external repository called "extras" to complain that their
> > upload is unreasonably blocked, and no sensible conflict resolution path
> > due to the lack of a common authority other than sabdfl.
> 
> That's not the only way to solve the procedural problem.  It's been 
> established that the Ubuntu technical board has the authority to set policy 
> for the Canonical Partner archive.  I have assumed that they do for extras as 
> well.  If they don't, that can be fixed without redefining what is part of 
> Ubuntu and what is external to it but related (for those unfamiliar, Partner 
> is served from a different archive (archive.canonical.com) and not formally 
> part of Ubuntu).

Ah, if the partner precedent is extensible also to extras (and it may well
be given that the ARB has often referred to the Tech Board to take decisions),
then that also satisfies the prerequisite, so that in the event of conflict,
there is a means to request external resolution (either from the Tech Board,
or, if this becomes a significant burden, from some delegated authority),
which we might reasonably expect to be binding for Ubuntu Developers.

Independently of the conflict resolution model, I still don't understand
why we would need to consider such a collection of software "external",
beyond the historical precedent: is there a reason we want to prevent an
interested Ubuntu Developer from fixing a bug in one of these packages?
Do we feel that having two clearly documented competing different processes
for the inclusion of software will help our users to be able to run the
software they prefer?  Do we want to prevent upstream developers from being
able to maintain their software as part of Ubuntu?  Should flavours be
restricted from seeding packages submitted by the original developers
unless they are willing to override the upstream submission?

Yes, there are trust issues, especially if the submission is the first
time we encounter some developer, but these can be handled by limiting
what their software is permitted to do, without necessarily resorting to
banishing the software from Ubuntu, and doing so in such a way that also
delivers it to users to that later inclusion in Ubuntu may require even
more discussion and coordination than has historically been required,
rather than just a shift between different parts of Ubuntu as the
necessary trust is established.

> FWIW, I think it's entirely appropriate for Ubuntu developers to not be 
> overly 
> worried about third party repositories.  Any solution that requires them to 
> be 
> concerned will drive Ubuntu to a forked namespace with Debian and that will 
> be 
> a fundamental change in the way the distribution works that I don't think we 
> want.

My apologies if I created this impression: while I agree that we should
impose no requirement that Ubuntu Developers be aware of the content of
arbitrary third-party repositories, I believe it is in our interest to both
have a clear means in place to resolve potential conflicts resulting from
those third-party repositories we have agreed to present to users as
potential defaults, and to follow namespace claims for what we might expect
to be promoted as "the way" to request that software be made available to
our users by the developers of that software.  Of course, if the inclusion
requirements for such a third-party repository are such that it prevents
any namespace claims, then there is no need to be concerned, but from the
discussion so far, I don't believe we will reach this during the quantal
cycle.

-- 
Emmet HIKORY

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Thursday, September 06, 2012 04:14:40 AM Emmet Hikory wrote:
> > * File name conflicts: there I would suggest exploring Daniel's proposal
> > of relying on a conflict checker that works across all archives, so that
> > before an upload is accepted this service checks for any potential
> > clashes and informs the uploader to fix the package before they can do
> > the next submission. The uploader would either be an Ubuntu developer
> > (through the main archive) or an app developer (through extras, via
> > MyApps). This would not only benefit the app developer process, but also
> > fix the existing issue in the regular Ubuntu upload process.
> 
> To avoid conflicts of interest, I suggest that agreeing that the
> content of the repository to be populated through MyApps is to be treated
> as part of Ubuntu is a prerequisite: otherwise there is too much potential
> for those Ubuntu Developers who have decided to entirely ignore the presence
> of a third-party external repository called "extras" to complain that their
> upload is unreasonably blocked, and no sensible conflict resolution path
> due to the lack of a common authority other than sabdfl.

That's not the only way to solve the procedural problem.  It's been 
established that the Ubuntu technical board has the authority to set policy 
for the Canonical Partner archive.  I have assumed that they do for extras as 
well.  If they don't, that can be fixed without redefining what is part of 
Ubuntu and what is external to it but related (for those unfamiliar, Partner 
is served from a different archive (archive.canonical.com) and not formally 
part of Ubuntu.

Aside from that, I think that if the goal is 100K MyApps packages the only 
thing that does drown under it's own weight is a containerizaton based 
approach where each MyApps package has it's own namespace and it can't conflict 
with either other MyApps packages or the distribution proper.

FWIW, I think it's entirely appropriate for Ubuntu developers to not be overly 
worried about third party repositories.  Any solution that requires them to be 
concerned will drive Ubuntu to a forked namespace with Debian and that will be 
a fundamental change in the way the distribution works that I don't think we 
want.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

David Planella wrote:
> I fully sympathize and understand the advocacy for the use of /opt at
> the packaging level and I would in principle support it: it is the
> standard FHS location for add-on software packages and it creates a
> separate installation namespace that prevents file collisions. In
> theory, it seems the best approach.
> 
> However, reality has shown that (a) it is a big hurdle for app
> developers (who are actually the people we're trying to make the process
> easier for) to follow this policy and (b) we're enforcing a policy not
> even our tools support.
> 
> For (b), what I believe is that it is also clear that no one is going to
> work to improve /opt support in tooling or in developer toolkits in the
> near or distant future. So for this alone I consider it to be a dead end.

Given the work that has been ongoing on this for the past few years,
I don't believe that we should simply dismiss it, especially by saying that
nobody is doing it or will do it, so obviously it won't get done.  It may
be that our current tools are insufficient, and that we determine that the
use of /opt is not part of our insulation solution, but let's take that
decision based on the relative merits of different mechanisms and the
identified volume of work required for them, rather than on the basis of
either assumptions about the motivations of others or circular reasoning.

Similarly, if we are not implementing this in a way that is completely
transparent to the application developers, we are doing them a disservice,
as we are suggesting their upstream software be written in a manner that
reduces the ability for it to be packaged in other ways.

> Just to illustrate the kind of issues we've bumped into in MyApps
> submissions, here's just an example [1]: GtkBuilder does not work with
> the gettext Python library if you specify an alternative location to
> load translations from

Thank you for providing an example without a trivial rebuttal: this
significantly increases my confidence that it is not just intransigence
that keeps us from wholeheartedly adopting this.

Do we have bugs for all of these?  Are they tracked with a common
tag?  Even if we don't use this as part of our implementation, it remains
something that we should fix, if only in terms of FHS compliance (or, if
there are enough bugs that are truly insoluable, we should provide this
as input for future revisions of the FHS).

> While I like Emmet's idea of delegating the changes required to work
> with /opt to MyApps, this would also mean that the complexity in the
> logic behind the server would greatly increase. And in some cases (e.g.
> hardcoded paths) we would also need to actually modify the sources to
> ensure an app runs.

To be clear, my intent was to isolate the complexity in the tools
used to build packages, expecting it to be a matter of appropriate
debhelper plugins, potentially combined with a package mangler installed
in the extras build chroots: if there is logic to enforce /opt stored in
the MyApps server code beyond that required to use the debhelper plugins,
then I'd claim the implementation widely differed from the requirements.

Essentially, anything that can't be trivially emulated by someone
with a local build environment becomes unsuitable, as it limits the
ability of the developer to understand what is happening as a result
of their submission.  While some developers may have little or no
interest, there are plenty who would want to perform some local testing
prior to upload to MyApps.

> Summarizing, I think these are the 3 main points of discussion right now:

Unless everyone happens to be silently agreeing with me, I think there
is a fourth: that it makes sense for MyApps to be able to deliver
applications into a qualified human-based review process for inclusion
in the primary repositories in cases where the application wants or needs
closer integration to the system than is provided by the automatic
insulation mechanisms proposed, and that the result of such review should
include backporting the application to the current release so that the
application remains delivered to the target intended by the submitter.

> * Package name conflicts: it seems that we've agreed that a solution for
> package name conflicts is to rely on 'extras-' prefixing on the server
> side (i.e. MyApps). This seems to be easy enough to do, fit well with
> other parts of the spec and would be transparent to app developers.

Rather than implementing entirely in MyApps code, could this be
implemented as some text-transformation library which can be called
identically from MyApps, some command-line tool, and any arbitrary
packaging helpers or IDEs that happen to want this support?

> * File name conflicts: there I would suggest exploring Daniel's proposal
> of relying on a conflict checker that works across all archives, so that
> before an upload is accepted this service checks for any potential
> clashes and infor

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Didier Roche wrote:
> I don't agree it's only for low quality apps. More than once, people
> asked me to package their apps into ubuntu. This is particularly
> annoying when I have no interest at all in the package itself. The
> last case is pyromaths, which turned out to be quite popular in
> schools. It's definitively a quality apps and I kind of regret to
> upload it to ubuntu proper instead of letting them deal with myapps
> for now as I have to admit I won't do a good job tracking them.

In this case, do you see any reason that the developers of this
package should not be able to be granted PPU access to keep their
software well maintained in the archive now that it has been packaged,
rather than being subjected to arbitrary restrictions on what the
software may be allowed to do?

If so, what do you think would be required for you to feel
comfortable endorsing such a grant of access?  If not, why should
any Ubuntu Developer refrain from such an endorsement if they
consider the application well-packaged (either as a result of
their collaboration or review of the packaging)?

> The MyApps story is to avoid those 2 pitfalls to occur:
> * having ubuntu developers working on what they want to work on and
> focus their effort on that, following the components they selected
> closely and be able to help them.
> * having application developers being able to have the control where
> they should have: their own applications and decide when they
> update. We can enable them to update as long as they do no harm to
> the platform (like in the file conflict case, and many other use
> cases requiring insulation). The ultimate goal is that all the
> packaging part is helped/assisted to them: it's not because I want
> to upload my application to ubuntu that I have to become a packager
> and know every detail of the debian policy. I just want to deliver
> my application to users.

While these are both laudable goals, I don't understand why there
needs to be such a firm separation between "packages that are in Ubuntu"
and "packages that upstream authors may update".  While I am in favor
of taking care to insulate our users from potentially malicious packages
in the presence of a completely automated acceptance process, I expect
that the vast majority of software authors would also be perfectly happy
to be able to upload their software directly after receiving some manual
review or assistance from someone knowledgeable about packaging, and I
believe that we both can and should attempt to enable as many as we feel
we can trust to do just this, rather than relegating them to some more
limited packaging area just because we don't want to be personally
responsible for the package.

-- 
Emmet HIKORY

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Micah Gersten]

On 09/05/2012 11:46 AM, Didier Roche wrote:
> Le 05/09/2012 16:26, David Planella a écrit :
>> Al 05/09/12 14:29, En/na Scott Kitterman ha escrit:
>>>
>>> David Planella  wrote:
>>>
 * File name conflicts: there I would suggest exploring Daniel's
 proposal
 of relying on a conflict checker that works across all archives, so
 that
 before an upload is accepted this service checks for any potential
 clashes and informs the uploader to fix the package before they can do
 the next submission. The uploader would either be an Ubuntu developer
 (through the main archive) or an app developer (through extras, via
 MyApps). This would not only benefit the app developer process, but
 also
 fix the existing issue in the regular Ubuntu upload process.
>>> This would be useful, but insufficient.
>>>
>> Could you elaborate on why you think it would be insufficient and what
>> alternative you believe would be a solution for file name conflicts in
>> this context?
>>
 * Namespace ownership: even with conflict checking there is the issue
 of
 who gets to own a particular file name or namespace. E.g. would "Mad
 Feathered Creatures" (/usr/bin/birds, from Extras) have priority in
 owning the binary's name if it had been submitted before "Jolly Flying
 Animals" (also /usr/bin/birds, from Universe)? I think if we want to
 make apps first-class citizens, even if not part of the distro, a
 simple
 suggestion would just be to do it on a first-come-first-serve basis.
>>> I think it is fundamentally incorrect to give something built on
>>> Ubuntu namespace priority over Ubuntu itself. Additionally, if this
>>> service proves popular, this approach would drive a permanent
>>> namespace wedge between Debian and Ubuntu that might, over time,
>>> significanly change the nature of the relationship between the two
>>> distributions.
>>>
>> I can see the point of the namespace wedge if we become immensely
>> popular. What do you think the alternatives are?
>>
 What are your thoughts on these?

 Finally, I believe we do need to provision for those cases, but my
 understanding is that the potential amount of occurrences would be
 rather low. So do you think they justify additional Engineering work,
 or
 rather they could be dealt manually on a case-by-case basis?

>>> You've got to consider it now or it won't scale.  IIRC the original
>>> presentations on this topic at UDS Orlando(?), the intent was to be
>>> able to scale out to hit numbers of applications equal to or greater
>>> than the Apple Appstore/Google Play.  If you hit that, then MyApps
>>> ends up being several times the size of the Ubuntu archive.
>>>
>> And that's what we're doing right now. My only concern is not to block
>> on a situation that will concern just a fraction of the uploads, even at
>> a higher scale. That's what I'm trying to get a feel for.
>
> For all those 3 namespacing/files issues, maybe we can think of a
> simple solution.
>
> I really like Daniel 's idea of a conflict-check-before-publish
> service. One of the case that was raised on the thread is about "you
> can't predict the future". What about that example taking back the
> "Mad feathered creatures" shipping /usr/bin/birds
> - in precise, only the apps from extras is there
> - in quantal, we sync from debian, or upload directly to universe
> "Jolly Flying Animals" which ships the same file (new package or update)
> -> nothing happens at this stage (remember that extras is not opened)
> - a little bit (like 3 weeks?) before opening extras, we run (and then
> continuously run) the conflict-check-before-publish service. This one
> will detect the new conflict between the two packages, and:
> 1. add a Conflicts: in "Mad feathered creatures" debian/control file
> in extras against the package in universe.
> 2. will send an email to the app developer telling "hey, maybe not all
> ubuntu user will be able to use your apps as there is this excellent
> new application <…> into the archive"
>
> At the extreme, if the component in conflict is a core component, as
> the ubuntu archive have an higher priority than the extra one
> (right?), then the core component will be preferred on dist-upgrade.
> This has the advantage of:
> - pushing the burden to the app developer, not to ubuntu developers
> - avoid having to do conflicts/replaces on our side and so diverging
> from debian
> - by pushing the burden to the app developer, still having a automatic
> update solution integrated into myapps, but mailing them, we ensure to
> have people committed to their application in ubuntu
>
> I think this solution would fit for what will really be and stay,
> IMHO, a corner case. I doubt with all the precautions taken into the
> naming and namespace that will happen with every cycles and the few
> developers in that case will be warned and have time to react before
> the extras opening. In case they don't react, we hav

Re: Proposing a New App Developer Upload Process

[Steve Langasek]

On Wed, Sep 05, 2012 at 12:30:46PM +0200, Sebastien Bacher wrote:
> Le 05/09/2012 03:55, Steve Langasek a écrit :
> >Note that, independent of any packaging issues for Ubuntu, upstream best
> >practice would have the absolute paths in these files generated at build
> >time, once it's known where they will be installed.
> The specification [1] allows both unity the command name only or the
> full qualified path to it without recommending a solution.

> Looking through /usr/share/applications the vast majority of
> programs nowadays opt for the simple form, likely to avoid having to
> deal with an extra .desktop.in -> desktop through sed handling

That's explicitly the opposite of what you need for /opt however, where the
.desktop files need to point to the executables which are not on a path.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Didier Roche wrote:
> I completely concur with David here. Having implemented it in
> Karmic, the /opt hacks for our build system (touching a lot of build
> components, like cdbs, debhelper, distutils…) for extra support and
> that were reverted erroneously some times, I can attest that
> supporting /opt is not easy. In addition to that and as already
> mentioned, ones need to patch all services, desktop file support,
> lens, scope and having various hacks like the gtkbuilder ones to get
> it working properly. I infer those are only part of the iceberg we
> discovered trying to support /opt and they are many others.
> 
> Trying to workaround with /opt/extras.ubuntu.com/applications or
> /opt/extras.ubuntu.com/share/dbus-1/services seems to be just to
> move the issue one step aside until the next one will come and won't
> address the issue with all use cases we will have in the future.

Regardless of our final decision with regard to the implementation
limiting the impact on users of potentially malicious automatically
reviewed applications, I think we ought continue to make efforts so
that our tools support /opt properly.  There are many existing popular
and successful free and commercial applications using /opt today for
other unices, and we can only benefit by reducing the porting effort
for this software to Ubuntu.

> I can't wait for
> the Quickly commit which will remove all the /opt hacks we had with
> Mike to do. :)

I'm in full agreement with this: Quickly should certainly not have
any of the existing /opt hacks.  If our typical tools have good support
for /opt, the most Quickly should have is some option to allow the
package build to be placed in /opt or not, where that option is
simply passed to the underlying tools.  If our typical tools do not
have good support for /opt, we should be filing bugs against those
tools and resolving this, rather than attempting to work around it.

> I really like Daniel 's idea of a conflict-check-before-publish
> service. One of the case that was raised on the thread is about "you
> can't predict the future". What about that example taking back the
> "Mad feathered creatures" shipping /usr/bin/birds
> - in precise, only the apps from extras is there
> - in quantal, we sync from debian, or upload directly to universe
> "Jolly Flying Animals" which ships the same file (new package or
> update)
> -> nothing happens at this stage (remember that extras is not opened)
> - a little bit (like 3 weeks?) before opening extras, we run (and
> then continuously run) the conflict-check-before-publish service.
> This one will detect the new conflict between the two packages, and:
> 1. add a Conflicts: in "Mad feathered creatures" debian/control file
> in extras against the package in universe.
> 2. will send an email to the app developer telling "hey, maybe not
> all ubuntu user will be able to use your apps as there is this
> excellent new application <…> into the archive"
> 
> At the extreme, if the component in conflict is a core component, as
> the ubuntu archive have an higher priority than the extra one
> (right?), then the core component will be preferred on dist-upgrade.
> This has the advantage of:
> - pushing the burden to the app developer, not to ubuntu developers
> - avoid having to do conflicts/replaces on our side and so diverging
> from debian
> - by pushing the burden to the app developer, still having a
> automatic update solution integrated into myapps, but mailing them,
> we ensure to have people committed to their application in ubuntu
> 
> I think this solution would fit for what will really be and stay,
> IMHO, a corner case. I doubt with all the precautions taken into the
> naming and namespace that will happen with every cycles and the few
> developers in that case will be warned and have time to react before
> the extras opening. In case they don't react, we have the automatic
> metadata addition in conflicts: which enables apt to deal with it.

I can think of two classes of potential conflict here, both of
which we likely want to have a general solution for prior to agreeing
on this implementation:

1)  Backports

We end up with an emerging essential tool that we want to promote
as standard for the next release, is suitable for backporting, and for
which there is user demand for availability in the current release.
Unfortunately, there is a filename collision with something in extras,
and we believe that the final solution should be in favour of the new
package we wish to backport.

One potential way to address this issue would be to notify developers
of extras packages that we may remove or modify their packages in extras
or include them as system packages without notice and at our discretion
to resolve potential future filesystem conflicts: this gives us a selection
of several technical means resolve potential conflicts.  I imagine we ought
to be able to grant PPU access to affected developers in these cases so as
to reduce the i

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

Matthew Paul Thomas  wrote:

>-BEGIN PGP SIGNED MESSAGE-
>Hash: SHA1
>
>Scott Kitterman wrote on 05/09/12 04:39:
>> 
>> On Tuesday, September 04, 2012 06:01:56 PM Steve Langasek wrote: 
>> ...
>>> 
>>> There's a general sense that the Ubuntu archive can't scale out 
>>> to the degree it needs to in order to take on the next challenges
>>> for the platform. While Debian packaging isn't hard for you or
>>> me, and while it's definitely gotten much easier over the past 4
>>> years or so, it's still not so easy that we blindly trust outside
>>> contributors to get the packaging right without review by an
>>> Ubuntu developer.  We do not have an infinite supply of Ubuntu
>>> developers to do this review. Should the set of software
>>> available to Ubuntu users through apt be limited to only that
>>> software that Ubuntu developers (or Debian maintainers) have time
>>> and interest to take care of directly?  Or should users have
>>> better (not perfect, but better) ways to install software that's
>>> not gotten the attention of the elite inner circle?
>> 
>> First, this elite inner circle has an open door.
>
>Which requires using Launchpad Bugs (regardless of where you actually
>track bugs for your application), knowing what "Debian" is, using the
>archaic IRC chat system, signing a "Code of Conduct" that is almost
>entirely about contributing to Ubuntu itself, and other things that,
>for a typical application author, have no relevance whatsoever.
>
>This inner circle didn't deliberately set out to be elite.
>
>> Second we also have a limited supply of ARB reviewers.  Starting a 
>> new review process (ARB) because their aren't enough reviewers was 
>> clearly doomed from the start, so much of my "I don't understand 
>> this" comes from the current process where we substitute one review
>> process with insufficient reviewers for another one with 
>> insufficient reviewers.
>
>Yes, I sighed when the ARB was announced for the same reason. But the
>conclusion should not be to go back to the first process that still
>has insufficient reviewers. It should be to find a process that does
>not require reviewers -- or at least, requires them to spend on the
>order of a hundredth of the time per application that they do now.

I don't disagree with that, but we ought to build something designed to scale 
and while I think this design would be a step in the right direction, I think 
it will fail to scale sufficiently if it is successful in the way you envision. 
 It would be better to get a design that has sufficient potential for 
scalability rather than one we know we'll have to throw but and redo.

I think application quality is a much bigger issue than the difficulty of 
getting such applications in Ubuntu.

>> Additionally, I think the notion that "Oh, X has a bazillion apps, 
>> so we need that many too" is mistaken in many regards.  How many 
>> office suites do we need? I'd say one that works robustly, 
>> reliably, and compatibly with it's proprietary competition (and 
>> despite a huge amount of work by people deeply interested in 
>> solving this problem, IMO we don't have it). How many solitaire 
>> games do we need?  I'm not sure, but I'm confident it's fewer than 
>> one finds in whatever Android Marketplace is called now.
>
>Nearly two decades have passed since operating systems were judged
>primarily by their office suites and solitaire games. Photo
>retouching, online note syncing, genealogy, kiosk-style UI for the
>elderly, music notation, home accounting, calendaring, paying taxes,
>making greeting cards, chess, Web design, screencasting, CAD, school
>timetabling, wedding planning, screenwriting. For thousands of "long
>tail" genres like those, competing OSes have multiple applications to
>choose from -- but the published choices in Ubuntu are either
>non-existent or, not to put too fine a point on it, terrible.
>
>Now, there are many reasons for that: difficulty of publishing is far
>from the only one. But it would be a subtle error to think that an
>application not existing for Ubuntu at all means that difficulty of
>publishing is unimportant. It may be one of the reasons nobody
>bothered to develop the application in the first place.

It's possible, but considering we have many major application classes without a 
single entry that really qualifies as a peer competitor (including ones that 
did get in), I think this is very unlikely to be a significant factor.

>> Historically, Linux distros have included a curated collection, 
>> some larger, some smaller, of relevant applications, libraries, etc
>> that can be used on the base operating system.  That curation 
>> process is one of the real strengths of Linux distributions...
>
>That's a near-tautology. "Distributions" are named after the assumption
>that selecting and packaging other people's software is a way to
>produce a useful operating system.
>
>That may work for a few hundred thousand or even a few million
>notebook/desktop users, bu

Re: Proposing a New App Developer Upload Process

[Didier Roche]

Le 05/09/2012 15:30, Scott Kitterman a écrit :


Sebastien Bacher  wrote:


Le 05/09/2012 14:31, Scott Kitterman a écrit :

Why invest any time at all?

Your question is worded in a way which I've an hard time parsing, so I
will try to reply as I understand it, let me know if that's not what
you
mean

"Why do I think people would want to invest any time on Ubuntu then" is

the question?

In which case I would say the OS and the applications are two different

things...

* Building a distribution,platform,desktop environment,OS is hard, it
does require discipline and work from lot of people collaborating, I
don't think you can "lower the standard" for the base of the system.

Why invest any time at all? Different contributors have different
motivations: it's fun, it's interesting work, they like the result,
they
are proud of what they work on, they like the people they are working
with, etc...

* Dealing with applications is another topic, there is no reason we
would need so much coordination, reviews, testing for those. There is
no
reason we need to tight them to our release cycle and freezes. We are
not the ones "owning" those apps, upstream are. Those upstream, for
most, don't ask to be part of our project, many don't even run Ubuntu,
they just want their apps to reach users. That's a different world and
a
different problem space...


Sorry for being unclear.

You seemed to be suggesting that MyApps was a suitable path for a reduced 
effort mechanism for getting low quality applications available to users.

My question was if they are so bad, why expend any effort on them at all.

I get the "next month's beer festival app" use case, but that doesn't seem to 
be where most of the focus is.  Most of the focus seems to be on less ephemeral apps.  
AFAICT, these pretty much fall into things that should be in the archive and things of 
insufficient quality where it's a positive feature not to deliver them.


I don't agree it's only for low quality apps. More than once, people 
asked me to package their apps into ubuntu. This is particularly 
annoying when I have no interest at all in the package itself. The last 
case is pyromaths, which turned out to be quite popular in schools. It's 
definitively a quality apps and I kind of regret to upload it to ubuntu 
proper instead of letting them deal with myapps for now as I have to 
admit I won't do a good job tracking them.


So it means it's not to packagers to decide of the importance for the 
user of a package, and naturally, they would prefer to scratch their own 
itch. It also means it's more difficult for the application developer to 
get a newer version in, with latency of answering, reaching the right 
contact, and so on.


The MyApps story is to avoid those 2 pitfalls to occur:
* having ubuntu developers working on what they want to work on and 
focus their effort on that, following the components they selected 
closely and be able to help them.
* having application developers being able to have the control where 
they should have: their own applications and decide when they update. We 
can enable them to update as long as they do no harm to the platform 
(like in the file conflict case, and many other use cases requiring 
insulation). The ultimate goal is that all the packaging part is 
helped/assisted to them: it's not because I want to upload my 
application to ubuntu that I have to become a packager and know every 
detail of the debian policy. I just want to deliver my application to users.


Didier

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Allison Randal wrote:
> Here's my perspective, in short form:
> 
> - Extras is PPU + Backports + training wheels.
> 
> - Extras is a place to experiment with how the training wheels should
> work, without disrupting the main archives.
> 
> - Eventually Extras should go away, once we've extracted the best value
> from the experiment.

While this makes sense, I think that if we want extreme scalability,
we would need to have somewhere with the training wheels firmly bolted
in place, as there are likely many possible programs that simply will
not notice or care if there are limitations to their activities as a
result of our efforts to insulate our users, and if we can create the
appropriate mechanisms to protect Ubuntu systems, I see no technical
reason not to provide for an entirely automated application submission
and acceptance process, but I do believe that such a process should
place the results in a separate repository or component or something,
because some of our users may wish to intentionally block such packages
from being made available to their systems.

I do think that we would do best to have whatever location we decide
to be appropriate for the automatically reviewed and approved pacakges be
something we consider part of Ubuntu (as distinct from how extras is today),
as I believe the separation of this project from Ubuntu proper does little
to ensure close integration while being a meaningless distinction in the
perception of our users.

It is worth noting that it may not be possible to have a completely
automatic process, as those who would redistribute the content may be
limited in some ways in terms of what content they can redistribute, or
to which locations they can redistribute that content, but these are
social and legal issues, rather than technical, and best handled by
asking those we would expect to redistribute the content to determine,
rather than attempting to construct a process that requires human review
because there may be an issue: there are enough existing means to either
create an artifical interruption between acceptance and publication or
cease the distribution of a package that we can surely allow the
distributor to make the appropriate choice as part of the deployment of
whatever implementation we agree upon.

> These are what I consider to be the most crucial immediate questions:
> 
> - Should we change Extras operation so it's less like REVU (approving
> each package) and more like PPU (approving a particular developer for a
> particular package name)?

I would prefer to answer this question with "No", for three distinct
reasons.  Firstly, I believe strongly in peer review, and think that any
package that anyone submits should be reviewed *as a package*, and secondly
I don't think we want to mix our conception of code and people: bad code
from good people should be fixed before being provided to our users, and
we should be able to accept good code from those who we might not want to
immediately grant upload rights (given an acceptable compromise to all
concerned).

One of the issues previously found with *any* of the review-based
systems was that packages would sometimes fall into what seemed to be
an endless submit-review-resubmit-rereview... process.  During the more
active days of REVU, we experimented with collaboration for the packaging
of a few packages, with each reviewer fixing all the issues they could find
in the package, and resubmitting, but this collapsed due to disagreements
about the credit for the work done: if we can find a way to appropriately
show the various contributions of collaborative initial packagers, such an
approach would likely significantly reduce the apparent difficulty of
accepting manually reviewed packages and help introduce those submitting
packages to the collaborative workflow commonly used in Ubuntu.

So, thirdly, I suggest that if we start approving applications
in a per-person manner, rather than a per-application manner, we limit
the potential for pre-inclusion collaboration: this ought happen *within*
our default submission tools, rather than being something that needs to
happen externally.  I also believe that in cases where there was such
collaboration, we would want to have the set of folk to be granted PPU
rights for the package based on the final outcome of the packaging,
rather than the identity of the initial submitter.

Note that I consider this "No" to be only applicable for manually
reviewed packages: where we can provide mechanisms to protect users and
allow automated review, I think we ought automatically grant upload rights
for the package to both the submitter and anyone else already able to upload
to any unseeded package.

> - Should the DMB be made responsible for approving those Extras PPU rights?

If we are to use the PPU mechanism (which seems appropriate to me), I
believe that only the DMB can be responsible for the oversight of granting
these rights: anything else ought invol

Re: Proposing a New App Developer Upload Process

[David Planella]

Al 05/09/12 17:49, En/na Allison Randal ha escrit:
> On 09/04/2012 10:02 AM, Emmet Hikory wrote:
>>
>> The above notwithstanding, I'm all in favour of being able to safely add
>> new or frequently changing packages with policy-compliant filenames to 
>> Ubuntu,
>> both during the life of a release and as part of future releases.  I'm just
>> concerned that our attempts to work around some of the problems we have had
>> with such systems in the past may lead to us creating new systems that 
>> closely
>> parallel older systems without addressing the issues that caused the prior
>> systems to no longer be considered ideal for future progress.  Given my
>> experience with cleanup after the pull of all the random apt repos, and work
>> with REVU, I feel fairly strongly that we ought create an implementation that
>> closely integrates with exsting Debian tools and avoids as many potential
>> sources of conflict as possible.  If we are to have a high-throughput mostly
>> automated mechanism for new packages to be added, we need to be sure that we
>> have not created too many bottlenecks relying on human discussion, whether
>> this be too high a reviewer overhead, a need to coordinate yet another
>> namespace conflict resolution forum, or anything else.
>>
>> Based on this belief, while we might have any arbitrary process by which
>> applications are submitted for review, the only reason I can see for treating
>> the packages as any different than any other unseeded package is either some
>> technical limitation or if we somehow felt that we had different levels of
>> trust for packages in one place or another (as reflected by our trust in the
>> reviewers, presumably).  The more effort we spend to create a "special" and
>> "different" class of applications, the greater the chance we have created
>> a division in the self-perceived identities of our various developers where
>> none need exist.
>>
>> Of course, this may not be considered safe (see all the sandboxing
>> discussion in the spec), but I believe we'd do better to help those who
>> are developing applications follow a model that generates software that
>> we would be happy to welcome as regular system software than to encourage
>> those who might need greater system access to find ways to work around
>> whatever limitations we might attempt to put in place.  If we can use
>> the same front-end to the application developers for *both* types of
>> applications, we reduce both confusion and dissention, and likely find
>> that our implementation need have fewer loopholes, as there is a means
>> to easily move software from the restricted facility to a more regular
>> facility, with full filesytem access, no network restrictions, etc.
>> which we may use whenever we find a package that might otherwise require
>> an extension to the facilities available.
> 
> Well said!
> 
> Here's my perspective, in short form:
> 
> - Extras is PPU + Backports + training wheels.
> 
> - Extras is a place to experiment with how the training wheels should
> work, without disrupting the main archives.
> 
> - Eventually Extras should go away, once we've extracted the best value
> from the experiment.
> 

I personally don't feel strongly about which location third party apps
are distributed from, be it in Extras or somewhere else, but I can see
the benefits of having a separate archive that does not disrupt the main
one.

That said, there is one part where I disagree: in considering Extras (if
we identify Extras with the App Developer Process) an experiment or
training ground for Ubuntu platform development, for which we already
have a process in place.

I consider the App Developer Process simply a way to enable application
authors to seamlessly and securely publish their apps. Human review and
packaging policies have proven to be the main bottlenecks for this
happening, and these (plus security) are the main points the proposal is
trying to address.

> The proposal that started this thread is rather lengthy, and contains
> 2-3 years worth of work on tooling.

Indeed, and work that will benefit the distro too, such as sandboxing.

> It's a valuable seed for discussion,
> but really too much to digest all at once. I'd like to focus on what to
> do in the next Ubuntu release cycle. Partly because each cycle *is* a
> fresh experiment for Extras.

I think we've got different views on this: I believe considering each
cycle as an experiment will not bring us to the end goal.

The fact that it will take some cycles to implement such a spec simply
means that when agreed upon, the work will be divided and scoped into
cycles, so this is also about focusing on what to do in the next Ubuntu
release cycle.

> What we've learned in previous cycles has
> radically changed the plan already, and is sure to do so in the future.
> And partly because we may not agree right now on the ultimate future
> (some won't agree with me that Extras should go away, others won't agree
> with me that

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

Michael Hall  wrote:

>
>On 09/04/2012 06:05 PM, Scott Kitterman wrote:
>> On Tuesday, September 04, 2012 01:19:50 PM Michael Hall wrote:
>>> On 09/04/2012 01:02 PM, Emmet Hikory wrote:
 Michael Hall wrote:
> Not only would it require the we carry these patches to system
>tools and
> libraries indefinitely, it would also mean that we encourage
>developers
> to produce apps and packages that are not compatible with our
>Upstream
> (Debian) or any other distro.
>
 We rather ought encourage developers to use build systems that
>allow

 one to specify an install location: then the difference between
>installing
 to /opt/ and system locations becomes entirely a matter of a
>preference in
 the packaging: other format distributions will not even notice. 
 Distributions that use Debian-format packages are likely to be
>downstream
 from either Debian or Ubuntu: if we share the optional /opt/
>packaging
 preference with Debian, then it is trivial for any distribution to
>adopt
 the package as either extra or part of the system with very little
 effort.
>>>
>>> If it can be reduced to a matter of build-time preferences, that
>would
>>> be fine.  The current /opt/ rules and support, however, require both
>a
>>> significant amount of extra work in the packaging, and usually
>>> additional changes to the code itself.
>>>
 As for patches in system tools, at least for icon cache, path,
>and
 desktop

 file discovery, this is one or two lines in a configuration file
>that
 already carries Ubuntu-specific patches.  Is there a specific
>example of
 a discovery mechanism that is more complicated to change?
>>>
>>> .desktop files, .service files (for dbus), .lens and .scope files
>(for
>>> Unity) all needed to be changed to point to executables or images in
>>> /opt/extra.ubuntu.com/${appname}/ when being packaged for Extras. 
>Unity
>>> APIs that require a .desktop or image file name had to use absolute
>>> paths to the files in /opt/extras.ubuntu.com/{$appname}.  These all
>>> required changes to the app's source in order to support the
>>> installation under /opt/.
>> 
>> These sorts of changes should be generalizable to be supported at
>build time.  
>> It would take some work to set up the first time, but any build
>system I've 
>> worked with would be able to do this.
>> 
>> Scott K
>> 
>To support them at build-time, you end up having to make various
>changes
>to the original source.  For example, a typical foo.desktop file will
>contain the following lines:
>
>Icon=foo
>Exec=foo
>
>Now, if the app has to be installed into /opt/extras.ubuntu.com/foo/,
>those lines will need to be changed to:
>
>Icon=/opt/extras.ubuntu.com/foo/foo.png
>Exec=/opt/extras.ubuntu.com/foo/bin/foo
>
>Which requires running sed against foo.desktop using overrides in
>debian/rules.  We need to do this for DBus services and Unity lenses
>and
>scopes as well.  Other build-time hacks are required to support
>translations in /opt/, and apport crashdb.  None of these are
>especially
>difficult, but they accumulate into a large number of changes for even
>the most basic application.
>
>I have attached an example debian/rules file from one of the App
>Showdown entries.  This was a simple python app with a valid setup.py,
>built using Quickly.  You can see how much it's having to change to
>install into /opt/, and this was typical of all App Showdown
>debian/rules files.
>
>In addition, a quick scan of the source for some of those apps shows a
>number of places where paths to /opt/ are being hard-coded elsewhere in
>the source.  I know this isn't something they should be hard-coding,
>but
>these are the kind of independent app developers we want to target
>Ubuntu and this is what they are likely going to keep doing in order to
>support having their app installed in /opt/.

It is quite common for upstreams to make their code less portable than they 
might.  One of the basic reasons make exists is to allow building code in 
portable ways.  It doesn't surprise me a bit the code isn't written to be 
portable, but that's doesn't mean it can't be.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

Michael Hall  wrote:

>
>On 09/04/2012 05:50 PM, Scott Kitterman wrote:
>> On Wednesday, September 05, 2012 12:11:18 AM Emmet Hikory wrote:
>>> For application developers who prefer standard locations, I
>don't see
>>> any reason we oughtn't simply add their packages to the repository
>with
>>> immediate backport: in addition to allowing application developers
>to put
>>> their files in any policy-compliant location, it automatically
>provides a
>>> safe upgrade path for users.  Even for software with release cycles
>that
>>> require significantly more frequent updates than typically provided
>by our
>>> release cycle, there are few barriers to keeping this updated in
>backports,
>>> and users who have installed from backports will remain with
>backports, so
>>> their most active and interested users may be expected to always
>have the
>>> latest version, while highly conservative users will be able to
>enjoy a
>>> consistent ABI during the life of a given release (although these
>users
>>> would need to wait for our next release before using the package).
>> 
>> +1.  I've never understood why there is resistance to this.
>> 
>> Scott K
>> 
>
>App developers don't want to be distro maintainers.  That's why so many
>never attempt to get their apps into Universe or Debian.
>
>Right now PPAs are the easiest way for app developers to deliver their
>apps, so that's what they're using.  It makes things worse (and less
>safe) for users, but when they are faced with the choice between
>installing yet another PPA from someone they don't know, or not getting
>the app they want, they have been installing the PPA.
>
Certainly, but the current ARB level of effort could have gotten packages into 
the distro via backports with almost certainly less effort on the part of ARB 
members and no more on the part of app developers.  Absent a highly automated 
system, the technical rationale for extras is very weak.

Scott K


-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Matthew Paul Thomas]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Scott Kitterman wrote on 05/09/12 04:39:
> 
> On Tuesday, September 04, 2012 06:01:56 PM Steve Langasek wrote: 
> ...
>> 
>> There's a general sense that the Ubuntu archive can't scale out 
>> to the degree it needs to in order to take on the next challenges
>> for the platform. While Debian packaging isn't hard for you or
>> me, and while it's definitely gotten much easier over the past 4
>> years or so, it's still not so easy that we blindly trust outside
>> contributors to get the packaging right without review by an
>> Ubuntu developer.  We do not have an infinite supply of Ubuntu
>> developers to do this review. Should the set of software
>> available to Ubuntu users through apt be limited to only that
>> software that Ubuntu developers (or Debian maintainers) have time
>> and interest to take care of directly?  Or should users have
>> better (not perfect, but better) ways to install software that's
>> not gotten the attention of the elite inner circle?
> 
> First, this elite inner circle has an open door.

Which requires using Launchpad Bugs (regardless of where you actually
track bugs for your application), knowing what "Debian" is, using the
archaic IRC chat system, signing a "Code of Conduct" that is almost
entirely about contributing to Ubuntu itself, and other things that,
for a typical application author, have no relevance whatsoever.

This inner circle didn't deliberately set out to be elite.

> Second we also have a limited supply of ARB reviewers.  Starting a 
> new review process (ARB) because their aren't enough reviewers was 
> clearly doomed from the start, so much of my "I don't understand 
> this" comes from the current process where we substitute one review
> process with insufficient reviewers for another one with 
> insufficient reviewers.

Yes, I sighed when the ARB was announced for the same reason. But the
conclusion should not be to go back to the first process that still
has insufficient reviewers. It should be to find a process that does
not require reviewers -- or at least, requires them to spend on the
order of a hundredth of the time per application that they do now.

> Additionally, I think the notion that "Oh, X has a bazillion apps, 
> so we need that many too" is mistaken in many regards.  How many 
> office suites do we need? I'd say one that works robustly, 
> reliably, and compatibly with it's proprietary competition (and 
> despite a huge amount of work by people deeply interested in 
> solving this problem, IMO we don't have it). How many solitaire 
> games do we need?  I'm not sure, but I'm confident it's fewer than 
> one finds in whatever Android Marketplace is called now.

Nearly two decades have passed since operating systems were judged
primarily by their office suites and solitaire games. Photo
retouching, online note syncing, genealogy, kiosk-style UI for the
elderly, music notation, home accounting, calendaring, paying taxes,
making greeting cards, chess, Web design, screencasting, CAD, school
timetabling, wedding planning, screenwriting. For thousands of "long
tail" genres like those, competing OSes have multiple applications to
choose from -- but the published choices in Ubuntu are either
non-existent or, not to put too fine a point on it, terrible.

Now, there are many reasons for that: difficulty of publishing is far
from the only one. But it would be a subtle error to think that an
application not existing for Ubuntu at all means that difficulty of
publishing is unimportant. It may be one of the reasons nobody
bothered to develop the application in the first place.

> Historically, Linux distros have included a curated collection, 
> some larger, some smaller, of relevant applications, libraries, etc
> that can be used on the base operating system.  That curation 
> process is one of the real strengths of Linux distributions...

That's a near-tautology. "Distributions" are named after the assumption
that selecting and packaging other people's software is a way to
produce a useful operating system.

That may work for a few hundred thousand or even a few million
notebook/desktop users, but it has failed to grow beyond that. The
distro model discourages application developers, slows application
updates (making the installed base less reliable and less secure), and
distracts Ubuntu developers from improving Ubuntu itself. Eventually
the time comes to say "enough, let's try something else".

> ...
> 
> There are a LOT of Debian/Ubuntu developers and non-developers 
> involved in packaging, so I suspect the good stuff will get 
> attention (I may be wrong).  If such a system existed, then, if it 
> was really clearly distinct from Ubuntu, I think it might make 
> sense, but what's been done so far doesn't meet that goal and 
> neither does what's specified.
> 
> ...

There are a lot of developers involved in packaging, compared to what?
Two years ago there were 160 MOTUs. Today there are 149. That isn't
how you

Re: Proposing a New App Developer Upload Process

[Didier Roche]

Le 05/09/2012 16:26, David Planella a écrit :

Al 05/09/12 14:29, En/na Scott Kitterman ha escrit:


David Planella  wrote:


Al 05/09/12 05:18, En/na Emmet Hikory ha escrit:

Steve Langasek wrote:

On Tue, Sep 04, 2012 at 05:20:47PM -0400, Michael Hall wrote:

It's possible that namespacing within /usr - for instance,
requiring each subdirectory and binary name to be prefixed with
'extras.' - would give better results with regards to the upstream
build systems, I'm not sure. But if we are going to try to do
namespacing of extras packages to avoid the need for coordination,
we definitely would need improved package helpers that support
namespacing of packages (i.e., debhelper support).

Would it be reasonable for MyApps to add a package name prefix to

the

submitted package, rather than requiring the developer to do so?
MyApps will already be modifying the submitted package to include

the

generator AppArmor profile.
So, for example, if I submit quickly-gtk as a source package to
MyApps, it would convert it to extras-quickly-gtk, add the AppArmor
profile, and re-build the source package before submitting it to

the

PPA/buildds.

For package renaming that seems easy enough to do, but if we're

worried

about file conflicts, dynamically prefixing filenames when building

the

package is going to have an even worse impact on the upstream code

than

changing directories does.

 Changing the base filename indeed generates more complicated

issues,

not only in terms of collisions, but also in terms of coordination

between

code and data (e.g. code expects to find ${CONFIG_DIR}/package and

fails

to initialise properly with only ${CONFIG_DIR}/extras-package.

 However, if we consider "renaming" to apply to the entire path,

rather

than the bare filename, this becomes considerably less of an issue.

If

MyApps were to call tooling that changed the default installation

location

for files while preserving bare filenames, then there is less

potential

for conflict.  The standard mechanism for this is to place the

non-system

files in per-package namespaced directories in /opt.


I fully sympathize and understand the advocacy for the use of /opt at
the packaging level and I would in principle support it: it is the
standard FHS location for add-on software packages and it creates a
separate installation namespace that prevents file collisions. In
theory, it seems the best approach.

However, reality has shown that (a) it is a big hurdle for app
developers (who are actually the people we're trying to make the
process
easier for) to follow this policy and (b) we're enforcing a policy not
even our tools support.

For (b), what I believe is that it is also clear that no one is going
to
work to improve /opt support in tooling or in developer toolkits in the
near or distant future. So for this alone I consider it to be a dead
end.

We are assuming that build systems and libraries are flexible enough to
cater for an alternative installation prefix, and that it will all just
work at runtime. Unfortunately, this has proven not to be the case. And
I think the amount of coordination and work that'd be required to
provide solid /opt support in Ubuntu would be best put in other parts
of
the spec, such as its central one: sandboxing.

Just to illustrate the kind of issues we've bumped into in MyApps
submissions, here's just an example [1]: GtkBuilder does not work with
the gettext Python library if you specify an alternative location to
load translations from (e.g.
'/opt/extras.ubuntu.com/$APP/share/locale/'). So we had to either wait
or contribute to fix the upstream bug (unlikely as per the complexity
involved) or implement a workaround. The workaround was to get Quickly
to modify the source code to use 'locale' instead of 'gettext': an
effective but nasty solution.

And that's been the journey with /opt so far: continually playing catch
with the next exception we have to fix or work around. This does not
sound like a very solid approach or a good experience to provide to app
developers. And again, I think we should rather direct resources where
higher priorities lie.

While I like Emmet's idea of delegating the changes required to work
with /opt to MyApps, this would also mean that the complexity in the
logic behind the server would greatly increase. And in some cases (e.g.
hardcoded paths) we would also need to actually modify the sources to
ensure an app runs.

I understand it would be a lot of work and people aren't working on it.  What's 
your basis for believing there will be resources available to implement this 
alternate approach?


We want to make Ubuntu a platform for app developers. This process has a
driver, if agreed upon, blueprints will be drafted and discussed at UDS,
and resources assigned in the same way we do every cycle.

As far as I know, there is no one driving, volunteering or particularly
interested in doing the work to better support /opt across our main
build systems and runtime or d

Re: Proposing a New App Developer Upload Process

[Allison Randal]

On 09/04/2012 10:02 AM, Emmet Hikory wrote:
> 
> The above notwithstanding, I'm all in favour of being able to safely add
> new or frequently changing packages with policy-compliant filenames to Ubuntu,
> both during the life of a release and as part of future releases.  I'm just
> concerned that our attempts to work around some of the problems we have had
> with such systems in the past may lead to us creating new systems that closely
> parallel older systems without addressing the issues that caused the prior
> systems to no longer be considered ideal for future progress.  Given my
> experience with cleanup after the pull of all the random apt repos, and work
> with REVU, I feel fairly strongly that we ought create an implementation that
> closely integrates with exsting Debian tools and avoids as many potential
> sources of conflict as possible.  If we are to have a high-throughput mostly
> automated mechanism for new packages to be added, we need to be sure that we
> have not created too many bottlenecks relying on human discussion, whether
> this be too high a reviewer overhead, a need to coordinate yet another
> namespace conflict resolution forum, or anything else.
> 
> Based on this belief, while we might have any arbitrary process by which
> applications are submitted for review, the only reason I can see for treating
> the packages as any different than any other unseeded package is either some
> technical limitation or if we somehow felt that we had different levels of
> trust for packages in one place or another (as reflected by our trust in the
> reviewers, presumably).  The more effort we spend to create a "special" and
> "different" class of applications, the greater the chance we have created
> a division in the self-perceived identities of our various developers where
> none need exist.
> 
> Of course, this may not be considered safe (see all the sandboxing
> discussion in the spec), but I believe we'd do better to help those who
> are developing applications follow a model that generates software that
> we would be happy to welcome as regular system software than to encourage
> those who might need greater system access to find ways to work around
> whatever limitations we might attempt to put in place.  If we can use
> the same front-end to the application developers for *both* types of
> applications, we reduce both confusion and dissention, and likely find
> that our implementation need have fewer loopholes, as there is a means
> to easily move software from the restricted facility to a more regular
> facility, with full filesytem access, no network restrictions, etc.
> which we may use whenever we find a package that might otherwise require
> an extension to the facilities available.

Well said!

Here's my perspective, in short form:

- Extras is PPU + Backports + training wheels.

- Extras is a place to experiment with how the training wheels should
work, without disrupting the main archives.

- Eventually Extras should go away, once we've extracted the best value
from the experiment.

The proposal that started this thread is rather lengthy, and contains
2-3 years worth of work on tooling. It's a valuable seed for discussion,
but really too much to digest all at once. I'd like to focus on what to
do in the next Ubuntu release cycle. Partly because each cycle *is* a
fresh experiment for Extras. What we've learned in previous cycles has
radically changed the plan already, and is sure to do so in the future.
And partly because we may not agree right now on the ultimate future
(some won't agree with me that Extras should go away, others won't agree
with me that it should stick around for a little while longer), but I'm
confident that we *can* agree on what the immediate next steps should be.

These are what I consider to be the most crucial immediate questions:

- Should we change Extras operation so it's less like REVU (approving
each package) and more like PPU (approving a particular developer for a
particular package name)?

- Should the DMB be made responsible for approving those Extras PPU rights?

- Should we remove the /opt install requirement?

- Should we start developing tooling for more automated package
checking/sandboxing?

My vote is: Yes, make it more like a PPU. I see advantages to recruiting
the DMB for approvals, but they may not want the extra work. No, we
shouldn't remove the /opt requirement, at least not yet. Maybe later
when we have more advanced tools. And heck yeah!, we should start
working on the tools. We can keep doing manual reviews until the tools
are good enough that we trust them. (And will always make the tools kick
back to manual review for anything they can't handle.)


My ideal future looks something like: Debian and Ubuntu co-habiting on
DebExpo (either on mentors.debian.net, or two sites with federated
data), with a slick UI, and fantastic integrated tools for automated
package checks. But, there's a lot of development work between here a

Re: Proposing a New App Developer Upload Process

[David Planella]

Al 05/09/12 14:29, En/na Scott Kitterman ha escrit:
> 
> 
> David Planella  wrote:
> 
>> Al 05/09/12 05:18, En/na Emmet Hikory ha escrit:
>>> Steve Langasek wrote:
 On Tue, Sep 04, 2012 at 05:20:47PM -0400, Michael Hall wrote:
>> It's possible that namespacing within /usr - for instance,
>> requiring each subdirectory and binary name to be prefixed with
>> 'extras.' - would give better results with regards to the upstream
>> build systems, I'm not sure. But if we are going to try to do
>> namespacing of extras packages to avoid the need for coordination,
>> we definitely would need improved package helpers that support
>> namespacing of packages (i.e., debhelper support).

> Would it be reasonable for MyApps to add a package name prefix to
>> the
> submitted package, rather than requiring the developer to do so?
> MyApps will already be modifying the submitted package to include
>> the
> generator AppArmor profile.

> So, for example, if I submit quickly-gtk as a source package to
> MyApps, it would convert it to extras-quickly-gtk, add the AppArmor
> profile, and re-build the source package before submitting it to
>> the
> PPA/buildds.

 For package renaming that seems easy enough to do, but if we're
>> worried
 about file conflicts, dynamically prefixing filenames when building
>> the
 package is going to have an even worse impact on the upstream code
>> than
 changing directories does.
>>>
>>> Changing the base filename indeed generates more complicated
>> issues,
>>> not only in terms of collisions, but also in terms of coordination
>> between
>>> code and data (e.g. code expects to find ${CONFIG_DIR}/package and
>> fails
>>> to initialise properly with only ${CONFIG_DIR}/extras-package.
>>>
>>> However, if we consider "renaming" to apply to the entire path,
>> rather
>>> than the bare filename, this becomes considerably less of an issue. 
>> If
>>> MyApps were to call tooling that changed the default installation
>> location
>>> for files while preserving bare filenames, then there is less
>> potential
>>> for conflict.  The standard mechanism for this is to place the
>> non-system
>>> files in per-package namespaced directories in /opt.
>>>
>>
>> I fully sympathize and understand the advocacy for the use of /opt at
>> the packaging level and I would in principle support it: it is the
>> standard FHS location for add-on software packages and it creates a
>> separate installation namespace that prevents file collisions. In
>> theory, it seems the best approach.
>>
>> However, reality has shown that (a) it is a big hurdle for app
>> developers (who are actually the people we're trying to make the
>> process
>> easier for) to follow this policy and (b) we're enforcing a policy not
>> even our tools support.
>>
>> For (b), what I believe is that it is also clear that no one is going
>> to
>> work to improve /opt support in tooling or in developer toolkits in the
>> near or distant future. So for this alone I consider it to be a dead
>> end.
>>
>> We are assuming that build systems and libraries are flexible enough to
>> cater for an alternative installation prefix, and that it will all just
>> work at runtime. Unfortunately, this has proven not to be the case. And
>> I think the amount of coordination and work that'd be required to
>> provide solid /opt support in Ubuntu would be best put in other parts
>> of
>> the spec, such as its central one: sandboxing.
>>
>> Just to illustrate the kind of issues we've bumped into in MyApps
>> submissions, here's just an example [1]: GtkBuilder does not work with
>> the gettext Python library if you specify an alternative location to
>> load translations from (e.g.
>> '/opt/extras.ubuntu.com/$APP/share/locale/'). So we had to either wait
>> or contribute to fix the upstream bug (unlikely as per the complexity
>> involved) or implement a workaround. The workaround was to get Quickly
>> to modify the source code to use 'locale' instead of 'gettext': an
>> effective but nasty solution.
>>
>> And that's been the journey with /opt so far: continually playing catch
>> with the next exception we have to fix or work around. This does not
>> sound like a very solid approach or a good experience to provide to app
>> developers. And again, I think we should rather direct resources where
>> higher priorities lie.
>>
>> While I like Emmet's idea of delegating the changes required to work
>> with /opt to MyApps, this would also mean that the complexity in the
>> logic behind the server would greatly increase. And in some cases (e.g.
>> hardcoded paths) we would also need to actually modify the sources to
>> ensure an app runs.
> 
> I understand it would be a lot of work and people aren't working on it.  
> What's your basis for believing there will be resources available to 
> implement this alternate approach?
> 

We want to make Ubuntu a platform for app developers. This process h

Re: Proposing a New App Developer Upload Process

[David Planella]

Al 05/09/12 15:30, En/na Scott Kitterman ha escrit:
> 
> 
> Sebastien Bacher  wrote:
> 
>> Le 05/09/2012 14:31, Scott Kitterman a écrit :
>>> Why invest any time at all?
>> Your question is worded in a way which I've an hard time parsing, so I 
>> will try to reply as I understand it, let me know if that's not what
>> you 
>> mean
>>
>> "Why do I think people would want to invest any time on Ubuntu then" is
>>
>> the question?
>>
>> In which case I would say the OS and the applications are two different
>>
>> things...
>>
>> * Building a distribution,platform,desktop environment,OS is hard, it 
>> does require discipline and work from lot of people collaborating, I 
>> don't think you can "lower the standard" for the base of the system.
>>
>> Why invest any time at all? Different contributors have different 
>> motivations: it's fun, it's interesting work, they like the result,
>> they 
>> are proud of what they work on, they like the people they are working 
>> with, etc...
>>
>> * Dealing with applications is another topic, there is no reason we 
>> would need so much coordination, reviews, testing for those. There is
>> no 
>> reason we need to tight them to our release cycle and freezes. We are 
>> not the ones "owning" those apps, upstream are. Those upstream, for 
>> most, don't ask to be part of our project, many don't even run Ubuntu, 
>> they just want their apps to reach users. That's a different world and
>> a 
>> different problem space...
>>
> Sorry for being unclear.
> 
> You seemed to be suggesting that MyApps was a suitable path for a reduced 
> effort mechanism for getting low quality applications available to users.
> 
> My question was if they are so bad, why expend any effort on them at all.
> 

The focus of this proposal is to empower app developers to easily
publish their own apps in a streamlined yet secure way. It is not about
judging the quality of apps, which ultimately the users will do through
ratings and reviews.

In terms of quality, though, if we're to judge by the app showdown
submissions, I think we're to expect quite a lot of high quality apps
(lower quality too, but I think that's much less relevant when compared
to security).

We're precisely presenting this proposal to spend less time on apps and
provide an automated process that will give control to app developers to
publish their own apps with minimum delay and with tight security policies.

> I get the "next month's beer festival app" use case, but that doesn't seem to 
> be where most of the focus is.  Most of the focus seems to be on less 
> ephemeral apps.  AFAICT, these pretty much fall into things that should be in 
> the archive and things of insufficient quality where it's a positive feature 
> not to deliver them.
> 

I think Seb pretty much nails it there:

Al 05/09/12 15:14, En/na Sebastien Bacher ha escrit:
> * Dealing with applications is another topic, there is no reason we
> would need so much coordination, reviews, testing for those. There
> is no
> reason we need to tight them to our release cycle and freezes. We are
> not the ones "owning" those apps, upstream are. Those upstream, for
> most, don't ask to be part of our project, many don't even run Ubuntu,
> they just want their apps to reach users. That's a different world
> and a different problem space...

Cheers,
David.



signature.asc
Description: OpenPGP digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Sebastien Bacher]

Le 05/09/2012 15:30, Scott Kitterman a écrit :

There was a GSOC project in Debian last summer to build a tool that would 
create a Debian package for every Python package on PyPi.  The new Ruby policy 
is built around gem2deb.  Automatically generating some or all of a package 
isn't particularly novel.  I think it's better to focus on getting more good 
stuff into Ubuntu than to invent complex new mechanisms to provide packages 
that either could have been provided without the mechanisms or aren't of real 
benefit to the users.
Those tools are great but don't resolve the problem ... the issue is not 
to get things in, it's to give the control where it belongs: to upstream.


The current model just create tons of work for us, we do what we can and 
often at the end get upstream and users angry at us for different 
reasons: because we sometime make bad calls on what version to use, or 
get stucked by our resources limitations and let bugs unfixed, or add 
patches upstream don't agree with, etc.


It does make sense to make some calls for better integration on our main 
components, but I don't think we have anything to win by getting on the 
upstream<->user way for every applications out there...


Cheers,
Sebastien Bacher

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

Sebastien Bacher  wrote:

>Le 05/09/2012 14:43, Scott Kitterman a écrit :
>> Unless you want a BSD like o/s - applications split, this new process
>is not going to affect core apps like Chromium or Gimp.
>Not sure what you call "BSD like", but it would be great if e.g Gimp
>was 
>moved out of universe to the end of upstream and maintained in a more 
>flexible way that what we do at the moment, they would probably do a
>way 
>better job than we do (we have number of upstreams who are unhappy
>about 
>us shipping a version "known to be buggy" of their code in our stable 
>release, or distro patching things in a way they don't approve of, or 
>not taking updates to fix the issues they fixed on their side for
>months).

BSD Unix's ship their core o/s in /usr and this is what is released with a new 
"release".  All applications install into /usr/local and are continually 
updated through their various ports systems:

http://en.m.wikipedia.org/wiki/Ports_collection

IME, the Debian/Ubuntu archives do a far better job providing a consistent, 
reliable experience than you get from such an approach and we would be heading 
down the wrong road to try and replicate it.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

Sebastien Bacher  wrote:

>Le 05/09/2012 14:31, Scott Kitterman a écrit :
>> Why invest any time at all?
>Your question is worded in a way which I've an hard time parsing, so I 
>will try to reply as I understand it, let me know if that's not what
>you 
>mean
>
>"Why do I think people would want to invest any time on Ubuntu then" is
>
>the question?
>
>In which case I would say the OS and the applications are two different
>
>things...
>
>* Building a distribution,platform,desktop environment,OS is hard, it 
>does require discipline and work from lot of people collaborating, I 
>don't think you can "lower the standard" for the base of the system.
>
>Why invest any time at all? Different contributors have different 
>motivations: it's fun, it's interesting work, they like the result,
>they 
>are proud of what they work on, they like the people they are working 
>with, etc...
>
>* Dealing with applications is another topic, there is no reason we 
>would need so much coordination, reviews, testing for those. There is
>no 
>reason we need to tight them to our release cycle and freezes. We are 
>not the ones "owning" those apps, upstream are. Those upstream, for 
>most, don't ask to be part of our project, many don't even run Ubuntu, 
>they just want their apps to reach users. That's a different world and
>a 
>different problem space...
>
Sorry for being unclear.

You seemed to be suggesting that MyApps was a suitable path for a reduced 
effort mechanism for getting low quality applications available to users.

My question was if they are so bad, why expend any effort on them at all.

I get the "next month's beer festival app" use case, but that doesn't seem to 
be where most of the focus is.  Most of the focus seems to be on less ephemeral 
apps.  AFAICT, these pretty much fall into things that should be in the archive 
and things of insufficient quality where it's a positive feature not to deliver 
them.

There was a GSOC project in Debian last summer to build a tool that would 
create a Debian package for every Python package on PyPi.  The new Ruby policy 
is built around gem2deb.  Automatically generating some or all of a package 
isn't particularly novel.  I think it's better to focus on getting more good 
stuff into Ubuntu than to invent complex new mechanisms to provide packages 
that either could have been provided without the mechanisms or aren't of real 
benefit to the users.

Scott K


-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Riku Voipio]

On 5 September 2012 06:39, Scott Kitterman  wrote:
> How many
> solitaire games do we need?  I'm not sure, but I'm confident it's fewer than
> one finds in whatever Android Marketplace is called now.

> Historically, Linux distros have included a curated collection, some larger,
> some smaller, of relevant applications, libraries, etc that can be used on the
> base operating system.  That curation process is one of the real strengths of
> Linux distributions and I think "Oh, let's have a bazillion of everything
> because it's there" is the wrong way to go.

That's a bit Stockholm Syndromish - the collection is only curated for
packaging quality, not the quality of the application itself or the
amount of overlapping applications. Observe the  ~20 solitaire game
packages in Ubuntu not to mention the amount of window managers etc.
The "curation process" is not to have less or better apps, it is a
necessity since Debian packaging is complex and any package could ruin
users install (postinst scripts run as root!).

The extras process should not be about "having gazillion apps like
android" but rather "having upstreams upload their apps themself like
they do on android". The key end user story it provides is getting new
versions of applications when the new application is released rather
than when the whole distro is upgraded. Right now, precise users can't
easily install gimp 2.8 or chromium 21. The secondary advantage is
that Ubuntu can concentrate in core distro and leave applications to
3rd parties.

These goals can only be reached if the 3rd party uploaded applications
are insulated so well that the review process can be done mostly
automatically - else the reviewers will continue to be the bottleneck.

Riku

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Sebastien Bacher]

Le 05/09/2012 14:43, Scott Kitterman a écrit :

Unless you want a BSD like o/s - applications split, this new process is not 
going to affect core apps like Chromium or Gimp.
Not sure what you call "BSD like", but it would be great if e.g Gimp was 
moved out of universe to the end of upstream and maintained in a more 
flexible way that what we do at the moment, they would probably do a way 
better job than we do (we have number of upstreams who are unhappy about 
us shipping a version "known to be buggy" of their code in our stable 
release, or distro patching things in a way they don't approve of, or 
not taking updates to fix the issues they fixed on their side for months).


--
Sebastien Bacher

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Sebastien Bacher]

Le 05/09/2012 14:31, Scott Kitterman a écrit :

Why invest any time at all?
Your question is worded in a way which I've an hard time parsing, so I 
will try to reply as I understand it, let me know if that's not what you 
mean


"Why do I think people would want to invest any time on Ubuntu then" is 
the question?


In which case I would say the OS and the applications are two different 
things...


* Building a distribution,platform,desktop environment,OS is hard, it 
does require discipline and work from lot of people collaborating, I 
don't think you can "lower the standard" for the base of the system.


Why invest any time at all? Different contributors have different 
motivations: it's fun, it's interesting work, they like the result, they 
are proud of what they work on, they like the people they are working 
with, etc...


* Dealing with applications is another topic, there is no reason we 
would need so much coordination, reviews, testing for those. There is no 
reason we need to tight them to our release cycle and freezes. We are 
not the ones "owning" those apps, upstream are. Those upstream, for 
most, don't ask to be part of our project, many don't even run Ubuntu, 
they just want their apps to reach users. That's a different world and a 
different problem space...


Cheers,
Sebastien Bacher



--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

Riku Voipio  wrote:

>On 5 September 2012 06:39, Scott Kitterman 
>wrote:
>> How many
>> solitaire games do we need?  I'm not sure, but I'm confident it's
>fewer than
>> one finds in whatever Android Marketplace is called now.
>
>> Historically, Linux distros have included a curated collection, some
>larger,
>> some smaller, of relevant applications, libraries, etc that can be
>used on the
>> base operating system.  That curation process is one of the real
>strengths of
>> Linux distributions and I think "Oh, let's have a bazillion of
>everything
>> because it's there" is the wrong way to go.
>
>That's a bit Stockholm Syndromish - the collection is only curated for
>packaging quality, not the quality of the application itself or the
>amount of overlapping applications. Observe the  ~20 solitaire game
>packages in Ubuntu not to mention the amount of window managers etc.
>The "curation process" is not to have less or better apps, it is a
>necessity since Debian packaging is complex and any package could ruin
>users install (postinst scripts run as root!).
>
>The extras process should not be about "having gazillion apps like
>android" but rather "having upstreams upload their apps themself like
>they do on android". The key end user story it provides is getting new
>versions of applications when the new application is released rather
>than when the whole distro is upgraded. Right now, precise users can't
>easily install gimp 2.8 or chromium 21. The secondary advantage is
>that Ubuntu can concentrate in core distro and leave applications to
>3rd parties.
>
>These goals can only be reached if the 3rd party uploaded applications
>are insulated so well that the review process can be done mostly
>automatically - else the reviewers will continue to be the bottleneck.

This process won't help either of those was cases.  

Packages are routinely removed from both Debian and Ubuntu for being buggy in 
their upstream code.  It's not just about packaging.  Additionally, good distro 
developers maintain a relationship with their upstreams and help them improve 
the quality of their package.

I agree the curation process has significant room for improvement, but that 
doesn't make it valueless as is.

We have backports so that, where there is demand, users don't have to wait.  
Claims that they do are generally incorrect (backports that require new 
libraries can be problematic, but the MyApps apps are in no better position).

Unless you want a BSD like o/s - applications split, this new process is not 
going to affect core apps like Chromium or Gimp.

Scott K


-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

Sebastien Bacher  wrote:

>Le 05/09/2012 05:39, Scott Kitterman a écrit :
>> No really, since I think that most of the packages that have been
>aimed at the
>> ARB are not "this month's beer festival" apps.  They are things that
>could
>> have been packaged normally just fine.
>Well, at the end of the day there is also the issue that we expect high
>
>quality for things that go in the archive and it means it takes time
>and 
>efforts to get something in (some of the stuff uploaded before ff still
>
>didn't get reviewed in NEW btw).
>
>I'm sure you will find that on the 130 apps from the app contest lot
>will be
>
>- not up to Debian,Ubuntu archive standards
>
>- having a developper who doesn't care much about Ubuntu itself, and
>our 
>distribution rules, cycle, freezes, etc ... app writer just want users 
>to be able to get their software
>
>- not maintained over time
>
>Why invest so much time and efforts to grow the number of things that 
>will then stay buggy and unmaintained in universe and makes our work 
>even harder (look at how many days we spend fixing years
>old unmaintained and unused stuff when e.g GTK starts deprecating 
>functions or when doing lib transitions)

Why invest any time at all?

Scott K


-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

David Planella  wrote:

>Al 05/09/12 05:18, En/na Emmet Hikory ha escrit:
>> Steve Langasek wrote:
>>> On Tue, Sep 04, 2012 at 05:20:47PM -0400, Michael Hall wrote:
> It's possible that namespacing within /usr - for instance,
> requiring each subdirectory and binary name to be prefixed with
> 'extras.' - would give better results with regards to the upstream
> build systems, I'm not sure. But if we are going to try to do
> namespacing of extras packages to avoid the need for coordination,
> we definitely would need improved package helpers that support
> namespacing of packages (i.e., debhelper support).
>>>
 Would it be reasonable for MyApps to add a package name prefix to
>the
 submitted package, rather than requiring the developer to do so?
 MyApps will already be modifying the submitted package to include
>the
 generator AppArmor profile.
>>>
 So, for example, if I submit quickly-gtk as a source package to
 MyApps, it would convert it to extras-quickly-gtk, add the AppArmor
 profile, and re-build the source package before submitting it to
>the
 PPA/buildds.
>>>
>>> For package renaming that seems easy enough to do, but if we're
>worried
>>> about file conflicts, dynamically prefixing filenames when building
>the
>>> package is going to have an even worse impact on the upstream code
>than
>>> changing directories does.
>> 
>> Changing the base filename indeed generates more complicated
>issues,
>> not only in terms of collisions, but also in terms of coordination
>between
>> code and data (e.g. code expects to find ${CONFIG_DIR}/package and
>fails
>> to initialise properly with only ${CONFIG_DIR}/extras-package.
>> 
>> However, if we consider "renaming" to apply to the entire path,
>rather
>> than the bare filename, this becomes considerably less of an issue. 
>If
>> MyApps were to call tooling that changed the default installation
>location
>> for files while preserving bare filenames, then there is less
>potential
>> for conflict.  The standard mechanism for this is to place the
>non-system
>> files in per-package namespaced directories in /opt.
>> 
>
>I fully sympathize and understand the advocacy for the use of /opt at
>the packaging level and I would in principle support it: it is the
>standard FHS location for add-on software packages and it creates a
>separate installation namespace that prevents file collisions. In
>theory, it seems the best approach.
>
>However, reality has shown that (a) it is a big hurdle for app
>developers (who are actually the people we're trying to make the
>process
>easier for) to follow this policy and (b) we're enforcing a policy not
>even our tools support.
>
>For (b), what I believe is that it is also clear that no one is going
>to
>work to improve /opt support in tooling or in developer toolkits in the
>near or distant future. So for this alone I consider it to be a dead
>end.
>
>We are assuming that build systems and libraries are flexible enough to
>cater for an alternative installation prefix, and that it will all just
>work at runtime. Unfortunately, this has proven not to be the case. And
>I think the amount of coordination and work that'd be required to
>provide solid /opt support in Ubuntu would be best put in other parts
>of
>the spec, such as its central one: sandboxing.
>
>Just to illustrate the kind of issues we've bumped into in MyApps
>submissions, here's just an example [1]: GtkBuilder does not work with
>the gettext Python library if you specify an alternative location to
>load translations from (e.g.
>'/opt/extras.ubuntu.com/$APP/share/locale/'). So we had to either wait
>or contribute to fix the upstream bug (unlikely as per the complexity
>involved) or implement a workaround. The workaround was to get Quickly
>to modify the source code to use 'locale' instead of 'gettext': an
>effective but nasty solution.
>
>And that's been the journey with /opt so far: continually playing catch
>with the next exception we have to fix or work around. This does not
>sound like a very solid approach or a good experience to provide to app
>developers. And again, I think we should rather direct resources where
>higher priorities lie.
>
>While I like Emmet's idea of delegating the changes required to work
>with /opt to MyApps, this would also mean that the complexity in the
>logic behind the server would greatly increase. And in some cases (e.g.
>hardcoded paths) we would also need to actually modify the sources to
>ensure an app runs.

I understand it would be a lot of work and people aren't working on it.  What's 
your basis for believing there will be resources available to implement this 
alternate approach?

>Summarizing, I think these are the 3 main points of discussion right
>now:
>
>* Package name conflicts: it seems that we've agreed that a solution
>for
>package name conflicts is to rely on 'extras-' prefixing on the server
>side (i.e. MyApps). This seems to be easy enough to do, fit well w

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Sebastien Bacher wrote:
> Le 05/09/2012 03:55, Steve Langasek a écrit :
> >Note that, independent of any packaging issues for Ubuntu, upstream best
> >practice would have the absolute paths in these files generated at build
> >time, once it's known where they will be installed.
> The specification [1] allows both unity the command name only or the
> full qualified path to it without recommending a solution.
> 
> Looking through /usr/share/applications the vast majority of
> programs nowadays opt for the simple form, likely to avoid having to
> deal with an extra .desktop.in -> desktop through sed handling
> 
> [1] http://standards.freedesktop.org/desktop-entry-spec/latest/ar01s06.html

Bare names are generally preferred, not because of any effort to
avoid preprocessing files (there are lots of examples where packages use
.desktop.in -> .desktop processing and leave barenames), but rather to
let $PATH operate as expected, and similar.  I suspect the use of the
bare name for the executable is also made more popular by the frequent
insistence by .desktop file reviewers that the icon should *not* be
an absolute path (as this breaks icon themes), and folk expect this to
apply to other entries as well.

-- 
Emmet HIKORY

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[David Planella]

Al 05/09/12 05:18, En/na Emmet Hikory ha escrit:
> Steve Langasek wrote:
>> On Tue, Sep 04, 2012 at 05:20:47PM -0400, Michael Hall wrote:
 It's possible that namespacing within /usr - for instance,
 requiring each subdirectory and binary name to be prefixed with
 'extras.' - would give better results with regards to the upstream
 build systems, I'm not sure. But if we are going to try to do
 namespacing of extras packages to avoid the need for coordination,
 we definitely would need improved package helpers that support
 namespacing of packages (i.e., debhelper support).
>>
>>> Would it be reasonable for MyApps to add a package name prefix to the
>>> submitted package, rather than requiring the developer to do so?
>>> MyApps will already be modifying the submitted package to include the
>>> generator AppArmor profile.
>>
>>> So, for example, if I submit quickly-gtk as a source package to
>>> MyApps, it would convert it to extras-quickly-gtk, add the AppArmor
>>> profile, and re-build the source package before submitting it to the
>>> PPA/buildds.
>>
>> For package renaming that seems easy enough to do, but if we're worried
>> about file conflicts, dynamically prefixing filenames when building the
>> package is going to have an even worse impact on the upstream code than
>> changing directories does.
> 
> Changing the base filename indeed generates more complicated issues,
> not only in terms of collisions, but also in terms of coordination between
> code and data (e.g. code expects to find ${CONFIG_DIR}/package and fails
> to initialise properly with only ${CONFIG_DIR}/extras-package.
> 
> However, if we consider "renaming" to apply to the entire path, rather
> than the bare filename, this becomes considerably less of an issue.  If
> MyApps were to call tooling that changed the default installation location
> for files while preserving bare filenames, then there is less potential
> for conflict.  The standard mechanism for this is to place the non-system
> files in per-package namespaced directories in /opt.
> 

I fully sympathize and understand the advocacy for the use of /opt at
the packaging level and I would in principle support it: it is the
standard FHS location for add-on software packages and it creates a
separate installation namespace that prevents file collisions. In
theory, it seems the best approach.

However, reality has shown that (a) it is a big hurdle for app
developers (who are actually the people we're trying to make the process
easier for) to follow this policy and (b) we're enforcing a policy not
even our tools support.

For (b), what I believe is that it is also clear that no one is going to
work to improve /opt support in tooling or in developer toolkits in the
near or distant future. So for this alone I consider it to be a dead end.

We are assuming that build systems and libraries are flexible enough to
cater for an alternative installation prefix, and that it will all just
work at runtime. Unfortunately, this has proven not to be the case. And
I think the amount of coordination and work that'd be required to
provide solid /opt support in Ubuntu would be best put in other parts of
the spec, such as its central one: sandboxing.

Just to illustrate the kind of issues we've bumped into in MyApps
submissions, here's just an example [1]: GtkBuilder does not work with
the gettext Python library if you specify an alternative location to
load translations from (e.g.
'/opt/extras.ubuntu.com/$APP/share/locale/'). So we had to either wait
or contribute to fix the upstream bug (unlikely as per the complexity
involved) or implement a workaround. The workaround was to get Quickly
to modify the source code to use 'locale' instead of 'gettext': an
effective but nasty solution.

And that's been the journey with /opt so far: continually playing catch
with the next exception we have to fix or work around. This does not
sound like a very solid approach or a good experience to provide to app
developers. And again, I think we should rather direct resources where
higher priorities lie.

While I like Emmet's idea of delegating the changes required to work
with /opt to MyApps, this would also mean that the complexity in the
logic behind the server would greatly increase. And in some cases (e.g.
hardcoded paths) we would also need to actually modify the sources to
ensure an app runs.

Summarizing, I think these are the 3 main points of discussion right now:

* Package name conflicts: it seems that we've agreed that a solution for
package name conflicts is to rely on 'extras-' prefixing on the server
side (i.e. MyApps). This seems to be easy enough to do, fit well with
other parts of the spec and would be transparent to app developers.

* File name conflicts: there I would suggest exploring Daniel's proposal
of relying on a conflict checker that works across all archives, so that
before an upload is accepted this service checks for any potential
clashes and informs the upl

Re: Proposing a New App Developer Upload Process

[Sebastien Bacher]

Le 05/09/2012 05:39, Scott Kitterman a écrit :

No really, since I think that most of the packages that have been aimed at the
ARB are not "this month's beer festival" apps.  They are things that could
have been packaged normally just fine.
Well, at the end of the day there is also the issue that we expect high 
quality for things that go in the archive and it means it takes time and 
efforts to get something in (some of the stuff uploaded before ff still 
didn't get reviewed in NEW btw).


I'm sure you will find that on the 130 apps from the app contest lot will be

- not up to Debian,Ubuntu archive standards

- having a developper who doesn't care much about Ubuntu itself, and our 
distribution rules, cycle, freezes, etc ... app writer just want users 
to be able to get their software


- not maintained over time

Why invest so much time and efforts to grow the number of things that 
will then stay buggy and unmaintained in universe and makes our work 
even harder (look at how many days we spend fixing years
old unmaintained and unused stuff when e.g GTK starts deprecating 
functions or when doing lib transitions)


Cheers,
Sebastien Bacher

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Sebastien Bacher]

Le 05/09/2012 03:55, Steve Langasek a écrit :

Note that, independent of any packaging issues for Ubuntu, upstream best
practice would have the absolute paths in these files generated at build
time, once it's known where they will be installed.
The specification [1] allows both unity the command name only or the 
full qualified path to it without recommending a solution.


Looking through /usr/share/applications the vast majority of programs 
nowadays opt for the simple form, likely to avoid having to deal with an 
extra .desktop.in -> desktop through sed handling



[1] http://standards.freedesktop.org/desktop-entry-spec/latest/ar01s06.html

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/04/2012 05:50 PM, Scott Kitterman wrote:
> On Wednesday, September 05, 2012 12:11:18 AM Emmet Hikory wrote:
>> For application developers who prefer standard locations, I don't see
>> any reason we oughtn't simply add their packages to the repository with
>> immediate backport: in addition to allowing application developers to put
>> their files in any policy-compliant location, it automatically provides a
>> safe upgrade path for users.  Even for software with release cycles that
>> require significantly more frequent updates than typically provided by our
>> release cycle, there are few barriers to keeping this updated in backports,
>> and users who have installed from backports will remain with backports, so
>> their most active and interested users may be expected to always have the
>> latest version, while highly conservative users will be able to enjoy a
>> consistent ABI during the life of a given release (although these users
>> would need to wait for our next release before using the package).
> 
> +1.  I've never understood why there is resistance to this.
> 
> Scott K
> 

App developers don't want to be distro maintainers.  That's why so many
never attempt to get their apps into Universe or Debian.

Right now PPAs are the easiest way for app developers to deliver their
apps, so that's what they're using.  It makes things worse (and less
safe) for users, but when they are faced with the choice between
installing yet another PPA from someone they don't know, or not getting
the app they want, they have been installing the PPA.

Michael Hall
mhall...@ubuntu.com

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/04/2012 06:05 PM, Scott Kitterman wrote:
> On Tuesday, September 04, 2012 01:19:50 PM Michael Hall wrote:
>> On 09/04/2012 01:02 PM, Emmet Hikory wrote:
>>> Michael Hall wrote:
 Not only would it require the we carry these patches to system tools and
 libraries indefinitely, it would also mean that we encourage developers
 to produce apps and packages that are not compatible with our Upstream
 (Debian) or any other distro.

>>> We rather ought encourage developers to use build systems that allow
>>>
>>> one to specify an install location: then the difference between installing
>>> to /opt/ and system locations becomes entirely a matter of a preference in
>>> the packaging: other format distributions will not even notice. 
>>> Distributions that use Debian-format packages are likely to be downstream
>>> from either Debian or Ubuntu: if we share the optional /opt/ packaging
>>> preference with Debian, then it is trivial for any distribution to adopt
>>> the package as either extra or part of the system with very little
>>> effort.
>>
>> If it can be reduced to a matter of build-time preferences, that would
>> be fine.  The current /opt/ rules and support, however, require both a
>> significant amount of extra work in the packaging, and usually
>> additional changes to the code itself.
>>
>>> As for patches in system tools, at least for icon cache, path, and
>>> desktop
>>>
>>> file discovery, this is one or two lines in a configuration file that
>>> already carries Ubuntu-specific patches.  Is there a specific example of
>>> a discovery mechanism that is more complicated to change?
>>
>> .desktop files, .service files (for dbus), .lens and .scope files (for
>> Unity) all needed to be changed to point to executables or images in
>> /opt/extra.ubuntu.com/${appname}/ when being packaged for Extras.  Unity
>> APIs that require a .desktop or image file name had to use absolute
>> paths to the files in /opt/extras.ubuntu.com/{$appname}.  These all
>> required changes to the app's source in order to support the
>> installation under /opt/.
> 
> These sorts of changes should be generalizable to be supported at build time. 
>  
> It would take some work to set up the first time, but any build system I've 
> worked with would be able to do this.
> 
> Scott K
> 
To support them at build-time, you end up having to make various changes
to the original source.  For example, a typical foo.desktop file will
contain the following lines:

Icon=foo
Exec=foo

Now, if the app has to be installed into /opt/extras.ubuntu.com/foo/,
those lines will need to be changed to:

Icon=/opt/extras.ubuntu.com/foo/foo.png
Exec=/opt/extras.ubuntu.com/foo/bin/foo

Which requires running sed against foo.desktop using overrides in
debian/rules.  We need to do this for DBus services and Unity lenses and
scopes as well.  Other build-time hacks are required to support
translations in /opt/, and apport crashdb.  None of these are especially
difficult, but they accumulate into a large number of changes for even
the most basic application.

I have attached an example debian/rules file from one of the App
Showdown entries.  This was a simple python app with a valid setup.py,
built using Quickly.  You can see how much it's having to change to
install into /opt/, and this was typical of all App Showdown
debian/rules files.

In addition, a quick scan of the source for some of those apps shows a
number of places where paths to /opt/ are being hard-coded elsewhere in
the source.  I know this isn't something they should be hard-coding, but
these are the kind of independent app developers we want to target
Ubuntu and this is what they are likely going to keep doing in order to
support having their app installed in /opt/.


Michael Hall
mhall...@ubuntu.com
#!/usr/bin/make -f
%:
ifneq ($(shell dh -l | grep -xF translations),)
dh $@ --with python2,translations
else
dh $@ --with python2
endif

override_dh_auto_install:
dh_auto_install -- --install-scripts=/opt/extras.ubuntu.com/kwikly  
   --install-data=/opt/extras.ubuntu.com/kwikly 
--install-lib=/opt/extras.ubuntu.com/kwikly

override_dh_python2:
dh_python2 /opt/extras.ubuntu.com/kwikly


override_dh_install:
dh_install
mkdir -p debian/kwikly/opt/extras.ubuntu.com/kwikly/bin
if [ -x debian/kwikly/opt/extras.ubuntu.com/kwikly/kwikly/kwikly ]; 
then mv debian/kwikly/opt/extras.ubuntu.com/kwikly/kwikly/kwikly 
debian/kwikly/opt/extras.ubuntu.com/kwikly/bin; fi
if [ -f 
debian/kwikly/opt/extras.ubuntu.com/kwikly/share/applications/kwikly.desktop ]; 
then \
mkdir -p debian/kwikly/usr/share/applications; \
mv 
debian/kwikly/opt/extras.ubuntu.com/kwikly/share/applications/kwikly.desktop 
debian/kwikly/usr/share/applications/extras-kwikly.desktop; \
rmdir --ignore-fail-on-non-empty 
debian/kwikly/opt/extras.ubuntu.com/kwikly/share/applications

Re: Proposing a New App Developer Upload Process

[Michael Hall]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256


On 09/04/2012 09:55 PM, Steve Langasek wrote:
> On Tue, Sep 04, 2012 at 01:19:50PM -0400, Michael Hall wrote:
> 
>>> As for patches in system tools, at least for icon cache, path,
>>> and desktop file discovery, this is one or two lines in a
>>> configuration file that already carries Ubuntu-specific
>>> patches.  Is there a specific example of a discovery mechanism
>>> that is more complicated to change?
> 
>> .desktop files, .service files (for dbus), .lens and .scope files
>> (for Unity) all needed to be changed to point to executables or
>> images in /opt/extra.ubuntu.com/${appname}/ when being packaged
>> for Extras.  Unity APIs that require a .desktop or image file
>> name had to use absolute paths to the files in
>> /opt/extras.ubuntu.com/{$appname}.  These all required changes to
>> the app's source in order to support the installation under
>> /opt/.
> 
> Note that, independent of any packaging issues for Ubuntu, upstream
> best practice would have the absolute paths in these files
> generated at build time, once it's known where they will be
> installed.  Is there any chance that we could help these app
> developers with implementing such best practices?  Perhaps an
> improvement to the quickly templates?
> 

We can provide helpful abstraction layers, yes, and Quickly's
ubuntu-application template already provides this with
get_media_file() and get_data_file().  But remember that most APIs are
telling the developer to give them a file path, and the developer is
usually going to know the file path, and hard-coding that path will
technically work.  Getting them to consider an abstraction layer
instead is going to be an uphill battle, even if we provide the
methods to do it.


Michael Hall
mhall...@ubuntu.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBCAAGBQJQRrVmAAoJEInUYcGJgfVyWqQP/R8FoW4wJeld2HUME2HcKMvP
ypdImgSiYUjGU/0w3p0ZNjiFHqtYe+SvHOEduYmspuAo1uVh/xQ9AZ2+svq3sc+c
uARurNvyyOUO8amUF58kNRNKNS5Nj7hVxcDiX3Oi5FTfcZBRLbZQ93cs7aUZorrt
i6ps9xieoVNOIVPPfBmBhyCd20owzfktipBaKxG7G98UyMwXIfafWWMkQvkTaav4
pLhefOpIsMxEuDVGo/fnYvsMYJuuwSWGtx9WqMHebDUNTbhFS7RxekjaeFk/BjJR
L6PPy04WGUjE7ucrdtV+004yhY4sQ72G0N0f9WXa2go7wRZNxT3Wd3WiK0G7F4IU
4Te7YEtoPc9cSS34wOlSouQHqozWaKQdytcC32l+OEyhn+xhWiG6iOBZeS/YR+4V
YKyOF6fuRHnIidPngiIiSdbTufiQjUpZsmE5HzGy0v4HsFdIfV0NoQiMQRl6gyYJ
xU0GTsEicTAykELWEWMGcY4LxINEqnDabDnyrur2QqelaEOE3w9aVzwsVn5NiNlA
P7kusaoS8FCKH6aBheoJsdkjW83TmbR4actQdGcElW1pS8/CEaKTcBCtniX60T5Y
N6+w1eYwkpdV0LYKyw5yUVTeZrXbLELZlnOa++oAtCuxd1QHeVdL2xyujiiaOPEG
5ImFhBj/cTNzX9MBJdSb
=7ndq
-END PGP SIGNATURE-

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Steve Langasek wrote:
> > On Wednesday, September 05, 2012 12:11:18 AM Emmet Hikory wrote:
> > > For application developers who prefer standard locations, I don't see
> > > any reason we oughtn't simply add their packages to the repository with
> > > immediate backport: in addition to allowing application developers to put
> > > their files in any policy-compliant location, it automatically provides a
> > > safe upgrade path for users.  Even for software with release cycles that
> > > require significantly more frequent updates than typically provided by our
> > > release cycle, there are few barriers to keeping this updated in 
> > > backports,
> > > and users who have installed from backports will remain with backports, so
> > > their most active and interested users may be expected to always have the
> > > latest version, while highly conservative users will be able to enjoy a
> > > consistent ABI during the life of a given release (although these users
> > > would need to wait for our next release before using the package).
> 
> There's a general sense that the Ubuntu archive can't scale out to the
> degree it needs to in order to take on the next challenges for the platform. 
> While Debian packaging isn't hard for you or me, and while it's definitely
> gotten much easier over the past 4 years or so, it's still not so easy that
> we blindly trust outside contributors to get the packaging right without
> review by an Ubuntu developer.  We do not have an infinite supply of Ubuntu
> developers to do this review.  Should the set of software available to
> Ubuntu users through apt be limited to only that software that Ubuntu
> developers (or Debian maintainers) have time and interest to take care of
> directly?  Or should users have better (not perfect, but better) ways to
> install software that's not gotten the attention of the elite inner circle?

This needn't be an XOR model.  Taking as given that our application
acceptance model has not only failed to scale, but has actually become less
efficient over the past couple years (roughly lucid through precise), and
that we *must* come up with a way to ensure that our users can easily get
new software in a safe manner (as opposed to use of random PPAs, arbitrary
third-party archives, random downloads and local compilation attempts, etc.),
we need not decide that this cannot interoperate with the existing processes
by which we add new applications, nor that the existing processes are now
inflexible to the point of no utility.

Historically, the limitation has often been insufficient reviewers, and
automation is a wonderful way to remove this, especially in combination with
sufficient insulation of the application from the rest of the system.  That
said, we oughtn't force *all* applications to be so limited: there are many
tools that we would welcome as regular packages, assuming we had reviewed
them.  Of course, unless we take care to ensure that, from the perspective
of the application developer, the process for having the package included is
very similar (and most importantly has the same widely-publicised entry point),
we only create division: random arguments about the "best" way to submit
packages in places we fail to see, people deciding not to submit because they
are unsure, or feeling rejected by one process and not wanting to start
another.

So, if we encourage the development of portable applications, and create
automation that can accept these packages, perform (limited) review, and wrap
them in sufficient insulation before presenting them to users, we should be
able to almost completely remove the requirement for human review (although
there are complexities that may require a lightweight gatekeeper check to
avoid repititions of the hotbabe discussions or similar).  That said, if an
application fails the automated insulation testing or requires deeper
integration with the system, we should make it immediately available to
human reviewers who would be empowered to accept the submission as a regular
system package.

For those applications requiring insulation, the use of the extras
repository is likely the simplest route, although I'd personally prefer
that if we do this we either claim extras to be part of Ubuntu (and place
it under similar trust metrics), or create a new repository that we then
consider part of Ubuntu.  For those applications requiring human review,
I don't see any reason we shouldn't strive for them to be included as
part of the regular repositories, and suspect backports is the appropriate
way to do this (although this may require some streamlining of the backports
process for *entirely new* packages, if we find that this is a blocking
factor).

Where this becomes complicated in when we think about trust: who do
we trust to perform the human reviews?  Do we trust application developers
who received human review to update their packages responsibly?  If we
wish to scale, I believe that we need to answer th

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Michael Hall wrote:
> .desktop files, .service files (for dbus), .lens and .scope files (for
> Unity) all needed to be changed to point to executables or images in
> /opt/extra.ubuntu.com/${appname}/ when being packaged for Extras.  Unity
> APIs that require a .desktop or image file name had to use absolute
> paths to the files in /opt/extras.ubuntu.com/{$appname}.  These all
> required changes to the app's source in order to support the
> installation under /opt/.

The XDG menu specification provides very wide flexibility in where
and how .desktop files are stored, and all the compliant implementations
of which I am aware use a configuration file to determine where to find
the files, and which to include where.  Similarly, dbus service files
may be placed in any of ${XDG_DATA_DIRS}/share/dbus-1/services.  Icons
(for use in .desktop files) may be stored in ${XDG_DATA_DIRS}/icons.  I
know less about .lens and .scope files, but presume that there would be
advantages beyond this discussion to have them check a runtime-determined
set of locations.

Where this gets extra annoying is the expectation that things
are found in application-specific locations: while it is possible
to add /opt/${PACKAGE}/applications to the menu search path for
each package, this quickly becomes unwieldy.  While it reduces the
separation between applications to some degree, use of a shared
namespace for this sort of file (e.g. /opt/extras.ubuntu.com/applications
or /opt/extras.ubuntu.com/share/dbus-1/services) reduces the issue to
a simple configuration change and a symlink from the shared location
to the package namespace.

The disadvantage of doing it this way is that it imposes a requirement
for all extras packages to share a single namespace for these resources
(desktop files, icon names, dbus services, etc.), although on a given system
we would likely wish to impose that anyway (two packages providing the same
apparent program name, or the same apparent dbus service may lead to many
more complicated issues in terms of user confusion than fienames).

-- 
Emmet HIKORY

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Tuesday, September 04, 2012 06:01:56 PM Steve Langasek wrote:
> Hi Scott,
> 
> On Tue, Sep 04, 2012 at 05:50:12PM -0400, Scott Kitterman wrote:
> > On Wednesday, September 05, 2012 12:11:18 AM Emmet Hikory wrote:
> > > For application developers who prefer standard locations, I don't
> > > see
> > > 
> > > any reason we oughtn't simply add their packages to the repository with
> > > immediate backport: in addition to allowing application developers to
> > > put
> > > their files in any policy-compliant location, it automatically provides
> > > a
> > > safe upgrade path for users.  Even for software with release cycles that
> > > require significantly more frequent updates than typically provided by
> > > our
> > > release cycle, there are few barriers to keeping this updated in
> > > backports,
> > > and users who have installed from backports will remain with backports,
> > > so
> > > their most active and interested users may be expected to always have
> > > the
> > > latest version, while highly conservative users will be able to enjoy a
> > > consistent ABI during the life of a given release (although these users
> > > would need to wait for our next release before using the package).
> > 
> > +1.  I've never understood why there is resistance to this.
> 
> There's a general sense that the Ubuntu archive can't scale out to the
> degree it needs to in order to take on the next challenges for the platform.
> While Debian packaging isn't hard for you or me, and while it's definitely
> gotten much easier over the past 4 years or so, it's still not so easy that
> we blindly trust outside contributors to get the packaging right without
> review by an Ubuntu developer.  We do not have an infinite supply of Ubuntu
> developers to do this review.  Should the set of software available to
> Ubuntu users through apt be limited to only that software that Ubuntu
> developers (or Debian maintainers) have time and interest to take care of
> directly?  Or should users have better (not perfect, but better) ways to
> install software that's not gotten the attention of the elite inner circle?

First, this elite inner circle has an open door.  Second we also have a 
limited supply of ARB reviewers.  Starting a new review process (ARB) because 
their aren't enough reviewers was clearly doomed from the start, so much of my 
"I don't understand this" comes from the current process where we substitute 
one review process with insufficient reviewers for another one with 
insufficient 
reviewers.

Additionally, I think the notion that "Oh, X has a bazillion apps, so we need 
that many too" is mistaken in many regards.  How many office suites do we need? 
 
I'd say one that works robustly, reliably, and compatibly with it's 
proprietary competition (and despite a huge amount of work by people deeply 
interested in solving this problem, IMO we don't have it).   How many 
solitaire games do we need?  I'm not sure, but I'm confident it's fewer than 
one finds in whatever Android Marketplace is called now.

Historically, Linux distros have included a curated collection, some larger, 
some smaller, of relevant applications, libraries, etc that can be used on the 
base operating system.  That curation process is one of the real strengths of 
Linux distributions and I think "Oh, let's have a bazillion of everything 
because it's there" is the wrong way to go.

Yes, extras is separate, but not in any way that's meaningful to a user.  
Fundamentally, I think this process takes us in the wrong direction.

So far the processes are not appropriate to the stated goal and the goal is, 
IMO, rather questionable.

> The intent of Extras was always to make it easy for upstream developers who
> know nothing about packaging policy to get their software in the hands of
> users in as reliable and bug-free a manner as possible.  To date, it has
> fallen far short of this goal.  We see 130 apps written for the recent app
> contest that will never make it into Ubuntu or Debian, because no one will
> ever package them the traditional way; and I think this is the tip of the
> iceberg.  Provided we can address the app isolation issues and streamline
> the packaging helpers, doesn't it make sense for us to make this software
> available to our users - all of it, not just the ones an Ubuntu dev looks at
> and decides are personally interesting to them?  Sure, there's going to be
> some low-quality stuff in there that our users aren't going to want.  But
> isn't it better to let users judge this quality for themselves instead of
> having it pre-judged by what Ubuntu developers choose to work on?  I may
> know a lot about putting together a distribution, but I'm a terrible judge
> of what the next big thing will be that's important to users, and I'd
> rather our users didn't miss out on it because of my poor ability to pick
> the winners.

There are a LOT of Debian/Ubuntu developers and non-developers involved in 
packaging, so I suspect the good st

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Steve Langasek wrote:
> On Tue, Sep 04, 2012 at 05:20:47PM -0400, Michael Hall wrote:
> > > It's possible that namespacing within /usr - for instance,
> > > requiring each subdirectory and binary name to be prefixed with
> > > 'extras.' - would give better results with regards to the upstream
> > > build systems, I'm not sure. But if we are going to try to do
> > > namespacing of extras packages to avoid the need for coordination,
> > > we definitely would need improved package helpers that support
> > > namespacing of packages (i.e., debhelper support).
> 
> > Would it be reasonable for MyApps to add a package name prefix to the
> > submitted package, rather than requiring the developer to do so?
> > MyApps will already be modifying the submitted package to include the
> > generator AppArmor profile.
> 
> > So, for example, if I submit quickly-gtk as a source package to
> > MyApps, it would convert it to extras-quickly-gtk, add the AppArmor
> > profile, and re-build the source package before submitting it to the
> > PPA/buildds.
> 
> For package renaming that seems easy enough to do, but if we're worried
> about file conflicts, dynamically prefixing filenames when building the
> package is going to have an even worse impact on the upstream code than
> changing directories does.

Changing the base filename indeed generates more complicated issues,
not only in terms of collisions, but also in terms of coordination between
code and data (e.g. code expects to find ${CONFIG_DIR}/package and fails
to initialise properly with only ${CONFIG_DIR}/extras-package.

However, if we consider "renaming" to apply to the entire path, rather
than the bare filename, this becomes considerably less of an issue.  If
MyApps were to call tooling that changed the default installation location
for files while preserving bare filenames, then there is less potential
for conflict.  The standard mechanism for this is to place the non-system
files in per-package namespaced directories in /opt.

For the convenience of developers, this should likely be implemented
through debhelper support, so that if one specifies `dh --with-extras`
one gets the alternate installation prefix, as well as the apparmor
profile, etc.

-- 
Emmet HIKORY

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Steve Langasek]

On Tue, Sep 04, 2012 at 01:19:50PM -0400, Michael Hall wrote:

> > As for patches in system tools, at least for icon cache, path, and 
> > desktop
> > file discovery, this is one or two lines in a configuration file that 
> > already
> > carries Ubuntu-specific patches.  Is there a specific example of a discovery
> > mechanism that is more complicated to change?

> .desktop files, .service files (for dbus), .lens and .scope files (for
> Unity) all needed to be changed to point to executables or images in
> /opt/extra.ubuntu.com/${appname}/ when being packaged for Extras.  Unity
> APIs that require a .desktop or image file name had to use absolute
> paths to the files in /opt/extras.ubuntu.com/{$appname}.  These all
> required changes to the app's source in order to support the
> installation under /opt/.

Note that, independent of any packaging issues for Ubuntu, upstream best
practice would have the absolute paths in these files generated at build
time, once it's known where they will be installed.  Is there any chance
that we could help these app developers with implementing such best
practices?  Perhaps an improvement to the quickly templates?

On Tue, Sep 04, 2012 at 05:20:47PM -0400, Michael Hall wrote:
> > It's possible that namespacing within /usr - for instance,
> > requiring each subdirectory and binary name to be prefixed with
> > 'extras.' - would give better results with regards to the upstream
> > build systems, I'm not sure. But if we are going to try to do
> > namespacing of extras packages to avoid the need for coordination,
> > we definitely would need improved package helpers that support
> > namespacing of packages (i.e., debhelper support).

> Would it be reasonable for MyApps to add a package name prefix to the
> submitted package, rather than requiring the developer to do so?
> MyApps will already be modifying the submitted package to include the
> generator AppArmor profile.

> So, for example, if I submit quickly-gtk as a source package to
> MyApps, it would convert it to extras-quickly-gtk, add the AppArmor
> profile, and re-build the source package before submitting it to the
> PPA/buildds.

For package renaming that seems easy enough to do, but if we're worried
about file conflicts, dynamically prefixing filenames when building the
package is going to have an even worse impact on the upstream code than
changing directories does.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Steve Langasek]

Hi Scott,

On Tue, Sep 04, 2012 at 05:50:12PM -0400, Scott Kitterman wrote:
> On Wednesday, September 05, 2012 12:11:18 AM Emmet Hikory wrote:
> > For application developers who prefer standard locations, I don't see
> > any reason we oughtn't simply add their packages to the repository with
> > immediate backport: in addition to allowing application developers to put
> > their files in any policy-compliant location, it automatically provides a
> > safe upgrade path for users.  Even for software with release cycles that
> > require significantly more frequent updates than typically provided by our
> > release cycle, there are few barriers to keeping this updated in backports,
> > and users who have installed from backports will remain with backports, so
> > their most active and interested users may be expected to always have the
> > latest version, while highly conservative users will be able to enjoy a
> > consistent ABI during the life of a given release (although these users
> > would need to wait for our next release before using the package).

> +1.  I've never understood why there is resistance to this.

There's a general sense that the Ubuntu archive can't scale out to the
degree it needs to in order to take on the next challenges for the platform. 
While Debian packaging isn't hard for you or me, and while it's definitely
gotten much easier over the past 4 years or so, it's still not so easy that
we blindly trust outside contributors to get the packaging right without
review by an Ubuntu developer.  We do not have an infinite supply of Ubuntu
developers to do this review.  Should the set of software available to
Ubuntu users through apt be limited to only that software that Ubuntu
developers (or Debian maintainers) have time and interest to take care of
directly?  Or should users have better (not perfect, but better) ways to
install software that's not gotten the attention of the elite inner circle?

The intent of Extras was always to make it easy for upstream developers who
know nothing about packaging policy to get their software in the hands of
users in as reliable and bug-free a manner as possible.  To date, it has
fallen far short of this goal.  We see 130 apps written for the recent app
contest that will never make it into Ubuntu or Debian, because no one will
ever package them the traditional way; and I think this is the tip of the
iceberg.  Provided we can address the app isolation issues and streamline
the packaging helpers, doesn't it make sense for us to make this software
available to our users - all of it, not just the ones an Ubuntu dev looks at
and decides are personally interesting to them?  Sure, there's going to be
some low-quality stuff in there that our users aren't going to want.  But
isn't it better to let users judge this quality for themselves instead of
having it pre-judged by what Ubuntu developers choose to work on?  I may
know a lot about putting together a distribution, but I'm a terrible judge
of what the next big thing will be that's important to users, and I'd rather
our users didn't miss out on it because of my poor ability to pick the
winners.

Given this goal, then, backports is not a solution.  Backports is a
necessarily heavy-weight process, of getting the package prepared,
sponsored (possibly after several rounds of feedback), through the NEW queue
for the devel release, and only then backported.  That works for some
upstreams who can invest that time and energy, but it won't work for all of
them, and if it did I think the backports team would quickly reconsider the
workload.  And it's a process that largely doesn't map to the goals of the
upstream who is trying to get their software into the hands of the users
running the current stable release, *not* to get their software integrated
into Ubuntu itself.  It's entirely possible that some of this software won't
be relevant by the time the next Ubuntu release comes out!  Why put an
upstream through a process optimized for long-term integration of the
distribution when all they care about is getting an app out to users that
gives them information about this month's beer festival?

Does this explain better why there's resistance to using the backports
process for this class of package?

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

Michael Hall  wrote:

>-BEGIN PGP SIGNED MESSAGE-
>Hash: SHA256
>
>
>On 09/04/2012 04:41 PM, Steve Langasek wrote:
>> Hi Michael,
>> 
>> On Tue, Sep 04, 2012 at 10:50:21AM -0400, Michael Hall wrote:
> On 09/04/2012 09:39 AM, Scott Kitterman wrote:
>> The problem isn't just with file conflicts with current
>> packages, it's that these packages will now start using up
>> distro namespace.  If some app developer package ships the
>> file /usr/games/bird-game, even though there's no current
>> conflict, there is a package sync'ed from Debian that also
>> ships /usr/games/bird-game then there's a conflict we have
>> to resolve.  In /opt in a proper vendor namespace this can
>> never happen.
>> 
> If bird-game already exists in Extras, and then a different
> package is allowed into backports that will install files
> into the same location, then yes there is a possibility for a
> conflict.  But I assume part of the backports approval
> process already checks for conflicts, as they may exist with
> another package in the stable release already, so that 
> process could easily be extended to include Extras packages
> as well.
>> 
 I think people are more concerned with auto-import from Debian
 than backports. Debian's approval process knows nothing about
 Extras, so there is no mechanism or approval process that
 checks for conflicts.
>> 
>>> Since Extras package won't be in the development release until 
>>> FeatureFreeze, we can check them against the new packages
>>> imported from Debian before we forward-copy them from the
>>> previous stable release.
>> 
>> That's possible, but has some drawbacks:
>> 
>> - If Ubuntu chooses to rename the package in universe to address
>> this collision, that likely increases the burden on Ubuntu: Debian
>> is unlikely to agree to rename a package, or binaries within a
>> package, in response to namespace claims from third-party software
>> that's not in the distro. - If the package in universe is given
>> precedence, then users may have a frustrating experience when
>> upgrading to the new release: either update-manager needs to do
>> some complex mapping of extras packages to find the new name (if
>> any) of the package, or all extras packages that don't exist in the
>> new release need to be removed before upgrading.
>> 
>> In either case, it seems a certain amount of coordination would be
>> required to provide users a good experience in upgrades, and we
>> quite reasonably want to avoid the need for that coordination.  The
>> latter option, which seems to require the least coordination, would
>> also leave users who don't use update-manager for their updates out
>> in the cold.  And neither option really addresses the backports
>> issue, where a package may become available in a devel release that
>> there's user demand for post release, but there's already a package
>> of the same name (but different origin) in extras.
>> 
>> A namespace would be a good thing because it saves us from having
>> to deal with these coordination issues.  The /opt namespace hasn't
>> shown itself to be viable, in part because we don't have tooling
>> that will usefully generate the binary packages with the correct
>> layout without a lot of fuss, and also in part because upstream
>> sources seem not to be equipped to handle the split between /opt
>> and /usr for the points where the package will need to integrate.
>> 
>> It's possible that namespacing within /usr - for instance,
>> requiring each subdirectory and binary name to be prefixed with
>> 'extras.' - would give better results with regards to the upstream
>> build systems, I'm not sure. But if we are going to try to do
>> namespacing of extras packages to avoid the need for coordination,
>> we definitely would need improved package helpers that support
>> namespacing of packages (i.e., debhelper support).
>> 
>> 
>> 
>
>Would it be reasonable for MyApps to add a package name prefix to the
>submitted package, rather than requiring the developer to do so?
>MyApps will already be modifying the submitted package to include the
>generator AppArmor profile.
>
>So, for example, if I submit quickly-gtk as a source package to
>MyApps, it would convert it to extras-quickly-gtk, add the AppArmor
>profile, and re-build the source package before submitting it to the
>PPA/buildds.

I think that's the right direction to be looking in, both for package names and 
files in the system path.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Rick Spencer]

David, et al ...

Thanks for the hard word and driving to such detail.

I think the foundation of most of this work is the section on
application sandboxing. If we can make it much less likely that a
poorly written or out and out malicious application will bite users,
than we will have a lot more freedom to automate the process. This, in
turn, will allow us to let application developers reach users and
update their applications in a way that makes sense for their apps and
their users, without needing a third party (us!) to get in between.

Until we have the sandboxing in place, I don't think that there is
really much we can do dramatically improve this situation. As such, I
think sandboxing should to be a high priority for 13.04.

Cheers, Rick

PS: For dorky reasons, I prefer the term "application insulation" to
"sandboxing"

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256


On 09/04/2012 04:41 PM, Steve Langasek wrote:
> Hi Michael,
> 
> On Tue, Sep 04, 2012 at 10:50:21AM -0400, Michael Hall wrote:
 On 09/04/2012 09:39 AM, Scott Kitterman wrote:
> The problem isn't just with file conflicts with current
> packages, it's that these packages will now start using up
> distro namespace.  If some app developer package ships the
> file /usr/games/bird-game, even though there's no current
> conflict, there is a package sync'ed from Debian that also
> ships /usr/games/bird-game then there's a conflict we have
> to resolve.  In /opt in a proper vendor namespace this can
> never happen.
> 
 If bird-game already exists in Extras, and then a different
 package is allowed into backports that will install files
 into the same location, then yes there is a possibility for a
 conflict.  But I assume part of the backports approval
 process already checks for conflicts, as they may exist with
 another package in the stable release already, so that 
 process could easily be extended to include Extras packages
 as well.
> 
>>> I think people are more concerned with auto-import from Debian
>>> than backports. Debian's approval process knows nothing about
>>> Extras, so there is no mechanism or approval process that
>>> checks for conflicts.
> 
>> Since Extras package won't be in the development release until 
>> FeatureFreeze, we can check them against the new packages
>> imported from Debian before we forward-copy them from the
>> previous stable release.
> 
> That's possible, but has some drawbacks:
> 
> - If Ubuntu chooses to rename the package in universe to address
> this collision, that likely increases the burden on Ubuntu: Debian
> is unlikely to agree to rename a package, or binaries within a
> package, in response to namespace claims from third-party software
> that's not in the distro. - If the package in universe is given
> precedence, then users may have a frustrating experience when
> upgrading to the new release: either update-manager needs to do
> some complex mapping of extras packages to find the new name (if
> any) of the package, or all extras packages that don't exist in the
> new release need to be removed before upgrading.
> 
> In either case, it seems a certain amount of coordination would be
> required to provide users a good experience in upgrades, and we
> quite reasonably want to avoid the need for that coordination.  The
> latter option, which seems to require the least coordination, would
> also leave users who don't use update-manager for their updates out
> in the cold.  And neither option really addresses the backports
> issue, where a package may become available in a devel release that
> there's user demand for post release, but there's already a package
> of the same name (but different origin) in extras.
> 
> A namespace would be a good thing because it saves us from having
> to deal with these coordination issues.  The /opt namespace hasn't
> shown itself to be viable, in part because we don't have tooling
> that will usefully generate the binary packages with the correct
> layout without a lot of fuss, and also in part because upstream
> sources seem not to be equipped to handle the split between /opt
> and /usr for the points where the package will need to integrate.
> 
> It's possible that namespacing within /usr - for instance,
> requiring each subdirectory and binary name to be prefixed with
> 'extras.' - would give better results with regards to the upstream
> build systems, I'm not sure. But if we are going to try to do
> namespacing of extras packages to avoid the need for coordination,
> we definitely would need improved package helpers that support
> namespacing of packages (i.e., debhelper support).
> 
> 
> 

Would it be reasonable for MyApps to add a package name prefix to the
submitted package, rather than requiring the developer to do so?
MyApps will already be modifying the submitted package to include the
generator AppArmor profile.

So, for example, if I submit quickly-gtk as a source package to
MyApps, it would convert it to extras-quickly-gtk, add the AppArmor
profile, and re-build the source package before submitting it to the
PPA/buildds.

Michael Hall
mhall...@ubuntu.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBCAAGBQJQRnCvAAoJEInUYcGJgfVyWmoP/R5V3ASIAMwH0uZSyxr3A43s
NT7YUFnGnCL45ax4klEUGezEO3iwHanAHlEhgPDyLjiFXLir7AIVpCtF0X8aNHMD
we+14KgJEoq4lAnGzPyx13Enf7UikSSQqXnYVbTl19KG0bxtHhbU9/5r3NO/CtAq
pJS6yTKRZQbmFlhAd5HcgpHD0V/WXJirYdewdShxqYx9A9srptMwtHneTFC3Rxbh
Z+nU3Kj38GA63QQYMYSu0kPgF6vXXOaOuWDuTPxhR40p8Qk+jSpBfBMxw+rvVYFN
/dWJtFKuTwjEDf1BXuzegmBjx2V2jyy1hY1bcFriXai6EwDsj/II3ivwPSFTSMF7
uUeFhYF9rv34GAIWN7NSv80UW6ULsnIYivuIUfRKDgAKkKkYKGAsDhrN+UGOS8a5
9T5Ifmphnpc3aMeRBL63eDFgICm2e+v/lwqep/J906Wy/Ew+49He9iAYGS1h6VkF
C1RIwdcJNQ4v3nYHl

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Tuesday, September 04, 2012 01:19:50 PM Michael Hall wrote:
> On 09/04/2012 01:02 PM, Emmet Hikory wrote:
> > Michael Hall wrote:
> >> Not only would it require the we carry these patches to system tools and
> >> libraries indefinitely, it would also mean that we encourage developers
> >> to produce apps and packages that are not compatible with our Upstream
> >> (Debian) or any other distro.
> >> 
> > We rather ought encourage developers to use build systems that allow
> > 
> > one to specify an install location: then the difference between installing
> > to /opt/ and system locations becomes entirely a matter of a preference in
> > the packaging: other format distributions will not even notice. 
> > Distributions that use Debian-format packages are likely to be downstream
> > from either Debian or Ubuntu: if we share the optional /opt/ packaging
> > preference with Debian, then it is trivial for any distribution to adopt
> > the package as either extra or part of the system with very little
> > effort.
> 
> If it can be reduced to a matter of build-time preferences, that would
> be fine.  The current /opt/ rules and support, however, require both a
> significant amount of extra work in the packaging, and usually
> additional changes to the code itself.
> 
> > As for patches in system tools, at least for icon cache, path, and
> > desktop
> > 
> > file discovery, this is one or two lines in a configuration file that
> > already carries Ubuntu-specific patches.  Is there a specific example of
> > a discovery mechanism that is more complicated to change?
> 
> .desktop files, .service files (for dbus), .lens and .scope files (for
> Unity) all needed to be changed to point to executables or images in
> /opt/extra.ubuntu.com/${appname}/ when being packaged for Extras.  Unity
> APIs that require a .desktop or image file name had to use absolute
> paths to the files in /opt/extras.ubuntu.com/{$appname}.  These all
> required changes to the app's source in order to support the
> installation under /opt/.

These sorts of changes should be generalizable to be supported at build time.  
It would take some work to set up the first time, but any build system I've 
worked with would be able to do this.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Tuesday, September 04, 2012 11:24:28 AM Michael Hall wrote:
> On 09/04/2012 11:11 AM, Emmet Hikory wrote:
> > Daniel Holbach wrote:
> >> On 04.09.2012 15:28, Emmet Hikory wrote:
> >>> The difficulty here is that there is uncontrolled scope for future
> >>> conflict.  While the Contents.gz file is useful (and the conflictchecker
> >>> system more so), if both extras and backports are enabled by default,
> >>> there
> >>> is no means by which the review board can determine that a given
> >>> filename
> >>> will not be provided by a backport of a new package imported from
> >>> Debian.
> >> 
> >> The fairest solution to this problem would be to turn on a
> >> conflict-check-before-publish for all parts of the archive. This would
> >> help us find these (and pre-existing) issues immediately and resolve
> >> them amongst the maintainers and upstreams.
> > Completely independently of this discussion, this is a good idea,
> > 
> > although we probably ought have the system respect Breaks, Conflicts,
> > and Replaces, or we'll end up with lots of false positives.  Mind you,
> > given the vast number of reports from conflictchecker, it may be that
> > we would want to put up a review system for some time to clean up the
> > existing issues prior to actually blocking publication.
> > 
> > That said, while I agree that such issues can be sorted between the
> > various developers involved (perhaps easily, perhaps less so (see
> > "node")),
> > I am much more concerned about the effect on users.  Assume I'm using
> > Ubuntu Desktop, and I add an extras application to the Dock.  Now assume
> > there is a namespace conflict for that application, which is resolved by
> > the extras application taking a new name.  Depending on what I happened
> > to install on upgrade (as a new dependency of another installed package),
> > my Dock entry may behave entirely differently.  More generally, this may
> > happen for startup programs, so that I would upgrade, reboot, and end up
> > running something completely unexpected automatically, and I may not even
> > notice if the automatic execution doesn't load a window, and just believe
> > my upgrade broke (and potentially end up finding my computer slower, or
> > behaving oddly, depending on what the new program happened to do).
> 
> We could alternately prevent the importation of packages that conflict
> with a package name already in Extras.  That would prevent this from
> happening.  We would also have to prevent the importation of packages
> that Depends or Recommends on the package we won't be importing.  This
> would essentially make package names "first come, first served" between
> Extras and Debian.

Blocking Ubuntu development based on what's in external repositories would be 
an atrocity.  I for one would take the issue to the tech board and ask them no 
to allow such a thing.

> >> The /opt requirement on the other hand unfortunately imposes a huge
> >> amount of work on 1) app developers because our tools don't work this
> >> way very easily and 2) Ubuntu maintainers who have to enable path
> >> lookups in tools which don't know about /opt yet.
> >> 
> > Much of this can be avoided by allowing a common location for extras
> > 
> > files, for example /opt/extras/bin, /opt/extras/applications,
> > /opt/extras/pixmaps, etc.  If these locations are only permitted to
> > contain
> > namespace-protected symlinks, then it can be a matter of adding one or two
> > lines to the 8-12 tools that perform these sorts of discoveries.  No, this
> > isn't an ideal solution, but it significantly limits the effort required
> > to support a separate namespace.
> 
> Not only would it require the we carry these patches to system tools and
> libraries indefinitely, it would also mean that we encourage developers
> to produce apps and packages that are not compatible with our Upstream
> (Debian) or any other distro.

There is a standard process that people can use to get packages into the 
distribution.  It's not at all surprising that an alternative infrastructure 
would require alternative tools and processes.  I don't understand what's so 
hard about /opt (for any gui application it's the .desktop file that's going to 
drive the applications being started and that can point anywhere).

If the alternative to maintaining some extra processes and tools is to drive a 
namespace wedge between Debian and Ubuntu then I don't think you have a choice 
but to suck up the cost of providing the alternate process/tools if a 
Canonical standard method of delivering third party apps on Ubuntu is what 
Canonical wants to provide.

> > For application developers who prefer standard locations, I don't see
> > 
> > any reason we oughtn't simply add their packages to the repository with
> > immediate backport: in addition to allowing application developers to put
> > their files in any policy-compliant location, it automatically provides a
> > safe upgrade path for users.  Even for softwar

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Wednesday, September 05, 2012 12:11:18 AM Emmet Hikory wrote:
> For application developers who prefer standard locations, I don't see
> any reason we oughtn't simply add their packages to the repository with
> immediate backport: in addition to allowing application developers to put
> their files in any policy-compliant location, it automatically provides a
> safe upgrade path for users.  Even for software with release cycles that
> require significantly more frequent updates than typically provided by our
> release cycle, there are few barriers to keeping this updated in backports,
> and users who have installed from backports will remain with backports, so
> their most active and interested users may be expected to always have the
> latest version, while highly conservative users will be able to enjoy a
> consistent ABI during the life of a given release (although these users
> would need to wait for our next release before using the package).

+1.  I've never understood why there is resistance to this.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Tuesday, September 04, 2012 04:25:56 PM Daniel Holbach wrote:
> Hello,
> 
> On 04.09.2012 15:28, Emmet Hikory wrote:
> > The difficulty here is that there is uncontrolled scope for future
> > 
> > conflict.  While the Contents.gz file is useful (and the conflictchecker
> > system more so), if both extras and backports are enabled by default,
> > there
> > is no means by which the review board can determine that a given filename
> > will not be provided by a backport of a new package imported from Debian.
> 
> The fairest solution to this problem would be to turn on a
> conflict-check-before-publish for all parts of the archive. This would
> help us find these (and pre-existing) issues immediately and resolve
> them amongst the maintainers and upstreams.
> 
> My current expectation is only a very tiny fraction of uploads would get
> blocked due to this and the general amount of work to resolve them would
> be small too.
> 
> The /opt requirement on the other hand unfortunately imposes a huge
> amount of work on 1) app developers because our tools don't work this
> way very easily and 2) Ubuntu maintainers who have to enable path
> lookups in tools which don't know about /opt yet.

It seems to me there's an assumption embedded in that response that if an 
extras packages gets a namespace first then it 'owns' it.  I think that's 
inappropriate for that to be the case for any external repository.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Tuesday, September 04, 2012 10:21:15 AM Michael Hall wrote:
> On 09/04/2012 09:39 AM, Scott Kitterman wrote:
> > The problem isn't just with file conflicts with current packages, it's
> > that
> > these packages will now start using up distro namespace.  If some app
> > developer package ships the file /usr/games/bird-game, even though there's
> > no current conflict, there is a package sync'ed from Debian that also
> > ships /usr/games/bird-game then there's a conflict we have to resolve. 
> > In /opt in a proper vendor namespace this can never happen.
> 
> If bird-game already exists in Extras, and then a different package is
> allowed into backports that will install files into the same location,
> then yes there is a possibility for a conflict.  But I assume part of
> the backports approval process already checks for conflicts, as they may
> exist with another package in the stable release already, so that
> process could easily be extended to include Extras packages as well.

No.  There isn't.  There is a (reasonable) presumption that such issues have 
been resolved in the development release.  I would not be in favor of 
increasing the testing requirements for backports to accommodate external 
repositories.

> > A fairly contrived example derived from the above is if the Debian/Ubuntu
> > package that shipped /usr/games/bird-game was in the archive and an app
> > developer package was uploaded that shipped /usr/bin/bird-game, it could
> > be
> > run instead since /usr/bin precedes /usr/games on the standard user path.
> 
> This would only happen if two different apps were both called
> "bird-game" but otherwise different.  This situation is also possible
> already, even between two packages in Universe.  How is that currently
> handled?

Keep in mind there's no need for package name in the binary name to match, but 
if such a conflict happens, it's an RC bug in Debian terms and one has to be 
renamed.  The Debian bug on /usr/bin/node is an example of how ugly this can 
get.  It's not just enough to check for actual file conflicts though, you have 
to check the entire path to ensure that no files preempt files from other 
packages.  Installing in /opt makes this a non-problem.

> > maybe a Python based app ships an embedded copy of a module since they've
> > tested with that version and don't want to test with the distro version
> > and
> > they write their setup.py to install it in /usr/local/lib/python2.7/dist-
> > packages.  Suddenly every user of that module is now using the app
> > developer package version and not the distro version.
> 
> Installing python modules in the system directories is not allowed under
> this process.  Check the whitelist defined in the "App Preparation"
> section of the spec.

OK.  It seems odd not to allow install in /usr/games.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Kitterman]

On Tuesday, September 04, 2012 03:12:05 PM Matt Wheeler wrote:
> On Sep 4, 2012 3:00 PM, "Michael Terry"  wrote:
> [snip]
> 
> > (Though with namespaced packages, the user can end up in a weird
> 
> situation if the Extras package "foo" does find its way into Debian and
> Ubuntu quantal.  Then, they really do want the "foo" package from Debian
> but are left with the non-maintained "extras-foo" from precise.)
> 
> That can be handled the usual way package name transitions happen (e.g.
> git-core), perhaps it could even be totally automated?

To do that properly requires the package namespace be unused through an 
upgrade cycle.  Since LTS to LTS upgrades are supported, that means that if an 
app developer package was added to 12.04, the soonest it could be reused 
appropriately for another package in a git-core -> git scenario is in 14.10.

If they are, in fact, the same package then it's not an issue, but if it's two 
different packages fighting over the same name as in the case of git, it's not 
trivially resolvable.

Scott K

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/04/2012 01:02 PM, Emmet Hikory wrote:
> Michael Hall wrote:
>> Not only would it require the we carry these patches to system tools and
>> libraries indefinitely, it would also mean that we encourage developers
>> to produce apps and packages that are not compatible with our Upstream
>> (Debian) or any other distro.
> 
> We rather ought encourage developers to use build systems that allow
> one to specify an install location: then the difference between installing
> to /opt/ and system locations becomes entirely a matter of a preference in
> the packaging: other format distributions will not even notice.  Distributions
> that use Debian-format packages are likely to be downstream from either Debian
> or Ubuntu: if we share the optional /opt/ packaging preference with Debian,
> then it is trivial for any distribution to adopt the package as either extra
> or part of the system with very little effort.
> 
If it can be reduced to a matter of build-time preferences, that would
be fine.  The current /opt/ rules and support, however, require both a
significant amount of extra work in the packaging, and usually
additional changes to the code itself.

> As for patches in system tools, at least for icon cache, path, and desktop
> file discovery, this is one or two lines in a configuration file that already
> carries Ubuntu-specific patches.  Is there a specific example of a discovery
> mechanism that is more complicated to change?
> 
.desktop files, .service files (for dbus), .lens and .scope files (for
Unity) all needed to be changed to point to executables or images in
/opt/extra.ubuntu.com/${appname}/ when being packaged for Extras.  Unity
APIs that require a .desktop or image file name had to use absolute
paths to the files in /opt/extras.ubuntu.com/{$appname}.  These all
required changes to the app's source in order to support the
installation under /opt/.


Michael Hall
mhall...@ubuntu.com

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Steve Langasek]

Hi Michael,

On Tue, Sep 04, 2012 at 10:50:21AM -0400, Michael Hall wrote:
> >> On 09/04/2012 09:39 AM, Scott Kitterman wrote:
> >>> The problem isn't just with file conflicts with current packages, it's 
> >>> that
> >>> these packages will now start using up distro namespace.  If some app
> >>> developer package ships the file /usr/games/bird-game, even though 
> >>> there's no
> >>> current conflict, there is a package sync'ed from Debian that also ships
> >>> /usr/games/bird-game then there's a conflict we have to resolve.  In /opt 
> >>> in a
> >>> proper vendor namespace this can never happen.

> >> If bird-game already exists in Extras, and then a different package is
> >> allowed into backports that will install files into the same location,
> >> then yes there is a possibility for a conflict.  But I assume part of
> >> the backports approval process already checks for conflicts, as they may
> >> exist with another package in the stable release already, so that
> >> process could easily be extended to include Extras packages as well.

> > I think people are more concerned with auto-import from Debian than
> > backports. Debian's approval process knows nothing about Extras, so
> > there is no mechanism or approval process that checks for conflicts.

> Since Extras package won't be in the development release until
> FeatureFreeze, we can check them against the new packages imported from
> Debian before we forward-copy them from the previous stable release.

That's possible, but has some drawbacks:

 - If Ubuntu chooses to rename the package in universe to address this
   collision, that likely increases the burden on Ubuntu: Debian is unlikely
   to agree to rename a package, or binaries within a package, in response to
   namespace claims from third-party software that's not in the distro.
 - If the package in universe is given precedence, then users may have a
   frustrating experience when upgrading to the new release: either
   update-manager needs to do some complex mapping of extras packages to
   find the new name (if any) of the package, or all extras packages that
   don't exist in the new release need to be removed before upgrading.

In either case, it seems a certain amount of coordination would be required
to provide users a good experience in upgrades, and we quite reasonably want
to avoid the need for that coordination.  The latter option, which seems to
require the least coordination, would also leave users who don't use
update-manager for their updates out in the cold.  And neither option really
addresses the backports issue, where a package may become available in a
devel release that there's user demand for post release, but there's already
a package of the same name (but different origin) in extras.

A namespace would be a good thing because it saves us from having to deal
with these coordination issues.  The /opt namespace hasn't shown itself to
be viable, in part because we don't have tooling that will usefully generate
the binary packages with the correct layout without a lot of fuss, and also
in part because upstream sources seem not to be equipped to handle the split
between /opt and /usr for the points where the package will need to
integrate.

It's possible that namespacing within /usr - for instance, requiring each
subdirectory and binary name to be prefixed with 'extras.' - would give
better results with regards to the upstream build systems, I'm not sure. 
But if we are going to try to do namespacing of extras packages to avoid the
need for coordination, we definitely would need improved package helpers
that support namespacing of packages (i.e., debhelper support).

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Michael Hall wrote:
> We could alternately prevent the importation of packages that conflict
> with a package name already in Extras.  That would prevent this from
> happening.  We would also have to prevent the importation of packages
> that Depends or Recommends on the package we won't be importing.  This
> would essentially make package names "first come, first served" between
> Extras and Debian.

Package names are relatively easy: there aren't that many of them,
and they can be namespaced without significant impact.  My concern is
about filenames.

> >> The /opt requirement on the other hand unfortunately imposes a huge
> >> amount of work on 1) app developers because our tools don't work this
> >> way very easily and 2) Ubuntu maintainers who have to enable path
> >> lookups in tools which don't know about /opt yet.
> > 
> > Much of this can be avoided by allowing a common location for extras
> > files, for example /opt/extras/bin, /opt/extras/applications,
> > /opt/extras/pixmaps, etc.  If these locations are only permitted to contain
> > namespace-protected symlinks, then it can be a matter of adding one or two
> > lines to the 8-12 tools that perform these sorts of discoveries.  No, this
> > isn't an ideal solution, but it significantly limits the effort required to
> > support a separate namespace.
> > 
> 
> Not only would it require the we carry these patches to system tools and
> libraries indefinitely, it would also mean that we encourage developers
> to produce apps and packages that are not compatible with our Upstream
> (Debian) or any other distro.

We rather ought encourage developers to use build systems that allow
one to specify an install location: then the difference between installing
to /opt/ and system locations becomes entirely a matter of a preference in
the packaging: other format distributions will not even notice.  Distributions
that use Debian-format packages are likely to be downstream from either Debian
or Ubuntu: if we share the optional /opt/ packaging preference with Debian,
then it is trivial for any distribution to adopt the package as either extra
or part of the system with very little effort.

As for patches in system tools, at least for icon cache, path, and desktop
file discovery, this is one or two lines in a configuration file that already
carries Ubuntu-specific patches.  Is there a specific example of a discovery
mechanism that is more complicated to change?

> > For application developers who prefer standard locations, I don't see
> > any reason we oughtn't simply add their packages to the repository with
> > immediate backport: in addition to allowing application developers to put
> > their files in any policy-compliant location, it automatically provides a
> > safe upgrade path for users.  Even for software with release cycles that
> > require significantly more frequent updates than typically provided by our
> > release cycle, there are few barriers to keeping this updated in backports,
> > and users who have installed from backports will remain with backports, so
> > their most active and interested users may be expected to always have the
> > latest version, while highly conservative users will be able to enjoy a
> > consistent ABI during the life of a given release (although these users
> > would need to wait for our next release before using the package).
> > 
> 
> Sending these packages to Backports instead of Extras doesn't change any
> of the potential problems with file and namespace conflicts.

I don't care what the repository is called.  My understanding of the
backports process is that it requires the package *also* be present in the
following release, as a regular system package.  As such, it is automatically
tracked by existing tools in Debian and likely to cause discussion in the
event that someone wants to generate package name or filename conflicts.

Conversely, my understanding of the extras process is that there is no
expectation as to whether the package will be available for the following
release or not, and that it uses a separate repository not currently tracked
by any of the existing tools in Debian.  As a result, there are significant
chances that either 1) a conflict is introduced by some package in Debian,
or 2) an extras maintainer may significantly delay adding a package to extras
(perhaps until after release) as a result of conflict resolution discussions,
which may adversely impact end users.

The above notwithstanding, I'm all in favour of being able to safely add
new or frequently changing packages with policy-compliant filenames to Ubuntu,
both during the life of a release and as part of future releases.  I'm just
concerned that our attempts to work around some of the problems we have had
with such systems in the past may lead to us creating new systems that closely
parallel older systems without addressing the issues that caused the prior
systems to no longer be considered ideal for future pr

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Michael Terry wrote:
> On 04/09/12 10:29, Michael Hall wrote:
> >It's possible for me to upload a package to Universe in Precise's
> >development phase, then for an unrelated package by the same name to be
> >in Debian's archives when we do the Quantal sync.
> >
> >How is this currently handled?
> 
> I believe badly.  But it's not been a huge issue I suspect because
> (a) relatively low volume of packages which get put in Ubuntu first,
> and (b) due diligence by uploaders to universe that there aren't
> likely to be conflicts (i.e. there aren't other upstreams with the
> same name).

As a historical note, (a) was not always true, and as many as one
in twelve packages in Ubuntu were not present in Debian.  In part as
a result of the issues encountered with this, significant effort was
made to add these packages to Debian or remove them from the archive
(frequently replacing them with other packages from Debian).  While
this was a useful way to reduce the MOTU effort required to review
and maintain these packages, it did make (a) true, and contributed
significantly to the perception that adding packages to Ubuntu was
hard or impossible (as those less familiar with our development model
would often balk when told "Looks like a nice package, I'll upload it
to Debian for you, and we'll get it for the next release", especially
before backports was safe to use).

-- 
Emmet HIKORY

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/04/2012 11:11 AM, Emmet Hikory wrote:
> Daniel Holbach wrote:
>> On 04.09.2012 15:28, Emmet Hikory wrote:
>>> The difficulty here is that there is uncontrolled scope for future
>>> conflict.  While the Contents.gz file is useful (and the conflictchecker
>>> system more so), if both extras and backports are enabled by default, there
>>> is no means by which the review board can determine that a given filename
>>> will not be provided by a backport of a new package imported from Debian.
>>
>> The fairest solution to this problem would be to turn on a
>> conflict-check-before-publish for all parts of the archive. This would
>> help us find these (and pre-existing) issues immediately and resolve
>> them amongst the maintainers and upstreams.
> 
> Completely independently of this discussion, this is a good idea,
> although we probably ought have the system respect Breaks, Conflicts,
> and Replaces, or we'll end up with lots of false positives.  Mind you,
> given the vast number of reports from conflictchecker, it may be that
> we would want to put up a review system for some time to clean up the
> existing issues prior to actually blocking publication.
> 
> That said, while I agree that such issues can be sorted between the
> various developers involved (perhaps easily, perhaps less so (see "node")),
> I am much more concerned about the effect on users.  Assume I'm using
> Ubuntu Desktop, and I add an extras application to the Dock.  Now assume
> there is a namespace conflict for that application, which is resolved by
> the extras application taking a new name.  Depending on what I happened 
> to install on upgrade (as a new dependency of another installed package),
> my Dock entry may behave entirely differently.  More generally, this may
> happen for startup programs, so that I would upgrade, reboot, and end up
> running something completely unexpected automatically, and I may not even
> notice if the automatic execution doesn't load a window, and just believe
> my upgrade broke (and potentially end up finding my computer slower, or
> behaving oddly, depending on what the new program happened to do).
> 

We could alternately prevent the importation of packages that conflict
with a package name already in Extras.  That would prevent this from
happening.  We would also have to prevent the importation of packages
that Depends or Recommends on the package we won't be importing.  This
would essentially make package names "first come, first served" between
Extras and Debian.

>> The /opt requirement on the other hand unfortunately imposes a huge
>> amount of work on 1) app developers because our tools don't work this
>> way very easily and 2) Ubuntu maintainers who have to enable path
>> lookups in tools which don't know about /opt yet.
> 
> Much of this can be avoided by allowing a common location for extras
> files, for example /opt/extras/bin, /opt/extras/applications,
> /opt/extras/pixmaps, etc.  If these locations are only permitted to contain
> namespace-protected symlinks, then it can be a matter of adding one or two
> lines to the 8-12 tools that perform these sorts of discoveries.  No, this
> isn't an ideal solution, but it significantly limits the effort required to
> support a separate namespace.
> 

Not only would it require the we carry these patches to system tools and
libraries indefinitely, it would also mean that we encourage developers
to produce apps and packages that are not compatible with our Upstream
(Debian) or any other distro.

> For application developers who prefer standard locations, I don't see
> any reason we oughtn't simply add their packages to the repository with
> immediate backport: in addition to allowing application developers to put
> their files in any policy-compliant location, it automatically provides a
> safe upgrade path for users.  Even for software with release cycles that
> require significantly more frequent updates than typically provided by our
> release cycle, there are few barriers to keeping this updated in backports,
> and users who have installed from backports will remain with backports, so
> their most active and interested users may be expected to always have the
> latest version, while highly conservative users will be able to enjoy a
> consistent ABI during the life of a given release (although these users
> would need to wait for our next release before using the package).
> 

Sending these packages to Backports instead of Extras doesn't change any
of the potential problems with file and namespace conflicts.

Michael Hall
mhall...@ubuntu.com

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Michael Hall wrote:
> > We add an Extras package "foo" in precise.  Then in quantal, we sync in
> > some new, unrelated package "foo" from Debian that happens to share the
> > same name.  Now users that installed "foo" in precise that upgrade to
> > quantal will be replacing their "foo" package with a completely
> > unrelated one.
> > 
> 
> It's possible for me to upload a package to Universe in Precise's
> development phase, then for an unrelated package by the same name to be
> in Debian's archives when we do the Quantal sync.
> 
> How is this currently handled?

The package would be listed as manual-merge, and the developer who
looked at the merge would be expected to notice that these were not the
same package at all, potentially raising the issue on ubuntu-motu@ for
discussion, or potentially resolving the issue with interested parties
in some other way.  I don't remember this happening, although we've had
a few cases where packages added to Ubuntu were later added to Debian with
different names, in which cases we carried a transitional dummy binary
package and associated patch until the next LTS.

There have been a couple cases where we introduced filesystem conflicts
as a result of new-in-Ubuntu packages: these were generally resolved by
lengthy discussions with the relevant Debian maintainers and upstreams, with
the end result that both previously conflicting packages were added to Debian,
and the conflicts resolved there (I believe one of these transitions required
two Debian releases, so such a process may not be fast, but it worked before).

-- 
Emmet HIKORY

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Terry]

On 04/09/12 10:29, Michael Hall wrote:

It's possible for me to upload a package to Universe in Precise's
development phase, then for an unrelated package by the same name to be
in Debian's archives when we do the Quantal sync.

How is this currently handled?


I believe badly.  But it's not been a huge issue I suspect because (a) 
relatively low volume of packages which get put in Ubuntu first, and (b) 
due diligence by uploaders to universe that there aren't likely to be 
conflicts (i.e. there aren't other upstreams with the same name).


(b) is hard to automate and neither scales.  Since we are trying to 
scale better and automate more, this issue becomes more important.


-mt

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Emmet Hikory]

Daniel Holbach wrote:
> On 04.09.2012 15:28, Emmet Hikory wrote:
> > The difficulty here is that there is uncontrolled scope for future
> > conflict.  While the Contents.gz file is useful (and the conflictchecker
> > system more so), if both extras and backports are enabled by default, there
> > is no means by which the review board can determine that a given filename
> > will not be provided by a backport of a new package imported from Debian.
> 
> The fairest solution to this problem would be to turn on a
> conflict-check-before-publish for all parts of the archive. This would
> help us find these (and pre-existing) issues immediately and resolve
> them amongst the maintainers and upstreams.

Completely independently of this discussion, this is a good idea,
although we probably ought have the system respect Breaks, Conflicts,
and Replaces, or we'll end up with lots of false positives.  Mind you,
given the vast number of reports from conflictchecker, it may be that
we would want to put up a review system for some time to clean up the
existing issues prior to actually blocking publication.

That said, while I agree that such issues can be sorted between the
various developers involved (perhaps easily, perhaps less so (see "node")),
I am much more concerned about the effect on users.  Assume I'm using
Ubuntu Desktop, and I add an extras application to the Dock.  Now assume
there is a namespace conflict for that application, which is resolved by
the extras application taking a new name.  Depending on what I happened 
to install on upgrade (as a new dependency of another installed package),
my Dock entry may behave entirely differently.  More generally, this may
happen for startup programs, so that I would upgrade, reboot, and end up
running something completely unexpected automatically, and I may not even
notice if the automatic execution doesn't load a window, and just believe
my upgrade broke (and potentially end up finding my computer slower, or
behaving oddly, depending on what the new program happened to do).

> The /opt requirement on the other hand unfortunately imposes a huge
> amount of work on 1) app developers because our tools don't work this
> way very easily and 2) Ubuntu maintainers who have to enable path
> lookups in tools which don't know about /opt yet.

Much of this can be avoided by allowing a common location for extras
files, for example /opt/extras/bin, /opt/extras/applications,
/opt/extras/pixmaps, etc.  If these locations are only permitted to contain
namespace-protected symlinks, then it can be a matter of adding one or two
lines to the 8-12 tools that perform these sorts of discoveries.  No, this
isn't an ideal solution, but it significantly limits the effort required to
support a separate namespace.

For application developers who prefer standard locations, I don't see
any reason we oughtn't simply add their packages to the repository with
immediate backport: in addition to allowing application developers to put
their files in any policy-compliant location, it automatically provides a
safe upgrade path for users.  Even for software with release cycles that
require significantly more frequent updates than typically provided by our
release cycle, there are few barriers to keeping this updated in backports,
and users who have installed from backports will remain with backports, so
their most active and interested users may be expected to always have the
latest version, while highly conservative users will be able to enjoy a
consistent ABI during the life of a given release (although these users
would need to wait for our next release before using the package).

-- 
Emmet HIKORY


-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/04/2012 10:45 AM, Scott Howard wrote:
> On Tue, Sep 4, 2012 at 10:21 AM, Michael Hall  wrote:
>>
>> On 09/04/2012 09:39 AM, Scott Kitterman wrote:
>>> The problem isn't just with file conflicts with current packages, it's that
>>> these packages will now start using up distro namespace.  If some app
>>> developer package ships the file /usr/games/bird-game, even though there's 
>>> no
>>> current conflict, there is a package sync'ed from Debian that also ships
>>> /usr/games/bird-game then there's a conflict we have to resolve.  In /opt 
>>> in a
>>> proper vendor namespace this can never happen.
>>
>> If bird-game already exists in Extras, and then a different package is
>> allowed into backports that will install files into the same location,
>> then yes there is a possibility for a conflict.  But I assume part of
>> the backports approval process already checks for conflicts, as they may
>> exist with another package in the stable release already, so that
>> process could easily be extended to include Extras packages as well.
> 
> I think people are more concerned with auto-import from Debian than
> backports. Debian's approval process knows nothing about Extras, so
> there is no mechanism or approval process that checks for conflicts.
> 
Since Extras package won't be in the development release until
FeatureFreeze, we can check them against the new packages imported from
Debian before we forward-copy them from the previous stable release.


Michael Hall
mhall...@ubuntu.com

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

> What about the following scenario?
> 
> We add an Extras package "foo" in precise.  Then in quantal, we sync in
> some new, unrelated package "foo" from Debian that happens to share the
> same name.  Now users that installed "foo" in precise that upgrade to
> quantal will be replacing their "foo" package with a completely
> unrelated one.
> 

It's possible for me to upload a package to Universe in Precise's
development phase, then for an unrelated package by the same name to be
in Debian's archives when we do the Quantal sync.

How is this currently handled?


Michael Hall
mhall...@ubuntu.com

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Jonathan Carter]

Hi!

On 04/09/2012 10:21, Michael Hall wrote:
> On 09/04/2012 09:39 AM, Scott Kitterman wrote:
>> The problem isn't just with file conflicts with current packages, it's that 
>> these packages will now start using up distro namespace.  If some app 
>> developer package ships the file /usr/games/bird-game, even though there's 
>> no 
>> current conflict, there is a package sync'ed from Debian that also ships 
>> /usr/games/bird-game then there's a conflict we have to resolve.  In /opt in 
>> a 
>> proper vendor namespace this can never happen.
> 
> If bird-game already exists in Extras, and then a different package is
> allowed into backports that will install files into the same location,
> then yes there is a possibility for a conflict.  But I assume part of
> the backports approval process already checks for conflicts, as they may
> exist with another package in the stable release already, so that
> process could easily be extended to include Extras packages as well.

In extras, all the package files (with a few exceptions like the debian
changelog, copyright files, etc) gets installed under /opt. There are
some exceptions for things like unity lenses and .desktop entries, but
those are prefixed with "extras-" so that they are somewhat namespaced.

Packages in backports aren't allowed to install under /opt, so the
chance of a collision is very rare. It's possible for someone to have
collision if they happen to have the same unity lens or desktop entry
that happens to start with "extras-", but at least that's something
fairly unlikely and easy to check, maybe something that should be
considered suggesting to Debian policy and added to lintian.

Having two packages with colliding file names doesn't seem like it will
be a big problem. From what I can tell it seems like it's more likely
that a package would be introduced in Debian or Ubuntu at some point
that is also in extras. So far, many of the apps in extras have
*extremely* generic names. While reviewing apps for the app showdown
competition, I checked whether these packages existed in Debian/Ubuntu
first before +1'ing them and was surprised to see that none of these
names have been used before. IMHO it would be reasonable to prefix
something to extras packages to avoid a conflict, perhaps also have some
policy updates that prohibits a package name starting with "extras-" (or
whatever the prefix would be) in the archives.

-Jonathan

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Scott Howard]

On Tue, Sep 4, 2012 at 10:21 AM, Michael Hall  wrote:
>
> On 09/04/2012 09:39 AM, Scott Kitterman wrote:
>> The problem isn't just with file conflicts with current packages, it's that
>> these packages will now start using up distro namespace.  If some app
>> developer package ships the file /usr/games/bird-game, even though there's no
>> current conflict, there is a package sync'ed from Debian that also ships
>> /usr/games/bird-game then there's a conflict we have to resolve.  In /opt in 
>> a
>> proper vendor namespace this can never happen.
>
> If bird-game already exists in Extras, and then a different package is
> allowed into backports that will install files into the same location,
> then yes there is a possibility for a conflict.  But I assume part of
> the backports approval process already checks for conflicts, as they may
> exist with another package in the stable release already, so that
> process could easily be extended to include Extras packages as well.

I think people are more concerned with auto-import from Debian than
backports. Debian's approval process knows nothing about Extras, so
there is no mechanism or approval process that checks for conflicts.

>> A fairly contrived example derived from the above is if the Debian/Ubuntu
>> package that shipped /usr/games/bird-game was in the archive and an app
>> developer package was uploaded that shipped /usr/bin/bird-game, it could be
>> run instead since /usr/bin precedes /usr/games on the standard user path.
>
> This would only happen if two different apps were both called
> "bird-game" but otherwise different.  This situation is also possible
> already, even between two packages in Universe.  How is that currently
> handled?

New Universe packages are pretty much handled by Debian, and while it
is generally avoided there is occasionally a collision (see the
node.js example above in this thread). The problem is that the
proposal is introducing a mechanism for adding packages and files
which our current mechanism (Debian NEW queue) knows nothing about,
greatly increasing the possibility of collision.

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Michael Hall]

On 09/04/2012 09:39 AM, Scott Kitterman wrote:
> The problem isn't just with file conflicts with current packages, it's that 
> these packages will now start using up distro namespace.  If some app 
> developer package ships the file /usr/games/bird-game, even though there's no 
> current conflict, there is a package sync'ed from Debian that also ships 
> /usr/games/bird-game then there's a conflict we have to resolve.  In /opt in 
> a 
> proper vendor namespace this can never happen.

If bird-game already exists in Extras, and then a different package is
allowed into backports that will install files into the same location,
then yes there is a possibility for a conflict.  But I assume part of
the backports approval process already checks for conflicts, as they may
exist with another package in the stable release already, so that
process could easily be extended to include Extras packages as well.


> A fairly contrived example derived from the above is if the Debian/Ubuntu 
> package that shipped /usr/games/bird-game was in the archive and an app 
> developer package was uploaded that shipped /usr/bin/bird-game, it could be 
> run instead since /usr/bin precedes /usr/games on the standard user path. 

This would only happen if two different apps were both called
"bird-game" but otherwise different.  This situation is also possible
already, even between two packages in Universe.  How is that currently
handled?

> maybe a Python based app ships an embedded copy of a module since they've 
> tested with that version and don't want to test with the distro version and 
> they write their setup.py to install it in /usr/local/lib/python2.7/dist-
> packages.  Suddenly every user of that module is now using the app developer 
> package version and not the distro version.

Installing python modules in the system directories is not allowed under
this process.  Check the whitelist defined in the "App Preparation"
section of the spec.


Michael Hall
mhall...@ubuntu.com

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Re: Proposing a New App Developer Upload Process

[Matt Wheeler]

On Sep 4, 2012 3:00 PM, "Michael Terry"  wrote:
[snip]
> (Though with namespaced packages, the user can end up in a weird
situation if the Extras package "foo" does find its way into Debian and
Ubuntu quantal.  Then, they really do want the "foo" package from Debian
but are left with the non-maintained "extras-foo" from precise.)

That can be handled the usual way package name transitions happen (e.g.
git-core), perhaps it could even be totally automated?

--
Matt Wheeler
m...@funkyhat.org
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel