Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

Hi Paul,

On Tue, May 28, 2013 at 09:52:00AM +0800, Paul Wise wrote:
> Alternative proposal that I came up with while discussing this on IRC:
> 
> We could add a second page with a URL like [1] that would list all the
> blends/tasks for the binary packages of the source package. As a bonus
> this could contain more info than the normal PTS page. The normal PTS
> pages would then simply link to that page.
> 
> 1. http://packages.qa.debian.org/g/gimp/tasks.html

To me this sounds better than the list of numbers.

Kind regards

 Andreas.

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Paul Wise]

On Tue, May 28, 2013 at 4:49 AM, Andreas Tille wrote:

> I'm not fully convince about the numbered tasks.  Failing to be able to
> make a better suggestion I'd be fine if this would be implemented but
> perhaps somebody else might have a better idea.
...
> No problem for the moment.  I'm not sure if we should mix up generic
> tasksel tasks and Blends tasks in case we get at some point in time
> the tasksel data imported.  But that can be discussed later.
...
> Hmmm, I do not think so because finally the user would be forced to seek
> to sometimes a lot of tasks if he really want to find the dependency.
> That's also not friendly.

Alternative proposal that I came up with while discussing this on IRC:

We could add a second page with a URL like [1] that would list all the
blends/tasks for the binary packages of the source package. As a bonus
this could contain more info than the normal PTS page. The normal PTS
pages would then simply link to that page.

1. http://packages.qa.debian.org/g/gimp/tasks.html

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

Hi Paul,

On Mon, May 27, 2013 at 11:11:52PM +0800, Paul Wise wrote:
> > Sounds quite interesting.
> 
> http://wiki.debian.org/SummerOfCode2013/StudentApplications/SimonChopin
> http://lists.debian.org/20130425115056.21764.39271@mithrandir

Cool.

> > I have introduced a 'title' field.  Please check again
> >
> >   
> > http://anonscm.debian.org/gitweb/?p=blends/website.git;a=blob;f=misc/pts/taskinfo-example.py
> >
> > It now creates two different JSON files that should contain all needed
> > information.  Please tell me if something might be missing.
> 
> Looks good, here are some examples, I would be interested in your thoughts.
> 
> http://packages.qa.debian.org/~pabs/g/gromacs.html

Looks very good!

> http://packages.qa.debian.org/~pabs/g/gimp.html
> http://packages.qa.debian.org/~pabs/g/ghemical.html

I'm not fully convince about the numbered tasks.  Failing to be able to
make a better suggestion I'd be fine if this would be implemented but
perhaps somebody else might have a better idea.

> Let me know if you want some other packages rendered or any changes made.

The principle seems to be clear ...

> I changed the wording to tasks instead of blends since it is more 
> task-focused.

No problem for the moment.  I'm not sure if we should mix up generic
tasksel tasks and Blends tasks in case we get at some point in time 
the tasksel data imported.  But that can be discussed later.

> When there are more than 5 tasks, I currently display numbers instead.
> Maybe where there are more than 5 tasks I should list just the blends
> instead of individual tasks?

Hmmm, I do not think so because finally the user would be forced to seek
to sometimes a lot of tasks if he really want to find the dependency.
That's also not friendly.

KInd regards

 Andreas.

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Paul Wise]

On Mon, May 27, 2013 at 8:08 PM, Andreas Tille wrote:

> Sounds quite interesting.

http://wiki.debian.org/SummerOfCode2013/StudentApplications/SimonChopin
http://lists.debian.org/20130425115056.21764.39271@mithrandir

> I have introduced a 'title' field.  Please check again
>
>   
> http://anonscm.debian.org/gitweb/?p=blends/website.git;a=blob;f=misc/pts/taskinfo-example.py
>
> It now creates two different JSON files that should contain all needed
> information.  Please tell me if something might be missing.

Looks good, here are some examples, I would be interested in your thoughts.

http://packages.qa.debian.org/~pabs/g/gromacs.html
http://packages.qa.debian.org/~pabs/g/gimp.html
http://packages.qa.debian.org/~pabs/g/ghemical.html

Let me know if you want some other packages rendered or any changes made.

I changed the wording to tasks instead of blends since it is more task-focused.

When there are more than 5 tasks, I currently display numbers instead.
Maybe where there are more than 5 tasks I should list just the blends
instead of individual tasks?

> After a second thought things are not that simple as I've thought in the
> first place.  While it is easy to detect what packages do contain
> tasksel input the real dependencies of the tasks are hidden *inside* the
> package and not available in UDD.  In other words:  To create a link
> from a package to the task it might belong to we need to create a new
> UDD importer that parses those tasksel control files.

Hmm, ok.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

Hi Paul,

On Mon, May 27, 2013 at 01:06:14PM +0800, Paul Wise wrote:
> > IN fact I'm considering a not yet used update mechanism in UDD - perhaps
> > some commit hook or so which sounds more apropriate in the Blends
> > metapackage case.  A daily import (as usual in UDD) does not make much
> > sense because over several days the content remains static.  But at some
> > points in time there are several changes on one day.  I need to think
> > about this.
> 
> Long term I think we want to move Debian infrastructure towards some
> sort of realtime messaging system like fedmsg, AMPQ etc. So dak would
> emit events, UDD and other QA stuff would listen to those events and
> update themselves in realtime.

Sounds quite interesting.

> > This does not sound fully convinced.  Do you see any drawback in this
> > historical fact?
> 
> Just that the strings on the PTS are very long when they include the
> blend name, I worked around this by just including task names. With
> the tasks from tasksel having long names too, I might need to do some
> sort of shortening method.

I have introduced a 'title' field.  Please check again

  
http://anonscm.debian.org/gitweb/?p=blends/website.git;a=blob;f=misc/pts/taskinfo-example.py

It now creates two different JSON files that should contain all needed
information.  Please tell me if something might be missing.

> > In fact most Blends do create a tasksel package as well (debichem-tasks,
> > med-tasks, etc.)  I had the idea that not only packages matching 'task%'
> > in their name should match the query but rather those that depend from
> > tasksel package.  I'll come up with some suggestion for a JSON data file
> > soon.
> 
> Great, thanks.

After a second thought things are not that simple as I've thought in the
first place.  While it is easy to detect what packages do contain
tasksel input the real dependencies of the tasks are hidden *inside* the
package and not available in UDD.  In other words:  To create a link
from a package to the task it might belong to we need to create a new
UDD importer that parses those tasksel control files.

Kind regards

   Andreas.

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-blends-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130527120820.gb16...@an3as.eu


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Paul Wise]

On Mon, May 27, 2013 at 12:04 AM, Andreas Tille wrote:

> I bounced the bug some minutes later - I just droped it by accident in
> the first place.

Ah. My lists mail setup (gmail) didn't notice the second mail.

> IN fact I'm considering a not yet used update mechanism in UDD - perhaps
> some commit hook or so which sounds more apropriate in the Blends
> metapackage case.  A daily import (as usual in UDD) does not make much
> sense because over several days the content remains static.  But at some
> points in time there are several changes on one day.  I need to think
> about this.

Long term I think we want to move Debian infrastructure towards some
sort of realtime messaging system like fedmsg, AMPQ etc. So dak would
emit events, UDD and other QA stuff would listen to those events and
update themselves in realtime.

> This does not sound fully convinced.  Do you see any drawback in this
> historical fact?

Just that the strings on the PTS are very long when they include the
blend name, I worked around this by just including task names. With
the tasks from tasksel having long names too, I might need to do some
sort of shortening method.

> In fact most Blends do create a tasksel package as well (debichem-tasks,
> med-tasks, etc.)  I had the idea that not only packages matching 'task%'
> in their name should match the query but rather those that depend from
> tasksel package.  I'll come up with some suggestion for a JSON data file
> soon.

Great, thanks.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

Hi Paul,

On Sun, May 26, 2013 at 03:50:51PM +0800, Paul Wise wrote:
> [Keep the bug in CC]

I bounced the bug some minutes later - I just droped it by accident in
the first place.

> > I need to find out how to do / where to place this because I did not yet
> > dealt with CGIs on udd.d.o but once your specification (see below is
> > fullfilled I can do this if it helps).  The only thing I'm wondering
> > whether a CGI script should always run the query itself.  It would be
> > way more efficient to drop the result somewhere and just deliver the
> > content via CGI because the usual change of the data is either once per
> > day or once per change in the blends tasks files (I did not yet decided
> > how I might trigger the importer).
> 
> Your idea of caching the query does sound more efficient.

IN fact I'm considering a not yet used update mechanism in UDD - perhaps
some commit hook or so which sounds more apropriate in the Blends
metapackage case.  A daily import (as usual in UDD) does not make much
sense because over several days the content remains static.  But at some
points in time there are several changes on one day.  I need to think
about this.

> > [{"blend": "debichem", "tasksurl": 
> > "http://blends.alioth.debian.org/debichem/tasks/"},
> >  {"blend": "debian-med", "tasksurl": 
> > "http://debian-med.alioth.debian.org/tasks/"},
> >  ...
> > ]
> ...
> > I would think about to what export I might add this information in case
> > you agree to reduce the redundance a bit by providing two different data
> > files.
> 
> Yeah, that does make more sense.

OK.

> > The prefix of the metapackages can be differnet from the Blends names.
> > In fact if I remember right DebiChem is the only one were the prefix
> > fits:
> 
> Hmm, ok.

This does not sound fully convinced.  Do you see any drawback in this
historical fact?

> > You mean tasks that are in tasksel without using the Blends framework?
> 
> Correct.
> 
> > Hmmm, I'm not aware that this data is in UDD in some structured manner.
> > However, we could try with
> ...
> > If this is what you want to use (probably without the "Depends: tasksel") I 
> > could
> > probably create a similar JSON file.  As title we could use
> ..
> > Just tell me if this would be helpful and I could do this as well.
> 
> That would be helpful indeed. I think some part of the code should
> drop the task- from the package name. Probably I would merge blends
> tasks and tasksel tasks on the PTS pages.

In fact most Blends do create a tasksel package as well (debichem-tasks,
med-tasks, etc.)  I had the idea that not only packages matching 'task%'
in their name should match the query but rather those that depend from
tasksel package.  I'll come up with some suggestion for a JSON data file
soon.

Thanks for your nice ideas

  Andreas.

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

Hi Paul,

On Sun, May 26, 2013 at 02:06:06PM +0800, Paul Wise wrote:
> On Sun, May 26, 2013 at 3:34 AM, Andreas Tille wrote:
> 
> > I have prepared a Python script that creates JSON output with the
> > relevant data from UDD.  Please check out
> 
> Thanks. Could you move that to a CGI running on udd.d.o though?

I need to find out how to do / where to place this because I did not yet
dealt with CGIs on udd.d.o but once your specification (see below is
fullfilled I can do this if it helps).  The only thing I'm wondering
whether a CGI script should always run the query itself.  It would be
way more efficient to drop the result somewhere and just deliver the
content via CGI because the usual change of the data is either once per
day or once per change in the blends tasks files (I did not yet decided
how I might trigger the importer).

> > and run it from alioth.  Please let me know if you need any other
> > information.  Remark:  The Blends data are not yet updated via cron job
> > in UDD.  This needs to be organised once the data has proven to be
> > useful.
> 
> I've implemented a preliminary version based on what you provided and
> generated a page for gromacs:
> 
> http://packages.qa.debian.org/~pabs/g/gromacs.html
> 
> However, the data lacks some essential elements.
> 
> I would like to be able to generate this:
> 
> http://blends.alioth.debian.org/debichem/tasks/molmech#gromacs";
> title="DebiChem Molecular mechanics packages">molmech,
> http://debian-med.alioth.debian.org/tasks/bio#gromacs";
> title="Debian Med Biology packages">bio

I can add this easily but I think it would be very redundant to add the
full URL string for every single package in the JSON data set.  Would
you think it makes sense to deliver an additional JSON data file

[{"blend": "debichem", "tasksurl": 
"http://blends.alioth.debian.org/debichem/tasks/"},
 {"blend": "debian-med", "tasksurl": 
"http://debian-med.alioth.debian.org/tasks/"},
 ...
]

> http://blends.alioth.debian.org/";>blends
> 
> The JSON only has this:
> 
> {"component": "main", "dependency": "d", "task": "molmech", "blend":
> "debichem", "package": "gromacs"}
> {"component": "main", "dependency": "d", "task": "bio", "blend":
> "debian-med", "package": "gromacs"}
> {"component": "main", "dependency": "d", "task": "chemistry", "blend":
> "debian-science", "package": "gromacs"}
> 
> It needs at least:
> 
> The URL of the task.

See above.

> The title of the task.

I would think about to what export I might add this information in case
you agree to reduce the redundance a bit by providing two different data
files.

> Also, I'm surprised to see the blend name is debian-med when the task
> meta-packages in Debian are med-*

The prefix of the metapackages can be differnet from the Blends names.
In fact if I remember right DebiChem is the only one were the prefix
fits:

   Debian Med: med-*
   Debian Science: science-*
   Debian Edu: education-*
   ...

> It would be interesting to also import tasks from tasksel too if that
> is possible.

You mean tasks that are in tasksel without using the Blends framework?
Hmmm, I'm not aware that this data is in UDD in some structured manner.
However, we could try with

SELECT depends, recommends, suggests from packages where package = 
'task-gnome-desktop' ;
 depends  | 
 recommends 
 | 
suggests 
--+--+--
 tasksel, task-desktop, gnome-core, network-manager-gnome | gnome, 
libreoffice-gnome, libreoffice-evolution, gimp, synaptic, iceweasel, 
libreoffice, libreoffice-gcj, libreoffice-help-en-us, mythes-en-us, 
hunspell-en-us, hyphen-en-us | 

If this is what you want to use (probably without the "Depends: tasksel") I 
could
probably create a similar JSON file.  As title we could use

udd=# SELECT description from packages where package = 'task-gnome-desktop' ;
description
---
 GNOME desktop environment

Just tell me if this would be helpful and I could do this as well.

Kind regards

Andreas.

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Paul Wise]

[Keep the bug in CC]

On Sun, May 26, 2013 at 3:37 PM, Andreas Tille wrote:

> I need to find out how to do / where to place this because I did not yet
> dealt with CGIs on udd.d.o but once your specification (see below is
> fullfilled I can do this if it helps).  The only thing I'm wondering
> whether a CGI script should always run the query itself.  It would be
> way more efficient to drop the result somewhere and just deliver the
> content via CGI because the usual change of the data is either once per
> day or once per change in the blends tasks files (I did not yet decided
> how I might trigger the importer).

Your idea of caching the query does sound more efficient.

> I can add this easily but I think it would be very redundant to add the
> full URL string for every single package in the JSON data set.  Would
> you think it makes sense to deliver an additional JSON data file
>
> [{"blend": "debichem", "tasksurl": 
> "http://blends.alioth.debian.org/debichem/tasks/"},
>  {"blend": "debian-med", "tasksurl": 
> "http://debian-med.alioth.debian.org/tasks/"},
>  ...
> ]
...
> I would think about to what export I might add this information in case
> you agree to reduce the redundance a bit by providing two different data
> files.

Yeah, that does make more sense.

> The prefix of the metapackages can be differnet from the Blends names.
> In fact if I remember right DebiChem is the only one were the prefix
> fits:

Hmm, ok.

> You mean tasks that are in tasksel without using the Blends framework?

Correct.

> Hmmm, I'm not aware that this data is in UDD in some structured manner.
> However, we could try with
...
> If this is what you want to use (probably without the "Depends: tasksel") I 
> could
> probably create a similar JSON file.  As title we could use
..
> Just tell me if this would be helpful and I could do this as well.

That would be helpful indeed. I think some part of the code should
drop the task- from the package name. Probably I would merge blends
tasks and tasksel tasks on the PTS pages.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Paul Wise]

On Sun, May 26, 2013 at 3:34 AM, Andreas Tille wrote:

> I have prepared a Python script that creates JSON output with the
> relevant data from UDD.  Please check out

Thanks. Could you move that to a CGI running on udd.d.o though?

> and run it from alioth.  Please let me know if you need any other
> information.  Remark:  The Blends data are not yet updated via cron job
> in UDD.  This needs to be organised once the data has proven to be
> useful.

I've implemented a preliminary version based on what you provided and
generated a page for gromacs:

http://packages.qa.debian.org/~pabs/g/gromacs.html

However, the data lacks some essential elements.

I would like to be able to generate this:

http://blends.alioth.debian.org/debichem/tasks/molmech#gromacs";
title="DebiChem Molecular mechanics packages">molmech,
http://debian-med.alioth.debian.org/tasks/bio#gromacs";
title="Debian Med Biology packages">bio
http://blends.alioth.debian.org/";>blends

The JSON only has this:

{"component": "main", "dependency": "d", "task": "molmech", "blend":
"debichem", "package": "gromacs"}
{"component": "main", "dependency": "d", "task": "bio", "blend":
"debian-med", "package": "gromacs"}
{"component": "main", "dependency": "d", "task": "chemistry", "blend":
"debian-science", "package": "gromacs"}

It needs at least:

The URL of the task.
The title of the task.

Also, I'm surprised to see the blend name is debian-med when the task
meta-packages in Debian are med-*

It would be interesting to also import tasks from tasksel too if that
is possible.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

On Mon, Apr 01, 2013 at 09:35:23PM +0800, Paul Wise wrote:
> On Wed, Mar 27, 2013 at 9:43 PM, Andreas Tille wrote:
> 
> > With the package information there is the following problem regarding
> > the usage in PTS:  The different versions of the metapackages (in
> > oldstable, stable, testing) might contain different dependencies and
> > finally packages that are in unstable only are also not part of the
> > metapackages but topic of the Blends task.  Because I assume that PTS is
> > developer centric I would assume that you consider any dependency
> > mentioned in the tasks file and available in Debian should be part
> > of the output JSON database, right?
> 
> The PTS usually only deals with unstable, but for this case I think it
> would be reasonable to do that, right.

I have prepared a Python script that creates JSON output with the
relevant data from UDD.  Please check out

   
http://anonscm.debian.org/gitweb/?p=blends/website.git;a=blob;f=misc/pts/taskinfo-example.py

and run it from alioth.  Please let me know if you need any other
information.  Remark:  The Blends data are not yet updated via cron job
in UDD.  This needs to be organised once the data has proven to be
useful.

I'd regard my part to solve this bug now done in case you are happy
with this output.

Kind regards

  Andreas.

PS: Including the Blends metadata into UDD was a very good idea and
I will come up with some other interesting ideas as consequences
from this.


-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Paul Wise]

On Wed, Mar 27, 2013 at 9:43 PM, Andreas Tille wrote:

> With the package information there is the following problem regarding
> the usage in PTS:  The different versions of the metapackages (in
> oldstable, stable, testing) might contain different dependencies and
> finally packages that are in unstable only are also not part of the
> metapackages but topic of the Blends task.  Because I assume that PTS is
> developer centric I would assume that you consider any dependency
> mentioned in the tasks file and available in Debian should be part
> of the output JSON database, right?

The PTS usually only deals with unstable, but for this case I think it
would be reasonable to do that, right.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

Hi,

On Wed, Mar 20, 2013 at 05:05:33PM +0100, Andreas Tille wrote:
> On Wed, Mar 20, 2013 at 04:28:14PM +0800, Paul Wise wrote:
> 
> > Blend: Debian Med
> > Task: Biology
> > Task-name: med-bio
> > URL: http://debian-med.alioth.debian.org/tasks/bio
> > Packages: abacas acedb-other 
> > 
> > Blend: Debian Med
> > Task: Biology development
> > Task-name: med-bio-dev
> > URL: http://debian-med.alioth.debian.org/tasks/bio-dev
> > Packages: bioperl bioperl-run
> 
>   CREATE TABLE blends_metadata (
>   -- fieldname   type,   --  example value
>  blend   TEXT,   --  'debian-med'   (== the source package name)
>  blendname   TEXT,   --  'Debian Med'   (== human readable name)
>  projecturl  TEXT,   --  'http://debian-med.alioth.debian.org/'
>  tasksprefix TEXT,   --  'med'
>  PRIMARY KEY (blend)
>   );

Commited to UDD Git.

>   CREATE TABLE blends_tasks (
>   -- fieldname   type,   --  example value
>  blend   TEXT REFERENCES blends_metadata,
>  taskTEXT,   --  'bio'
>  PRIMARY KEY (blend, task)
>   );

Commited to UDD Git with additional flag

   metapackage BOOLEAN

to reflect the fact whether a metapackage should be created or not
(which is part of the data in a tasks file).


>   CREATE TABLE blends_packages (
>   -- fieldname  type,   --  example_value
>  blend  TEXT REFERENCES blends_metadata,
>  task   TEXT REFERENCES blends_tasks,
>  packageTEXT,   --  'gromacs'
>  PRIMARY KEY (blend, task, package)
>   )

Needs to be implemented.  I will add two additional columns

 dependency   CHARACTER(1) CHECK (dependency IN ('d', 's')), -- Depends / 
Suggests
 distribution TEXT CHECK (distribution IN ('debian', 'prospective', 
'ubuntu', 'other')),

With the package information there is the following problem regarding
the usage in PTS:  The different versions of the metapackages (in
oldstable, stable, testing) might contain different dependencies and
finally packages that are in unstable only are also not part of the
metapackages but topic of the Blends task.  Because I assume that PTS is
developer centric I would assume that you consider any dependency
mentioned in the tasks file and available in Debian should be part
of the output JSON database, right?

Kind regards

Andreas.

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

On Fri, Mar 22, 2013 at 01:31:58PM +0100, Michael Banck wrote:
> On Wed, Mar 20, 2013 at 05:05:33PM +0100, Andreas Tille wrote:
> > Well, machine readable file is one thing.  At first we need to define
> > the SQL table layout.  Currently the philosophy in UDD is to have the
> > tables not normalised but I think it makes sense to normalise to some
> > extend into three tables because it simplifies things we intend to do:
> > 
> >   CREATE TABLE blends_packages (
> >   -- fieldname  type,   --  example_value
> >  blend  TEXT REFERENCES blends_metadata,
> >  task   TEXT REFERENCES blends_tasks,
> >  packageTEXT,   --  'gromacs'
> 
> Shouldn't/couldn't that be a foreign key to a general binary-packages
> table, assuming that exists?

I'm hesitating for two reasons:

  1. UDD is intentionally very sparse with foreign keys (I used these
 here basically for clarifying issues and because these tables
 are not interconnected with other tables (except for the package
 name as you mentioned)

  2. We have the table blends_prospectivepackages and I could potentially
 imagine rather an additional flag field like this

 distribution TEXT,

 featuring values like 'debian', 'ubuntu', 'prospective' or something
 like this - this would break the foreign key constraint.  Even if
 I'm not sure whether this will be really implemented I do not see
 any reason for doing some over-design in the database table which
 might prevent some reasonable enhancement.

Thanks for the hint anyway

Andreas. 

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Michael Banck]

On Wed, Mar 20, 2013 at 05:05:33PM +0100, Andreas Tille wrote:
> Well, machine readable file is one thing.  At first we need to define
> the SQL table layout.  Currently the philosophy in UDD is to have the
> tables not normalised but I think it makes sense to normalise to some
> extend into three tables because it simplifies things we intend to do:
> 
>   CREATE TABLE blends_packages (
>   -- fieldname  type,   --  example_value
>  blend  TEXT REFERENCES blends_metadata,
>  task   TEXT REFERENCES blends_tasks,
>  packageTEXT,   --  'gromacs'

Shouldn't/couldn't that be a foreign key to a general binary-packages
table, assuming that exists?

>  PRIMARY KEY (blend, task, package)
>   )


Michael


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Paul Wise]

On Thu, Mar 21, 2013 at 12:05 AM, Andreas Tille wrote:

> Well, machine readable file is one thing.  At first we need to define
> the SQL table layout.  Currently the philosophy in UDD is to have the
> tables not normalised but I think it makes sense to normalise to some
> extend into three tables because it simplifies things we intend to do:
>
>   CREATE TABLE blends_metadata (
...
>   CREATE TABLE blends_tasks (
...
>   CREATE TABLE blends_packages (
...

I'm not an expert in SQL, but those look good to me.

> From the table layout above it is a simple task to create YAML,JSON
> which I consider rather *your* choice - I even wonder if there is any
> need for an intermediate format.  Isn't PTS not created from UDD
> queries?

The PTS isn't created from UDD queries, but I think/guess/hope the
planned rewrite would be:

http://wiki.debian.org/SummerOfCode2013/Projects#PTS_rewrite_in_Django

So for now we need an intermediate data format. JSON or YAML is fine,
both of those are currently parsed by the PTS, munged, turned into XML
and then rendered using XSLT. I'd recommend a list of tasks with their
names and lists of packages.

> prevent).  So this approach is a bit hackish because there is a reason
> why tasks are binary package centric (== user oriented) and PTS is
> source centric (== developer oriented).  I guess this mix of target
> groups leads to the technical problem we are facing.

Agreed.

> IMHO this would be the better solution because it resolves the conflict
> above.

Ok, lets do that.

> OK, that's fine for me.  I will not question PTS development decisions.

Please do question PTS decisions when appropriate, the PTS should
serve the needs of the Debian packaging community and if it isn't
doing that, we should fix it.

> In any case I'd applause the intention to make Blends more visible.

There are more ways to do that, I can think of at least two, will
bring those up in a new thread.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

On Wed, Mar 20, 2013 at 04:28:14PM +0800, Paul Wise wrote:
> [Please keep the bug in CC]

Yep.  I noticed my mistake in last mail and bounced it right after
sending to the bug
 
> On Wed, Mar 20, 2013 at 12:55 AM, Andreas Tille wrote:
> 
> > If it comes to Blends topic I am an UDD folk. ;-)
> > You should simply specify what data you need and I will go for it.
> 
> So, a machine readable file (YAML/JSON/deb822/etc, your choice),
> something like this:
> 
> Blend: Debian Med
> Task: Biology
> Task-name: med-bio
> URL: http://debian-med.alioth.debian.org/tasks/bio
> Packages: abacas acedb-other 
> 
> Blend: Debian Med
> Task: Biology development
> Task-name: med-bio-dev
> URL: http://debian-med.alioth.debian.org/tasks/bio-dev
> Packages: bioperl bioperl-run

Well, machine readable file is one thing.  At first we need to define
the SQL table layout.  Currently the philosophy in UDD is to have the
tables not normalised but I think it makes sense to normalise to some
extend into three tables because it simplifies things we intend to do:

  CREATE TABLE blends_metadata (
  -- fieldname   type,   --  example value
 blend   TEXT,   --  'debian-med'   (== the source package name)
 blendname   TEXT,   --  'Debian Med'   (== human readable name)
 projecturl  TEXT,   --  'http://debian-med.alioth.debian.org/'
 tasksprefix TEXT,   --  'med'
 PRIMARY KEY (blend)
  );

  CREATE TABLE blends_tasks (
  -- fieldname   type,   --  example value
 blend   TEXT REFERENCES blends_metadata,
 taskTEXT,   --  'bio'
 PRIMARY KEY (blend, task)
  );

  CREATE TABLE blends_packages (
  -- fieldname  type,   --  example_value
 blend  TEXT REFERENCES blends_metadata,
 task   TEXT REFERENCES blends_tasks,
 packageTEXT,   --  'gromacs'
 PRIMARY KEY (blend, task, package)
  )

These tables could be filled from configuration files in

   git://git.debian.org/git/blends/website.git

in dir webtools/webconf as well as from tasks files in SVN.  When
thinking twice it might be worth considering to drop the webconf stuff
in favour of putting the information in question right into the source
package metainformation.  But this could be decided / adapted later.

>From the table layout above it is a simple task to create YAML,JSON
which I consider rather *your* choice - I even wonder if there is any
need for an intermediate format.  Isn't PTS not created from UDD
queries?

> > I'm sorry I can not really parse what you want to tell me with this.
> 
> I'll try again. The PTS is primarily source package based but the
> blends infrastructure is binary package based. The PTS will want to
> link to the blends infrastructure, using per-package anchors. The
> anchors provided by the blends infrastructure are based on binary
> package names rather than source package names. Here is an example
> based on the logol source package:
> 
> http://debian-med.alioth.debian.org/tasks/bio#logol-bin";
> title="Debian Med Biology packages">med-bio
> http://blends.alioth.debian.org/";>blend
> 
> It would be much better if the PTS could link directly to a source
> package anchor instead, especially since blends can sometimes depend
> on multiple binary packages from one source package. Unfortunately it
> isn't yet possible to do this:
> 
> http://debian-med.alioth.debian.org/tasks/bio#logol";
> title="Debian Med Biology packages">med-bio
> http://blends.alioth.debian.org/";>blend

Ahhh, OK.  Adding an additional anchor mentioning the source should be
no problem in principle (at least not when the planed changes will be
implemented - I'll issue a GSoC proposal about this).  However, there is
always a number of binary packages from one single source package inside
a single task - this would result in duplicated source package anchors
which IMHO is not a really good idea (I guess browsers are defaulting to
the first anchor - but finally something is happening you really want to
prevent).  So this approach is a bit hackish because there is a reason
why tasks are binary package centric (== user oriented) and PTS is
source centric (== developer oriented).  I guess this mix of target
groups leads to the technical problem we are facing.

> Alternatively we can drop the anchor, since I guess folks clicking on
> blends links are looking for other related packages rather than the
> package itself.

IMHO this would be the better solution because it resolves the conflict
above.

> Thats a separate issue to what you pointed out with gromacs, the
> gromacs issue can be solved like this:
> 
> http://blends.alioth.debian.org/debichem/tasks/molmech#gromacs";
> title="DebiChem Molecular mechanics packages">debichem-molmech,
> http://debian-med.alioth.debian.org/tasks/bio#gromacs";
> title="Debian Med Biology packages">med-bio
> http://blends.alioth.debian.org/";>blends
> 
> And if the length of the blends list gets too long we can do it like this:
> 
> http://blends.alioth.debian.org/";>blends
> (http://blends.al

Bug#703402: PTS: link to the blends website for packages involved in blends

[Paul Wise]

[Please keep the bug in CC]

On Wed, Mar 20, 2013 at 12:55 AM, Andreas Tille wrote:

> If it comes to Blends topic I am an UDD folk. ;-)
> You should simply specify what data you need and I will go for it.

So, a machine readable file (YAML/JSON/deb822/etc, your choice),
something like this:

Blend: Debian Med
Task: Biology
Task-name: med-bio
URL: http://debian-med.alioth.debian.org/tasks/bio
Packages: abacas acedb-other 

Blend: Debian Med
Task: Biology development
Task-name: med-bio-dev
URL: http://debian-med.alioth.debian.org/tasks/bio-dev
Packages: bioperl bioperl-run

> I'm sorry I can not really parse what you want to tell me with this.

I'll try again. The PTS is primarily source package based but the
blends infrastructure is binary package based. The PTS will want to
link to the blends infrastructure, using per-package anchors. The
anchors provided by the blends infrastructure are based on binary
package names rather than source package names. Here is an example
based on the logol source package:

http://debian-med.alioth.debian.org/tasks/bio#logol-bin";
title="Debian Med Biology packages">med-bio
http://blends.alioth.debian.org/";>blend

It would be much better if the PTS could link directly to a source
package anchor instead, especially since blends can sometimes depend
on multiple binary packages from one source package. Unfortunately it
isn't yet possible to do this:

http://debian-med.alioth.debian.org/tasks/bio#logol";
title="Debian Med Biology packages">med-bio
http://blends.alioth.debian.org/";>blend

Alternatively we can drop the anchor, since I guess folks clicking on
blends links are looking for other related packages rather than the
package itself.

Thats a separate issue to what you pointed out with gromacs, the
gromacs issue can be solved like this:

http://blends.alioth.debian.org/debichem/tasks/molmech#gromacs";
title="DebiChem Molecular mechanics packages">debichem-molmech,
http://debian-med.alioth.debian.org/tasks/bio#gromacs";
title="Debian Med Biology packages">med-bio
http://blends.alioth.debian.org/";>blends

And if the length of the blends list gets too long we can do it like this:

http://blends.alioth.debian.org/";>blends
(http://blends.alioth.debian.org/debichem/tasks/molmech#gromacs";
title="DebiChem Molecular mechanics packages">1,
http://debian-med.alioth.debian.org/tasks/bio#gromacs";
title="Debian Med Biology packages">2,
http://debian-med.alioth.debian.org/tasks/cloud#gromacs";
title="Debian Med Cloud packages">3)

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

On Tue, Mar 19, 2013 at 11:14:45PM +0800, Paul Wise wrote:
> > This can be approached and I could perfectly imagine to put the content
> > of all tasks files into an UDD table if this might simplify things.  But
> > also here we should choose the table design apropriately:  You can either
> > give the Blend / Task preference like
> 
> The PTS converts data in various into XML files for rendering using
> XSLT, but importing via a UDD CGI sounds fine, we would need the UDD
> folks to add that script though.

If it comes to Blends topic I am an UDD folk. ;-)
You should simply specify what data you need and I will go for it.
 
> > They are:
> > 
> >http://debian-med.alioth.debian.org/tasks/bio#gromacs
> > 
> > to stick to the later example of gromacs in Debian Med.
> 
> Here is a more problematic example, #logol doesn't exist but #logol-bin
> does. It might be possible to do the second link, the first is easier.
> 
> http://debian-med.alioth.debian.org/tasks/bio#logol
> http://debian-med.alioth.debian.org/tasks/bio#logol-bin

I'm sorry I can not really parse what you want to tell my with this.

Kind regards

  Andreas.

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Paul Wise]

On Tue, 2013-03-19 at 10:47 +0100, Andreas Tille wrote:
> On Tue, Mar 19, 2013 at 04:33:41PM +0800, Paul Wise wrote:
> > jr, games blends
> 
> For the moment I see some problem here because a single package could
> perfectly be part of more than one Blend as well as part of more than
> one task of a single blend.  For instance consider gnuplot in Debian
> Science:

Right, hence the links to multiple blends in the example I gave.

> So putting a binary package into a task is not a strict (=exclusive)
> categorisation because one binary package could serve several different
> purposes.  I have no idea whether this can be reasonably reflected in
> PTS.

I think it can, with multiple links.

> This can be approached and I could perfectly imagine to put the content
> of all tasks files into an UDD table if this might simplify things.  But
> also here we should choose the table design apropriately:  You can either
> give the Blend / Task preference like

The PTS converts data in various into XML files for rendering using
XSLT, but importing via a UDD CGI sounds fine, we would need the UDD
folks to add that script though.

> They are:
> 
>http://debian-med.alioth.debian.org/tasks/bio#gromacs
> 
> to stick to the later example of gromacs in Debian Med.

Here is a more problematic example, #logol doesn't exist but #logol-bin
does. It might be possible to do the second link, the first is easier.

http://debian-med.alioth.debian.org/tasks/bio#logol
http://debian-med.alioth.debian.org/tasks/bio#logol-bin

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Bug#703402: PTS: link to the blends website for packages involved in blends

[Jonas Smedegaard]

Quoting Paul Wise (2013-03-19 09:33:41)
> For packages that are involved in a Debian Pure Blend, it would be good
> if the PTS were to link to the blends site from the links on the right.

Good idea!

I suggest to do it for _all_ Debian Blends, not only pure¹ ones, 
similar to how QA pages also include tracking of derivatives.


 - Jonas

¹ For distinction between "Debian Pure Blends" please see 
, specifically the recent 
update: 

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private


signature.asc
Description: signature

Bug#703402: PTS: link to the blends website for packages involved in blends

[Andreas Tille]

Hi Paul,

that's an interesting feature I have not thought about - thanks for
bringing this up.

On Tue, Mar 19, 2013 at 04:33:41PM +0800, Paul Wise wrote:
> For packages that are involved in a Debian Pure Blend, it would be good
> if the PTS were to link to the blends site from the links on the right.
> 
> jr, games blends

For the moment I see some problem here because a single package could
perfectly be part of more than one Blend as well as part of more than
one task of a single blend.  For instance consider gnuplot in Debian
Science:

$ grep gnuplot projects/science/trunk/debian-science/tasks/*
projects/science/trunk/debian-science/tasks/nanoscale-physics:Depends: gnuplot, 
grace
projects/science/trunk/debian-science/tasks/robotics:Depends: octave, gnuplot
projects/science/trunk/debian-science/tasks/viewing:Depends: gnuplot, grace, 
gri, labplot, graphviz, pyxplot, plotdrop, qtiplot

or gromacs in DebiChem and Debian Med:

$ grep gromacs projects/{debichem,med}/trunk/{debichem,debian-med}/tasks/* 
2>/dev/null
projects/debichem/trunk/debichem/tasks/molmech:Depends: gromacs
projects/med/trunk/debian-med/tasks/bio:Depends: gromacs
projects/med/trunk/debian-med/tasks/cloud:  gromacs,

So putting a binary package into a task is not a strict (=exclusive)
categorisation because one binary package could serve several different
purposes.  I have no idea whether this can be reasonably reflected in
PTS.
 
> Requirements on the blends side are a machine-readable mapping between
> the blends, their URLs and the packages that are available in them.

This can be approached and I could perfectly imagine to put the content
of all tasks files into an UDD table if this might simplify things.  But
also here we should choose the table design apropriately:  You can either
give the Blend / Task preference like

   create table blends (
 blend text,
 task  text,
 pkg   text
   )

or you could add an array valued field to the packages table mentioning

 ...
 blends text[],
 blends_tasks [],  -- task name needs to be prefixed by blend name
 ...

to reflect the nature of the tasks (and I'm in preference of the first
table layout because the second is more than hackish).

> The
> blends sites appear to be binary package oriented, we would probably
> also need anchors named after the source packages to link to?

They are:

   http://debian-med.alioth.debian.org/tasks/bio#gromacs

to stick to the later example of gromacs in Debian Med.

Kind regards

  Andreas.

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#703402: PTS: link to the blends website for packages involved in blends

[Paul Wise]

Package: qa.debian.org
Severity: normal
User: qa.debian@packages.debian.org
Usertags: pts
X-Debbugs-CC: debian-ble...@lists.debian.org

For packages that are involved in a Debian Pure Blend, it would be good
if the PTS were to link to the blends site from the links on the right.

jr, games blends

Requirements on the blends side are a machine-readable mapping between
the blends, their URLs and the packages that are available in them. The
blends sites appear to be binary package oriented, we would probably
also need anchors named after the source packages to link to?

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part