Bug#496655: State of pci.ids

[Otavio Salvador]

Adeodato Simó <[EMAIL PROTECTED]> writes:

>> The biggest issue is that we'd need to do it before d-i RC1 or another
>> installer upload would be required to get it in sync with the archive.
>
> It has been uploaded now. Shall I unblock it already? Does it need to
> get into testing fast, or just having it in unstable is enough for the
> RC1?

It needs to go together with other RC1 modules. Please let's wait
until libd-i and kernel gets built for all architectures so we can do
all it together. 

-- 
O T A V I OS A L V A D O R
-
 E-mail: [EMAIL PROTECTED]  UIN: 5906116
 GNU/Linux User: 239058 GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
-
"Microsoft sells you Windows ... Linux gives
 you the whole house."



--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Adeodato Simó]

* Otavio Salvador [Mon, 29 Sep 2008 20:51:20 -0300]:

> Aníbal Monsalve Salazar <[EMAIL PROTECTED]> writes:

> > In the changelog of pciutils 1:2.1.11-15.3, Christian Perrier applied a
> > patch by Colin Watson which stripped the subsystem ids. That changelog
> > is dated 3 Dec 2005.

> This is indeed a bug since it was suppose to affect only the udeb . If
> we can prepare a package with the unstriped version for regular deb
> and the striped one for the d-i I see no big problem in accepting it
> for Lenny.

> The biggest issue is that we'd need to do it before d-i RC1 or another
> installer upload would be required to get it in sync with the archive.

It has been uploaded now. Shall I unblock it already? Does it need to
get into testing fast, or just having it in unstable is enough for the
RC1?

-- 
Adeodato Simó dato at net.com.org.es
Debian Developer  adeodato at debian.org
 
The first step on the road to wisdom is the admission of ignorance. The
second step is realizing that you don't have to blab it to the world.




--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Martin Mares]

Hello!

> Readable: I don't find that actual version is readable, in particular
> with biggest vendor (i.e. I need a lot of scroll to see if I'm in the
> right section).

This is however not solved by repeating the vendor ID at every line,
because people usually do not remember the vendor by the ID ;-)

> It was a side note: if you need to change it (because of split, etc.)
> I propose to change also the format.  Merging different sources is not
> so simple (but also not so complex). I would move the logic from code
> (and runtime) to data.

The problem is that your proposal solves only a fraction of the whole
problem and quite likely the least important one. For example, if you
use `cat | sort -su' to merge ID files, you get the comments completely
wrong and you are not able to handle deletions.

The tools we use for the maintenance of the ID database work with a similar
format internally, but with several tab-separated fields on each line
(name, comment, approval state etc.), which is useful in scripts, but
awful as a distribution format (fixed order of fields etc.). So I would
rather like to keep the current distribution format and let the people
use the tools to convert it to whatever they wish.

> Could you elaborate this?
> grep '^ '
> grep '^   '
> would give the subsystem in two simple lines.

Yes, except that it depends on the version of the ID file. Three years
ago, the stand-alone subsystems did not exist, so people would assume
that the second line is sufficient. Suddenly, it no longer is. Extensibility
is important, therefore I believe that it is better to use the lookup
algorithm from libpci as a black box.

> But I don't find any of such kind of detection in kernel (e.g.
> no checking of subsystem ids without a fix vendor/device ids).

Many motherboard manufacturers tend to stick a common subsystem ID on
all devices on the board. It rarely makes a functional differences, so
the drivers usually do not match such ID's (even if the PCI core in the
kernel supports such matches), but lspci should tell that the ID belongs
to a motherboard and it is somewhat boring to repeat it all over for
all devices.

> ok. This would be nice! So maybe we could have a common tools that
> handle, grep, merge different ids files.

Yes, we definitely want to release such a toolbox.

Have a nice fortnight
-- 
Martin `MJ' Mares  <[EMAIL PROTECTED]>   
http://mj.ucw.cz/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
This message is transmitted on 100% recycled electrons.



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Giacomo A. Catenazzi]

Martin Mares wrote:

I think that changing the format of the file (with other suffix) would
also be helpful, i.e. instead of using tab-indent I would explicitly 
writing vendor id (ev. other implicit ids) in every line.


In this manner it is easier to grep for hardware, and also to merge
files from different sources (cat | sort -u), without requiring
external libraries or complex scripts.


It was a "side proposal", and probably offtopic, but anyway.



First, I do not believe it would help anything -- the file would be less
readable, larger and harder to edit. This is not a fair price for
simplifying a couple of scripts.


I agree that the file would be a lot bigger (so no ideal for
installations, but I think it is optimal to compress, and maybe a better
tradeoff between code and data sizes.
Readable: I don't find that actual version is readable, in particular
with biggest vendor (i.e. I need a lot of scroll to see if I'm in the
right section).
Harder to edit: see usb.ids: it is difficult to distinguish tab from
space (but the library could accept 8 spaces as tab).


Second, merging of pci.ids does not help with the updating problem
I have described. Do you see any way how it could?


It was a side note: if you need to change it (because of split, etc.)
I propose to change also the format.  Merging different sources is not
so simple (but also not so complex). I would move the logic from code
(and runtime) to data.



Third, while grepping for ID's is appealing, it is nowhere as simple as
it looks -- for example, the subsystems can be present either as
sub-items of the devices, or as stand-alone entries.


Could you elaborate this?
grep '^ '
grep '^   '
would give the subsystem in two simple lines.

But I don't find any of such kind of detection in kernel (e.g.
no checking of subsystem ids without a fix vendor/device ids).



[Note: other files *try* to use the same format, e.g. usb.ids, zorro.ids
and eisa.ids (in kernel sourcers). But usb.ids has frequently spaces
instead of tabs.]


I hope this will get much better soon as a common administration system
for all the ID files is almost finished.


ok. This would be nice! So maybe we could have a common tools that
handle, grep, merge different ids files. This would remove most of my 
concerns.


ciao
cate



--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Martin Mares]

> I think that changing the format of the file (with other suffix) would
> also be helpful, i.e. instead of using tab-indent I would explicitly 
> writing vendor id (ev. other implicit ids) in every line.
> 
> In this manner it is easier to grep for hardware, and also to merge
> files from different sources (cat | sort -u), without requiring
> external libraries or complex scripts.

First, I do not believe it would help anything -- the file would be less
readable, larger and harder to edit. This is not a fair price for
simplifying a couple of scripts.

Second, merging of pci.ids does not help with the updating problem
I have described. Do you see any way how it could?

Third, while grepping for ID's is appealing, it is nowhere as simple as
it looks -- for example, the subsystems can be present either as
sub-items of the devices, or as stand-alone entries.

> [Note: other files *try* to use the same format, e.g. usb.ids, zorro.ids
> and eisa.ids (in kernel sourcers). But usb.ids has frequently spaces
> instead of tabs.]

I hope this will get much better soon as a common administration system
for all the ID files is almost finished.

Have a nice fortnight
-- 
Martin `MJ' Mares  <[EMAIL PROTECTED]>   
http://mj.ucw.cz/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
There really exists a Microsoft product that doesn't suck -- unfortunately, 
it's a vacuum cleaner.



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Giacomo A. Catenazzi]

Martin Mares wrote:

Dropping this information in the udeb is if course a good way of saving
space, but the full package should contain everything.


In the future (after Lenny), I would like to solve one more problem:
with the current rate of development of new hardware, the pci.ids file
is getting out of date very quickly and there is no way how to update
it nicely.

The problem is somewhat mitigated by the network lookup feature present
since pciutils 3.0.0, but not all users are connected to the network
all the time (and not all programs linked with libpci know how to enable
the lookups), so it still makes sense to update the pci.ids file.

Of course, we have the update-pciids script, but does its job in a dirty
way as it rewrites the pci.ids file coming from the pciutils package, so
the file system is no longer consistent with the packages (and debsums
fail and so on).

We might make libpci look for the file at several places and keep the
updated copy somewhere in /var/, but it would make updates of the package
painful: which version is the more up-to-date? The one in /usr/share,
or the one in /var? So it does not help either.

I suggest that we should split off the pci.ids file to a separate 
arch-independent
package with a date-based version number, so that the users can mix pci.ids
from several sources and still get consistent upgrades:

   o  the base version present in the distribution;
   o  potential updates from debian-volatile (once in a month?);
   o  daily snapshots from the PCI ID database (I can easily make
  the snapshot machinery produce Debian packages as well).

What do you think of this approach?


I think that changing the format of the file (with other suffix) would
also be helpful, i.e. instead of using tab-indent I would explicitly 
writing vendor id (ev. other implicit ids) in every line.


In this manner it is easier to grep for hardware, and also to merge
files from different sources (cat | sort -u), without requiring
external libraries or complex scripts.

See for example pci.list in http://cateee.net/sources/lkddb


[Note: other files *try* to use the same format, e.g. usb.ids, zorro.ids
and eisa.ids (in kernel sourcers). But usb.ids has frequently spaces
instead of tabs.]

ciao
cate



--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Luk Claes]

Martin Mares wrote:

> In the future (after Lenny), I would like to solve one more problem:
> with the current rate of development of new hardware, the pci.ids file
> is getting out of date very quickly and there is no way how to update
> it nicely.

> I suggest that we should split off the pci.ids file to a separate 
> arch-independent
> package with a date-based version number, so that the users can mix pci.ids
> from several sources and still get consistent upgrades:
> 
>o  the base version present in the distribution;
>o  potential updates from debian-volatile (once in a month?);
>o  daily snapshots from the PCI ID database (I can easily make
>   the snapshot machinery produce Debian packages as well).
> 
> What do you think of this approach?

volatile is up to the volatile maintainers, though updating it on
volatile won't make it usefull for installing new systems...

As for etchnahalf, we are also planning to do a lennynahalf which can
include updated packages for new hardware. New updates can also be done
in point releases after *nhalf by the way. For more info you'd best also
contact the project-etchnahalf @ teams.debian.net mailing list.

Cheers

Luk



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Martin Mares]

> volatile is up to the volatile maintainers, though updating it on
> volatile won't make it usefull for installing new systems...

The ability to update the pci.ids from -volatile was meant as a small
icing at the top of the cake -- I see the real benefit in the possibility
of fetching a package with the new ID file from the PCI ID repository
while keeping consistent upgrades.

Martin



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Martin Mares]

> Dropping this information in the udeb is if course a good way of saving
> space, but the full package should contain everything.

In the future (after Lenny), I would like to solve one more problem:
with the current rate of development of new hardware, the pci.ids file
is getting out of date very quickly and there is no way how to update
it nicely.

The problem is somewhat mitigated by the network lookup feature present
since pciutils 3.0.0, but not all users are connected to the network
all the time (and not all programs linked with libpci know how to enable
the lookups), so it still makes sense to update the pci.ids file.

Of course, we have the update-pciids script, but does its job in a dirty
way as it rewrites the pci.ids file coming from the pciutils package, so
the file system is no longer consistent with the packages (and debsums
fail and so on).

We might make libpci look for the file at several places and keep the
updated copy somewhere in /var/, but it would make updates of the package
painful: which version is the more up-to-date? The one in /usr/share,
or the one in /var? So it does not help either.

I suggest that we should split off the pci.ids file to a separate 
arch-independent
package with a date-based version number, so that the users can mix pci.ids
from several sources and still get consistent upgrades:

   o  the base version present in the distribution;
   o  potential updates from debian-volatile (once in a month?);
   o  daily snapshots from the PCI ID database (I can easily make
  the snapshot machinery produce Debian packages as well).

What do you think of this approach?

Have a nice fortnight
-- 
Martin `MJ' Mares  <[EMAIL PROTECTED]>   
http://mj.ucw.cz/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
"The road to hell is paved with melting snowballs." -- Larry Wall



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Martin Mares]

Hello!

> My point is that pci.ids has been stripped in both the udeb and the
> normal binary packages for a number of debian versions of pciutils.

Yes, but only in testing. The Etch version is complete.

Have a nice fortnight
-- 
Martin `MJ' Mares  <[EMAIL PROTECTED]>   
http://mj.ucw.cz/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
"All that is necessary for the triumph of evil is that good men do nothing." -- 
E. Burke



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Martin Mares]

Hello and thanks for the prompt response!

> Martin, what is the pros of having this striped data available?

In many cases, the subsystem ID is needed to reliably identify a PCI card
as the manufacturers are used to make multiple cards with the same chip,
but wired differently.

Dropping this information in the udeb is if course a good way of saving
space, but the full package should contain everything.

Have a nice fortnight
-- 
Martin `MJ' Mares  <[EMAIL PROTECTED]>   
http://mj.ucw.cz/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
Homo homini lupus, frater fratri lupior, bohemus bohemo lupissimus.



-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Aníbal Monsalve Salazar]

On Mon, Sep 29, 2008 at 08:51:20PM -0300, Otavio Salvador wrote:
>The biggest issue is that we'd need to do it before d-i RC1 or another
>installer upload would be required to get it in sync with the archive.

I'll upload 1:3.0.0-5 for the consideration of the D-I and the release
teams this evening.


signature.asc
Description: Digital signature

Bug#496655: State of pci.ids

[Aníbal Monsalve Salazar]

On Tue, Sep 30, 2008 at 01:24:32AM +0200, Julien Cristau wrote:
>... for the udeb, not for the main package.

On Tue, Sep 30, 2008 at 12:25:19AM +0100, Steve McIntyre wrote:
>That's all well and good, but if something is stripped for D-I, it
>should only be stripped for the udeb, not for the version on the
>installed system (as supplied by the normal package).

My point is that pci.ids has been stripped in both the udeb and the
normal binary packages for a number of debian versions of pciutils.

I'll upload 1:3.0.0-5 to have a complete pci.ids in the normal binary
package for consideration of the D-I and release teams. The pci.ids in
the udeb will remain stripped and compressed.


signature.asc
Description: Digital signature

Bug#496655: State of pci.ids

[Otavio Salvador]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Aníbal Monsalve Salazar <[EMAIL PROTECTED]> writes:

> In the changelog of pciutils 1:2.1.11-15.3, Christian Perrier applied a
> patch by Colin Watson which stripped the subsystem ids. That changelog
> is dated 3 Dec 2005.

This is indeed a bug since it was suppose to affect only the udeb . If
we can prepare a package with the unstriped version for regular deb
and the striped one for the d-i I see no big problem in accepting it
for Lenny.

The biggest issue is that we'd need to do it before d-i RC1 or another
installer upload would be required to get it in sync with the archive.

Martin, what is the pros of having this striped data available?

- -- 
O T A V I OS A L V A D O R
- -
 E-mail: [EMAIL PROTECTED]  UIN: 5906116
 GNU/Linux User: 239058 GPG ID: 49A5F855
 Home Page: http://otavio.ossystems.com.br
- -
"Microsoft sells you Windows ... Linux gives
 you the whole house."
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8+ 

iEYEARECAAYFAkjhaekACgkQLqiZQEml+FWfiACeLoaAnavR+OP32jp/9jjgWD0d
rWkAn1AeygNXfxhDtfLmkVwgBA3w/RCd
=Uou+
-END PGP SIGNATURE-



--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Steve McIntyre]

On Tue, Sep 30, 2008 at 08:52:19AM +1000, Aníbal Monsalve Salazar wrote:
>On Mon, Sep 29, 2008 at 11:28:15PM +0200, Martin Mares wrote:
>>
>>I am concerned with the state of the pci.ids file in the current
>>pciutils package. It is seriously broken: It contains no subsystem
>>entries, no programming interface entries and no comments including the
>>top comment with version and copyright information.
>
>pci.ids has been stripped for a long time and it was done to reduce its
>size for the Debian-Installer.

That's all well and good, but if something is stripped for D-I, it
should only be stripped for the udeb, not for the version on the
installed system (as supplied by the normal package).

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
< sladen> I actually stayed in a hotel and arrived to find a post-it
  note stuck to the mini-bar saying "Paul: This fridge and
  fittings are the correct way around and do not need altering"




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Julien Cristau]

On Tue, Sep 30, 2008 at 09:17:31 +1000, Aníbal Monsalve Salazar wrote:

> In the changelog of pciutils 1:2.1.11-15.3, Christian Perrier applied a
> patch by Colin Watson which stripped the subsystem ids. That changelog
> is dated 3 Dec 2005.
> 
... for the udeb, not for the main package.

Cheers,
Julien



--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Bug#496655: State of pci.ids

[Aníbal Monsalve Salazar]

On Tue, Sep 30, 2008 at 08:52:19AM +1000, Anibal Monsalve Salazar wrote:
>On Mon, Sep 29, 2008 at 11:28:15PM +0200, Martin Mares wrote:
>>I am concerned with the state of the pci.ids file in the current
>>pciutils package. It is seriously broken: It contains no subsystem
>>entries, no programming interface entries and no comments including the
>>top comment with version and copyright information.
>
>pci.ids has been stripped for a long time and it was done to reduce its
>size for the Debian-Installer.

In the changelog of pciutils 1:2.1.11-15.3, Christian Perrier applied a
patch by Colin Watson which stripped the subsystem ids. That changelog
is dated 3 Dec 2005.

  pciutils (1:2.1.11-15.3) unstable; urgency=low

  [...]
[ Colin Watson ]
* Changes merged from Ubuntu patches:
  [...]
  - Reduce pciutils-udeb's copy of pci.ids by over 100KB by removing
subsystem ids, which I don't think we need in the installer, and
comments.
  [...]
   -- Christian Perrier <[EMAIL PROTECTED]>  Sat,  3 Dec 2005 08:16:54 +0100

>>As the upstream maintainer, I strongly believe that this problem should
>>be fixed for Lenny, because it makes the output of lspci too broken for
>>normal use.

We have used an stripped pci.ids for almost three years without
complains.


signature.asc
Description: Digital signature

Bug#496655: State of pci.ids

[Aníbal Monsalve Salazar]

On Mon, Sep 29, 2008 at 11:28:15PM +0200, Martin Mares wrote:
>Hello, world!\n
>
>I am concerned with the state of the pci.ids file in the current
>pciutils package. It is seriously broken: It contains no subsystem
>entries, no programming interface entries and no comments including the
>top comment with version and copyright information.

pci.ids has been stripped for a long time and it was done to reduce its
size for the Debian-Installer.

>As the upstream maintainer, I strongly believe that this problem should
>be fixed for Lenny, because it makes the output of lspci too broken for
>normal use.

My plan is to upload a complete pci.ids to unstable but not intended for
lenny unless both the D-I and release teams decide that they want
something different. At least, I can put back the copyright information.

>I have submitted this as Bug #496655 on August 26th, but the package
>maintainer has not reacted yet.
>
>Is there any standard mechanism for handling such problems?
>
>(If the current maintainer is no more interested in maintaining
>pciutils, I would volunteer to join Debian and take care of the package
>myself.)
>
>   Have a nice fortnight
>-- 
>Martin `MJ' Mares  <[EMAIL PROTECTED]>   
>http://mj.ucw.cz/
>Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
>"It is easier to port a shell than a shell script." -- Larry Wall


signature.asc
Description: Digital signature