Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-09 Thread Abhishek Dasgupta
2009/4/9 Daenyth Blank : > Can you pull from my sqlite-bash branch? I have some cleanup fixes > that will be easier to apply before proceeding, though the function is > still pretty much the same, > Done.

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-09 Thread Daenyth Blank
On Thu, Apr 9, 2009 at 13:50, Abhishek Dasgupta wrote: > 2009/4/9 Evangelos Foutras : >> Couldn't we simply introduce a new column, store the filename there and >> create an index on it? >> > > That's a nice idea. I'll try out and see how much the db size increases. > -- > Abhishek > Can you pull

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-09 Thread Abhishek Dasgupta
2009/4/9 Evangelos Foutras : > Couldn't we simply introduce a new column, store the filename there and > create an index on it? > That's a nice idea. I'll try out and see how much the db size increases. -- Abhishek

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-09 Thread Abhishek Dasgupta
2009/4/9 Daenyth Blank : > How is the filesize of the REPO.db? CMB was reporting that community > was taking up more than 75mb with his version. Regardless of how the > sqlite features would be implemented, I will also keep the flat file > version available, and configurable. How are the searching

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-09 Thread Evangelos Foutras
Chris Brannon wrote: "select * from files where filename like '%/getmail'" takes 5 seconds or so, because it has to iterate through all the filenames in the table. If sqlite were used for pkgfile, that second query would need to be quite a bit faster than it is now. -- Chris Couldn't we si

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-09 Thread Daenyth Blank
On Thu, Apr 9, 2009 at 13:40, Chris Brannon wrote: > Daenyth Blank wrote: >> How is the filesize of the REPO.db? CMB was reporting that community >> was taking up more than 75mb with his version. Regardless of how the >> sqlite features would be implemented, I will also keep the flat file >> versi

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-09 Thread Chris Brannon
Daenyth Blank wrote: > How is the filesize of the REPO.db? CMB was reporting that community > was taking up more than 75mb with his version. Regardless of how the > sqlite features would be implemented, I will also keep the flat file > version available, and configurable. How are the searching and

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-09 Thread Daenyth Blank
On Thu, Apr 9, 2009 at 12:31, Abhishek Dasgupta wrote: > 2009/4/9 Abhishek Dasgupta : >> I made a sqlite-bash branch on github [1] with the create_db >> function which I'm using right now. It's not incremental yet but >> adding a separate table with package versions like you've done >> would solve

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-09 Thread Abhishek Dasgupta
2009/4/9 Abhishek Dasgupta : > I made a sqlite-bash branch on github [1] with the create_db > function which I'm using right now. It's not incremental yet but > adding a separate table with package versions like you've done > would solve the problem. > Just added incremental update support to sqli

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-08 Thread Abhishek Dasgupta
2009/4/9 Chris Brannon : > I made a fork of pkgtools on github. > Have a look at the sqlite branch of http://github.com/CMB/pkgtools > I wrote a script which incrementally updates an SQLite database from a > REPO.files.tar.gz tarball.  It is in pkgtools/other/repofile2db.py > The code is terribly r

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-08 Thread Chris Brannon
Abhishek Dasgupta wrote: > The initial generation is taking really long! It's over two hours and still > generating... *SNIP* > Also, there has to be some way of escaping the filename so that sqlite > does not give errors like this: > SQL error: near "N": syntax error I made a fork of pkgtools on

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-08 Thread Abhishek Dasgupta
Also, I forgot to mention that the size of the dbs is quite large compared to the simple *files.tar.gz on ftp: core.files.db.tar.gz 220K core.db 2.1M core.db has 35796 files. extra.db (still generating) is already at 13M! -- Abhishek

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-08 Thread Abhishek Dasgupta
2009/4/8 Daenyth Blank : > That sounds promising... If we can find some method to construct only > the differences, then it would be doable. Have a method for the user > to do initial generation, update an existing db, and in addition, to > download a database from a known source (I could generate

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-08 Thread Daenyth Blank
On Wed, Apr 8, 2009 at 14:04, Abhishek Dasgupta wrote: > 2009/4/5 Daenyth Blank : >> That's not a bad idea, and it's one that's been recommended to me for >> pkgfile as well. Is there a simple (read: bash-usable) interface I >> could work with to speed up pkgfile? > > I was trying out the command

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-08 Thread Abhishek Dasgupta
2009/4/5 Daenyth Blank : > That's not a bad idea, and it's one that's been recommended to me for > pkgfile as well. Is there a simple (read: bash-usable) interface I > could work with to speed up pkgfile? I was trying out the command line sqlite3 tool to manage a database comprised of the followin

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-05 Thread Daenyth Blank
On Sun, Apr 5, 2009 at 09:29, Chris Brannon wrote: > Abhishek Dasgupta wrote: >> Thanks for the useful script! But shouldn't this line: >>         known_files[entry] =3D (repo, package) >> be >>         known_files[entry].append((repo, package)) > > That is a nice suggestion!  Using lists as hash

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-05 Thread Chris Brannon
Abhishek Dasgupta wrote: > Thanks for the useful script! But shouldn't this line: > known_files[entry] =3D (repo, package) > be > known_files[entry].append((repo, package)) That is a nice suggestion! Using lists as hash values could make the tool useful for purposes other than che

Re: [aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-04 Thread Abhishek Dasgupta
2009/4/5 Chris Brannon : > I wrote this code in anger.  Hopefully, others might find it useful. > There is a little story behind it. > > I maintain the nmh package in unsupported.  I just discovered that a > package from extra provides /usr/bin/dp, which is also provided by nmh. > We have this nice

[aur-general] pkgconflict: a tool to find file conflicts when building packages

2009-04-04 Thread Chris Brannon
I wrote this code in anger. Hopefully, others might find it useful. There is a little story behind it. I maintain the nmh package in unsupported. I just discovered that a package from extra provides /usr/bin/dp, which is also provided by nmh. We have this nice collection of files under /var/cach