[aur-dev] Package blacklist for the AUR
Hi! I recently started working on a package blacklist for the official AUR which can e.g. be used to prevent people from uploading packages in the official repos to [unsupported] (cf. FS#12902 [1]). Patches can be found in the "pkg-blacklist" branch of my working tree [2]. They currently include some code that adds a "PackageBlacklist" table and a hacky helper utility that can be used to update that table by sync'ing it with some binary repos. Constructive criticism and suggestions welcome! [1] https://bugs.archlinux.org/task/12902 [2] http://git.cryptocrack.de/aur.git/log/?h=pkg-blacklist
[aur-dev] Old, unapplied patches
Hello AUR developers, there is a great backlog of patches that work well, but are not applied. Among them are: * The right-to-left fixes for the hebrew translation * Making links in comments clickable * Deletion through the web interface It would be great if you could take some time to review and apply them. Regards, PyroPeter -- freenode/pyropeter ETAOIN SHRDLU
Re: [aur-dev] Package blacklist for the AUR
On Mon, Feb 7, 2011 at 4:06 AM, Lukas Fleischer wrote: > Hi! > > I recently started working on a package blacklist for the official AUR > which can e.g. be used to prevent people from uploading packages in the > official repos to [unsupported] (cf. FS#12902 [1]). Patches can be found > in the "pkg-blacklist" branch of my working tree [2]. They currently > include some code that adds a "PackageBlacklist" table and a hacky > helper utility that can be used to update that table by sync'ing it with > some binary repos. > > Constructive criticism and suggestions welcome! > > [1] https://bugs.archlinux.org/task/12902 > [2] http://git.cryptocrack.de/aur.git/log/?h=pkg-blacklist > All around pretty awesome. I wouldn't have written it in C myself but more power to you! What about provides lists from packages as well? Can you add those to the blacklist? I think you (or someone) should but maybe I am wrong. Two char indent? Eew ;) JK, it's better than tabs... -- -Justin
Re: [aur-dev] Package blacklist for the AUR
Lukas Fleischer wrote: > Hi! > > I recently started working on a package blacklist for the official AUR > which can e.g. be used to prevent people from uploading packages in the > official repos to [unsupported] (cf. FS#12902 [1]). Patches can be found > in the "pkg-blacklist" branch of my working tree [2]. They currently > include some code that adds a "PackageBlacklist" table and a hacky > helper utility that can be used to update that table by sync'ing it with > some binary repos. > > Constructive criticism and suggestions welcome! > > [1] https://bugs.archlinux.org/task/12902 > [2] http://git.cryptocrack.de/aur.git/log/?h=pkg-blacklist Whay are you using LOCK TABLE instead of a transaction? Are you using MyIsam? __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: [aur-dev] Package blacklist for the AUR
On Mon, Feb 7, 2011 at 3:06 AM, Lukas Fleischer wrote: > Hi! > > I recently started working on a package blacklist for the official AUR > which can e.g. be used to prevent people from uploading packages in the > official repos to [unsupported] (cf. FS#12902 [1]). Patches can be found > in the "pkg-blacklist" branch of my working tree [2]. They currently > include some code that adds a "PackageBlacklist" table and a hacky > helper utility that can be used to update that table by sync'ing it with > some binary repos. > > Constructive criticism and suggestions welcome! > > [1] https://bugs.archlinux.org/task/12902 > [2] http://git.cryptocrack.de/aur.git/log/?h=pkg-blacklist AUR side: * Using CHAR as a datatype is absolutely silly in new code, use VARCHAR, and why do anything shorter than 255 or 512? * On that note, almost all CHAR usages in the current schema are silly and should be using VARCHAR- anything on the Packages table, PackageCategories, PackageSources, TU_VoteInfo, AccountTypes, Username/Email/Passwd/IRCNick on Users. * Why not just make "Name" your primary key? The ID column is never used. * This is a slight step toward removing DummyPkg stuff from the Packages table- I'm trying to think out how to restructure that data to make things work and still be slightly sound from a relational and keys point of view. Blacklist helper side: * I won't lie, I think this is over-engineered a tad. This can be done in a much shorter and easier to hack shell script since all you need is package names- just pipe bsdtar output through some magic and you have package names. I've attached a sample starter script I use for archweb updates. I'd probably have it call bsdtar and then who knows what. * Oh my, I forgot we are still on MyISAM for the AUR. Please to god switch to InnoDB and use transactions instead. -Dan fetch_update_package_db.sh Description: Bourne shell script
[aur-dev] Moving packages to Community
Hello official and inofficial maintainers and AUR devs.
Following the discussion in aur-general, I think some automation could
solve many issues at once here. Justin Davis' community-blacklist
patch is an approach in about the right direction although in
relation to the discussion concerning these prevailing [more-official
repo] adoption issues, it seems that it does not yet go far enough at
the moment. There should be a redirection or at least a
[we're-now-official] error page showing sort of "congratulations and
sorry for the late notice" to the yet uninformed aur maintainer, if
his package is not reachable any more because it was adopted in said
[more-official repo] - along with this could be displayed the comments
of the former package. As well, it could just become invisible in the
package's search, and marked with a "I saw this" button in the former
AUR maintainer's package list. When that button is clicked, the
package would sink quietly into the official repo without anyone
needing to complain.
The only hurdle remaining is, that a package isn't up for deletion any
more when it gets adopted to [more-official repo], because that could
just be automated by the repo's backend maybe? I'm probably going to
download the aur source inclusively Justin's approach to look into
this a bit further. Even though my development skills aren't exactly
in line with such a task, and if someone feels more confident, one may
look into it h{im,er}self.
I'd like to hear your comments on this well.
cheers!
mar77i
Re: [aur-dev] Package blacklist for the AUR
On Mon, Feb 07, 2011 at 10:08:54AM -0600, Dan McGee wrote: > AUR side: > * Using CHAR as a datatype is absolutely silly in new code, use > VARCHAR, and why do anything shorter than 255 or 512? > * On that note, almost all CHAR usages in the current schema are silly > and should be using VARCHAR- anything on the Packages table, > PackageCategories, PackageSources, TU_VoteInfo, AccountTypes, > Username/Email/Passwd/IRCNick on Users. I just copied that from another table schema, but you're absolutely right. I'll work on replacing all that CHAR stuff with VARCHARs later. Might become a kinda ugly updating process from 1.7.0 to 1.8.0 :) > * Why not just make "Name" your primary key? The ID column is never used. I'm not really sure about this. Some people insist on always having an "ID" column. This will become useful if we add some web frontend to the blacklist, e.g. Not sure if there are any coding guidelines about this at all, but it's correct that we don't necessarily need this here. > Blacklist helper side: > * I won't lie, I think this is over-engineered a tad. This can be done > in a much shorter and easier to hack shell script since all you need > is package names- just pipe bsdtar output through some magic and you > have package names. I've attached a sample starter script I use for > archweb updates. I'd probably have it call bsdtar and then who knows > what. Our consensus was not to fetch tarballs via HTTP and extract them here, but use libalpm instead. We already discussed using a small PHP script that uses Archive::Tar (PEAR) but discarded that. Using libalpm, future database format changes won't affect us and it just seems cleaner. Of course, we could do that with some shell script which would have to do following things tho: - parse the AUR "config.inc" file: read MySQL host name, socket, user name, password - use pacman(8) to sync local databases - convert packages to MySQL queries using some sed(1)/awk(1) magic - pipe stuff to mysql(1), ensure no errors occurred I just think that it wouldn't be much shorter (that C helper has about 100 SLOC if you strip all that error handling stuff), nor cleaner, nor faster. Best thing would to have PHP bindings for libalpm here... Well. Recommendations and patches welcome :p > * Oh my, I forgot we are still on MyISAM for the AUR. Please to god > switch to InnoDB and use transactions instead. Full ack. I'll talk to Loui about that.
Re: [aur-dev] Package blacklist for the AUR
On Mon, Feb 07, 2011 at 06:50:14PM +0100, Lukas Fleischer wrote: > > Blacklist helper side: > > * I won't lie, I think this is over-engineered a tad. This can be done > > in a much shorter and easier to hack shell script since all you need > > is package names- just pipe bsdtar output through some magic and you > > have package names. I've attached a sample starter script I use for > > archweb updates. I'd probably have it call bsdtar and then who knows > > what. > > Our consensus was not to fetch tarballs via HTTP and extract them here, > but use libalpm instead. We already discussed using a small PHP script > that uses Archive::Tar (PEAR) but discarded that. Using libalpm, future > database format changes won't affect us and it just seems cleaner. > > Of course, we could do that with some shell script which would have to > do following things tho: > > - parse the AUR "config.inc" file: read MySQL host name, socket, user > name, password > - use pacman(8) to sync local databases > - convert packages to MySQL queries using some sed(1)/awk(1) magic > - pipe stuff to mysql(1), ensure no errors occurred > > I just think that it wouldn't be much shorter (that C helper has about > 100 SLOC if you strip all that error handling stuff), nor cleaner, nor > faster. Best thing would to have PHP bindings for libalpm here... Well. > Recommendations and patches welcome :p Oh, and I'll probably make this script add package names listed in packages' "provides" arrays to the blacklist as well (thanks to Justin Davis, just read his reply).
