Re: [gentoo-dev] stabilization candidates rss feed & html pages
> If the euscan guys want to integrate the feature, nice. > If not, lets just stick with this script. It is simple enough that even > ruby n00bs like me can understand what it does :P I'm a ruby noob and seems pretty simple, I don't think that porting it to python would be a huge effort. Moreover we can base it on existing euscan code. I'll talk with Corentin about that ;) Thanks! -- f. "There are only two hard things in Computer Science: cache invalidation, naming things and off-by-one errors." signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] RFC: new "qt" category
On 17/01/2013 14:57, Ben de Groot wrote: > Presently we already have a good number of split qt-* library packages > in x11-libs. How many? └> ls -d /usr/portage/x11-libs/qt* | wc -l 22 > We, the Gentoo Qt team, are of the opinion that the time has > come to split all these out into their own category. -1 No way. > This means x11-libs/qt-core will be moved to qt/core, and so on. Do you really want me to do "emerge core"? Moreover all other categories are called "major-minor" as Diego pointed out. Consistency matters. -- f. "There are only two hard things in Computer Science: cache invalidation, naming things and off-by-one errors." signature.asc Description: OpenPGP digital signature
[gentoo-dev] [RFC] euscan: Need to add more upstream info in metadata.xml
Hi everybody! euscan is available in portage as a dev package (app-portage/euscan-). This tool allows to check if a given package/ebuild has new upstream versions or not. It uses different heuristics to scan upstream and grab new versions and related urls. euscan can use either custom "handlers" for well known upstream (github, pypi, cpan, sourceforge, google-code, etc..) or use directory scanning using SRC_URI. If directory scan fails for some reason, euscan will fallback to brute force (generating possible next version number and trying to fetch those packages). The problem that we're facing with euscan is that some packages in upstream use strange version numbers or the list of available versions is placed in a location that is totally different from SRC_URI. Examples: - MySQL: most MySQL mirrors are not browsable (always fallback to brute force) - webalizer uses strange version numbers in upstream (ftp://ftp.mrunix.net/pub/webalizer/), in this case euscan should be aware that 2.21-02 is the version number in upstream and scan the ftp directory searching for webalizer-(\d+).(\d+)-(\d+).tar.gz. The last version of webalizer, 2.23.05, is not recognized by euscan and is not available in gentoo. - Authen-SASL-Cyrus in upstream uses “-server” in version numbers http://www.cpan.org/authors/id/P/PB/PBOETTCH/ - XML-Tidy that uses stranges letters in version number We thought about how to solve this issue and we agreed that the best way to handle the problem for every specific case was adding some more information in metadata.xml. In Debian, uscan uses information from debian/watch inside debian packages, hence as so much work is already done we thought about taking this info from watch files and save it in metadata.xml to make euscan use it. I wrote a simple script that patches metadata.xml adding an experimental tag with data from debian packages: https://github.com/volpino/euscan/blob/master/bin/euscan_patch_metadata A basic watch data contains a base url to scan and a pattern to search into it: Example: base: http://icedtea.classpath.org/download/source/ pattern: icedtea-([\d\.]+).tar.gz Which means "open that url and search for the links that match that pattern". This is useful for example when is not possible to retrieve the base url from SRC_URI (icedtea’s SRC_URI is http://icedtea.classpath.org/hg/release/icedtea7-forest-2.2/hotspot/archive/889dffcf4a54.tar.gz) Advanced usage with directory pattern: Example: base: http://ftp.gwdg.de/pub/misc/mysql/Downloads/MySQL-([\d\.]+) pattern: mysql-([\d\.]+).tar.gz Scans all directories that match the query looking for links that match the pattern We need also some options for mangling versions and download url: these options can contain regexps or names of mangling rules (e.g.: "cpan" means apply mangling rules for CPAN versions) Version mangling example: As mentioned above webalizer uses both dots and hyphens in version numbers, so an option like this is required versionmangle=”s/-/./” Download url mangling example: Page scan on berlios returns an url like this: http://prdownload.berlios.de/mirageiv/mirage-0.9.tar.gz that should be mangled to get a working download url with an option like downloadurlmangle=”s/prdownload/download/” (for more info see uscan manpage) Another example: dev-perl/Math-BaseCnv or XML-Tidy in upstream use strange version numbers like 1.8.B59BrZ that should be mangled to 1.8 Summarizing we need: - A base url and a file pattern to search for new upstream versions when SRC_URI is not suitable - some options for mangling retrieved data from the scan of upstream using base url and pattern or using remote-id information So our problem is: how can we store this data in a very flexible and efficient way? Proposed solutions: 1) Add an euscan tag with a custom namespace Example: http://euscan.iksaif.net";> ab Which means: apply regex s/a/b/ then apply cpan mangling rules and then gentoo mangling rules. 2) Change quite heavily the remote-id tag: - adding versionmanging and downloadmangling options that contain regexes - adding a new remote-id type called for example url, that tag will contain the base url and the pattern 3) Add a watch tag to with versionmangling and downloadmangling options. This tag can have a type (and in that case the data from remote-id is used) or can contain the base url and the file pattern. (this is what is currently implemented for our tests). So before going further, we would like some feedback from you on these approaches. What do you think about them? Which do you prefer? Do you think there’s a better approach or some steps can be changed in a more efficient way? Other examples: dev-perl/XML-Tidy: # We have to strip trailing letters in version and then apply cpan mangling rules XML-Tidy XML::Tidy sys-fs/dfc: # Download hosting sux and have download id in url http://projects.gw-computing.net/projects/dfc/files /attachments/download/[0-
Re: [gentoo-dev] RFC: euscan user dashboard
On 08/08/2012 21:53, Markos Chandras wrote: > Yes I would like to get email notifications every time there is a > version bump in the upstream repos. I also would like to receive an > email notification once a month listing all the packages that I > haven't bumped in portage. So let's say: for every scan performed by euscan send an email with all watched updated packages not already in portage. Just a list of the packages? (with a link to the euscan page? or would you prefer text-only emails?) Example: + a-category/package1-3.3.4-r0: [] + a-category/package2-0.1.1-r0: [] ... Settings for emails could be: - Activate / deactivate email notifications - Ignore unstable versions (alpha, beta, etc...) - Ignore unstable versions only if current version is stable Should we show removals in the email? Or only bumps? Would you like to receive infos about adds in portage or upstream only? Or a setting for that? Other suggestions? > I already use the rss feed. I just clicked on my name and then used > the rss url. Works quite well I must say. The only annoying thing is > that it also lists package removals. IMHO, this is not as useful as > version bumps. Great! So settings for rss feeds: - Show or not portage updates - Show or not upstream updates - Show or not removals - Show or not bumps - Ignore unstable versions (alpha, beta, etc...) - Ignore unstable versions only if current version is stable Settings are available in all feeds as url options, not just in the user custom feed (shows only watched stuff). Other suggestions? -- f. "Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live." (Martin Golding) signature.asc Description: OpenPGP digital signature
[gentoo-dev] RFC: euscan user dashboard
Hi! I'm Federico and I'm working for this year's Google Summer of Code on the euscan project with Corentin Chary. euscan is a tool to automatically scan upstream and find out if some packages in gentoo are outdated and should be bumped. We're working on a dashboard for maintainers where will be possible to receive a personalized report of packages. Currently on euscan (http://euscan.iksaif.net/) is possible to register and watch / unwatch packages. A simple dashboard is provided with a summary of the watched stuff (# of watched packages, # of outdated packages, etc...). We already received some suggestions and we implmented some of them, but we'd like to receive more detailed feedback about: - Mail newsletter: would you like to have it? which info would you like to receive specifically? A summary of the packages you're watching? When a scan is performed would you like an alert with all the new found version? Anything else? - RSS feed: would you like to have it? which info would you like? Only about new found versions of the packages you're watching? Or about portage updates too? - Settings/Preferences e.g.: Receive news only about stable versions, about unstable versions only if the version in gentoo is unstable, etc... Thank you for your help! -- f. "Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live." (Martin Golding) signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] euscan GSoC project - requesting feedback
On 27/06/2012 16:33, Michał Górny wrote: > Is the development version running somewhere already so we could play > with it a bit? For now is not available unfortunately :( We'll try to set it up! Thank you everybody for your feedback! The RSS feed is already implemented globally, we'll just tweak it to have a per user feed. I didn't considered the fact of separate stable and unstable but it's indeed a really cool feature. -- f. "Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live." (Martin Golding)
[gentoo-dev] euscan GSoC project - requesting feedback
Hi everybody! I'm working on a GSoC project for enhancing Euscan (http://euscan.iksaif.net/). Euscan allows to check if a given package/ebuild has new upstream versions or not. It uses different heuristic to scan upstream and grab new versions and related urls. Euscan has a web interface for surfing data but for now is only possible to see the scan results per package/category/herd/maintainers/overlay. We're working now on the possibility to register and provide a cool dashboard for maintainers, so that they can easily keep an eye on the upstream versions of the packages they maintain. The main question is: what would you like to have on this dashboard? Currently (in the development version) there's the possibility to login and watch/unwatch packages/categories/herds/... and see the watched stuff in the account dashboard. We're planning on implementing a weekly(?) custom newsletter based on the packages you're watching, which features would you like? The project repo for the GSoC is here: https://github.com/volpino/euscan Thanks! -- f. "Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live." (Martin Golding) () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments http://about.me/fox91 signature.asc Description: OpenPGP digital signature
