Re: arch-dependent searches and downloads
Drat. I forgot to move this to debian-www. On Fri, Nov 24, 2000 at 10:30:02AM -0500, Jay Treacy wrote: Web discussions are held on debian-www@lists.debian.org It would be a good idea if you notified people before you started such work so you don't waste your time. We have not implemented this in the past for some technical reasons. Now that packages.d.o is not being mirrored we have a lot more freedom to do things differently. It is possible you happen to have done things in an appropriate way. I haven't looked at your changes, and won't be able to until next week. I can tell you that the way of accessing package information will be changed as apt-cache will be used (no need for a local ftp archive then. In fact, packages.d.o does not have a copy of the unstable archive). Also, we were contemplating switching to dynamic pages. Since the content is updated daily static pages don't make a lot of sense. We could simply pull information about a package from apt-cache (it does simple package searching too, btw). -- James (Jay) Treacy [EMAIL PROTECTED] Martin Michlmayr [EMAIL PROTECTED] wrote: Over the last couple of days, I re-wrote the htmlscripts for packages.do.o and the download.pl script in order to support more than i386. Currently, download only supports i386 and the package search will not show packages which are not available for i386. For the latter problem, the Packages files of all architectures have to be merged. For the former problem, the pages.pl and download.pl scripts had to be re-written. Please copy: 1. master:~tbm/crontab to /org/packages.debian.org 2. master:~tbm/download.pl to /org/cgi.debian.org/cgi-bin/ 3. master:~tbm/htmlscripts to /org/packages.debian.org (the whole directory; or make a diff and apply it: ok, you can use ~tbm/htmlscripts-diff). 4. run crontab I have tested the scripts quite a bit and think they work. But please look at the changes I made yourself... (/me mumbles something about using CVS for those scripts would be nice). IMHO there is only one problem left: when you can choose the architecture you want to download, all archs are displayed even if the package is not available for that arch (it's not really a problem, there won't be an error just a message saying that the package is not available). It would be nicer just to offer those archs for which the package is actually available. I will implement that later. Let me know what you think. --=20 Martin Michlmayr [EMAIL PROTECTED] -- James (Jay) Treacy [EMAIL PROTECTED]
Re: arch-dependent searches and downloads
* James A. Treacy [EMAIL PROTECTED] [20001124 10:35]: Also, we were contemplating switching to dynamic pages. I don't know if this is a good idea performance wise. Since the content is updated daily static pages don't make a lot of sense. Why? Disk space doesn't matter, but performance does. -- Martin Michlmayr [EMAIL PROTECTED]
Re: arch-dependent searches and downloads
On Fri, Nov 24, 2000 at 05:03:10PM +, Martin Michlmayr wrote: * James A. Treacy [EMAIL PROTECTED] [20001124 10:35]: Also, we were contemplating switching to dynamic pages. I don't know if this is a good idea performance wise. There is no question that static pages are more efficient. The question is whether we can provide dynamic pages with the resources we have. Frankly, I don't have a lot of experience with dynamic pages, but I suspect it shouldn't be a problem. Note that we are only contemplating the use. No decision has been made. Since the content is updated daily static pages don't make a lot of sense. Why? Disk space doesn't matter, but performance does. Current disk space for packages.d.o: 75MB multiply by 10 ports: 750MB multiply by 1.5 for the times the dist is frozen: over 1.1GB We should be able to pack the information needed into about 1/10 that size in a db. Note that this assumes a seperate page for each port. Whether a seperate page, or one page is better is something that needs to be disussed. For a single page: - you save a lot of space. For individual pages: - Much simpler pages. The user isn't overloaded with info on 10 ports. - There can be significant version skew between ports. Different versions can have different dependencies, descriptions, even different licenses! The first point for using individual pages is something we may be able to work around. The second one is much more serious. -- James (Jay) Treacy [EMAIL PROTECTED]
Re: arch-dependent searches and downloads
On Fri, Nov 24, 2000 at 12:33:21PM -0500, James A. Treacy wrote: Also, we were contemplating switching to dynamic pages. I don't know if this is a good idea performance wise. There is no question that static pages are more efficient. The question is whether we can provide dynamic pages with the resources we have. Frankly, I don't have a lot of experience with dynamic pages, but I suspect it shouldn't be a problem. Note that we are only contemplating the use. No decision has been made. AFAICT master.d.o in its current state wouldn't be able to handle another nice CGI, or a PHP script accessing a PostgreSQL database, or anything like that. Admittedly future conversion of BTS to a real database will probably stop the abusive bots from DoSing machine every now and then, but there are still X annoyingly resource-expensive scripts that run on master almost all the time. We could have cool dynamic stuff using only CGIs if we had a non-US server carrying both archives, I think some Perl code could find out appropriate versions for each architecture. But we don't have such a server. -- Digital Electronic Being Intended for Assassination and Nullification
Re: arch-dependent searches and downloads
* Josip Rodin [EMAIL PROTECTED] [20001124 19:48]: We could have cool dynamic stuff using only CGIs if we had a non-US server carrying both archives, But we don't have such a server. Then let's find a server. Is there any official person or way to approach a sponsor, or can I simply go ahead and ask a company (I know a company which _might_ be willing to donate a machine, but I haven't talked to them for a quite a while). I think some Perl code could find out appropriate versions for each architecture. That's easy; the scripts in my home on master do that already. -- Martin Michlmayr [EMAIL PROTECTED]
Re: arch-dependent searches and downloads
On Fri, Nov 24, 2000 at 07:26:55PM +, Martin Michlmayr wrote: We could have cool dynamic stuff using only CGIs if we had a non-US server carrying both archives, But we don't have such a server. Then let's find a server. Is there any official person or way to approach a sponsor, or can I simply go ahead and ask a company (I know a company which _might_ be willing to donate a machine, but I haven't talked to them for a quite a while). Ask on -private, I guess... admin people (who are going to be responsible for it) will read it, too. I think some Perl code could find out appropriate versions for each architecture. That's easy; the scripts in my home on master do that already. Then why did you say: IMHO there is only one problem left: when you can choose the architecture you want to download, all archs are displayed even if the package is not available for that arch (it's not really a problem, there won't be an error just a message saying that the package is not available). It would be nicer just to offer those archs for which the package is actually available. I will implement that later. That's implemented, then? -- Digital Electronic Being Intended for Assassination and Nullification
Re: arch-dependent searches and downloads
* Josip Rodin [EMAIL PROTECTED] [20001124 20:40]: I think some Perl code could find out appropriate versions for each architecture. That's easy; the scripts in my home on master do that already. Then why did you say: My script looks at the Packages.arch file to get the proper version. You just take something like: Filename: dists/woody/main/binary-i386/admin/acct_6.3.5-21.deb ^ IMHO there is only one problem left: when you can choose the architecture you want to download, all archs are displayed even if the That's implemented, then? Currently, if the package is not available for the architecture you want, nothing is found in the packages file and 'sorry...' displayed. What the script should do is to look if the package is available BEFORE providing the choice of architectures. At the moment, the pages list offers all architectures for the download -- it should only list those for which the package is really available. -- Martin Michlmayr [EMAIL PROTECTED]
