Re: [gentoo-user] OT: Project Idea
On Tuesday 01 February 2005 11:35 pm, John Myers
[EMAIL PROTECTED] wrote:
On Tuesday 01 February 2005 16:03, Boyd Stephen Smith Jr. wrote:
[On in reply to any one message in particular, so not quoting anyone.]
Probably the best way to do this is to provide a drop-in replacement
for make w/ an optional patch to emerge/ebuild.
Probably, but in the shorter term, a patch would be more likely, as
rewriting make, making it bug-compatible, would be a project all its
own.
You don't have to make it bug compatible, just POSIX/SUS compliant. For
software using configure/auto* tools (Pretty much anything C/C++ for GNU,
and a lot of other devs use it, too), the makefiles generated are designed
to work with many different POSIX/SUS makes. In particular, it doesn't
have to be gmake.
Writing make is not trivial, but it might be easier than getting a patch
for a new project added into old projects like gcc and gmake.
Now that's not easy, but it'd probably be easier than asking the gmake
and gcc teams to call back into your daemon and/or making and
maintaining your own patches to these (and other) projects and having
users recompile them to support your stuff.
A patch to GCC isn't strictly neccessary, though. A wrapper could be
created to run gcc, or it could be done by make just the same as for
other processes that aren't progress-aware. I just thought that if it
was patched, it could tell what stage of compilation it was on
(preprocessing, compiling, assembling, etc.).
Also, How often does make actually have any major architectural changes?
I don't think a patch to make would be all that hard to maintain.
While the wrapper could tell something about what it's doing, based on the
command-line, it can't tell you how many times gcc (for example) will be
invoked in the future. A wrapper for make won't be able to do that
either, because you have to parse the makefile to get an accurate count.
I simply don't see a place a simple wrapper is going to give you the
information you need. You might be able to patch make (etc.) to give you
the correct information. That may be (probably is) easier that writing a
POPSIX/SUS make.
By scanning the filesystem and performing a few guesses, you could get an
estimate, but don't we already have a tool that does that?
AFAIK, gmake is fairly stable. I have a handful of ways it could be
improved, but there's only so much you really want make to do and gmake
follows the POSIX/SUS fairly well (plus, has a bunch of other features).
* Daemon should be light weight, and separate from the UI.
Umm, yeah.That was exactly my idea. The UI would be a completely
separate component, related only by a common communication protocol.
Same with the clients. The daemon just keeps track.
Yeah, I wasn't sure how much of this you had already covered. I know you
had a daemon process, I wasn't sure how much it was responsible for.
* UI could provide different looks:
- Progress bar tree (similar to a process tree)
- Progress bar stack (for purely/mostly sequential tasks)
- Nested progress bars (they'd fill up like eDonkey or GetRight's
progress bars; not strictly reading order)
Of course. The author of a viewer could make the progress bars any way
they want. As an example, I might even write a web-based viewer.
Sure, these are just ideas for the UI.
* Programs that report to the deamon could also act like UI agents; in
the case of make it could show a progress bar that's really the sum
progress of it and all it's child makes.
Maybe. This would be, in some ways, harder. I would go more for just
supressing console output and forking to the background
I wouldn't want to fork to the background. Heck, for e{merge,build} I
wouldn't want output supressed [but I would want their called make output
supressed :)].
I don't think either should be done by default, as most times it's easy to
append /dev/null 2 /var/log/background_errors.log to a command
line.
Also, here's an expansion of my concept: a client, bundled in the
distribution, to allow shell scripts (ebuilds, anyone?) to use the
system.
It would be nice if you had a small program that was called like:
prog-mon 'emerge sync' 'emerge -uD --newuse sys-apps/portage' 'emerge -uD
--newuse system' 'emerge -uD --newuse world'
It would run each of it's arguments as a command line, in sequence, and
display a progress bar. (Or hide and fork and be monitorable otherwise.)
Here's how I see it working:
Two parts:
1) a *client* daemon. This would be the process that actually
communicates with the master daemon. It would keep the connection to the
master open, and would also have a consistent PID (unlike a shell
script, which, AFAIK, may not). We don't want the master daemon to think
we mysteriously died and report an error. This client daemon would keep
track of just the one job's tasks.
What do you mean by a consistent PID. A shell script will not change
PIDs
Re: [gentoo-user] OT: Project Idea
On Wed, 2 Feb 2005 01:48:02 -0600, Boyd Stephen Smith Jr. wrote: While the wrapper could tell something about what it's doing, based on the command-line, it can't tell you how many times gcc (for example) will be invoked in the future. A wrapper for make won't be able to do that either, because you have to parse the makefile to get an accurate count. There was a project on the forums that did something like this. It provided a replacement make that ran make in a pretend mode to get the count, then ran it again. The gcc output was then redirected to a file and all the user saw was a progress bar. It was a nice idea let down by the simple flaw that it completely screwed up some compilations. For example, when emerging transcode, it would build the executables, but the make install stage didn't install them :( You have to be very careful about hacking into the build process, which is why I now prefer something that just makes a guesstimate, like eprogress or genlop --current. -- Neil Bothwick An example of hard water is ice. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
On Wednesday 02 February 2005 03:16 am, Neil Bothwick [EMAIL PROTECTED] wrote: On Wed, 2 Feb 2005 01:48:02 -0600, Boyd Stephen Smith Jr. wrote: While the wrapper could tell something about what it's doing, based on the command-line, it can't tell you how many times gcc (for example) will be invoked in the future. A wrapper for make won't be able to do that either, because you have to parse the makefile to get an accurate count. There was a project on the forums that did something like this. It provided a replacement make that ran make in a pretend mode to get the count, then ran it again. The gcc output was then redirected to a file and all the user saw was a progress bar. It was a nice idea let down by the simple flaw that it completely screwed up some compilations. For example, when emerging transcode, it would build the executables, but the make install stage didn't install them :( Very wierd. I suppose that approach is very good though. [I had completely forgot about make's pretend mode.] -- Boyd Stephen Smith Jr. [EMAIL PROTECTED] ICQ: 514984 YM/AIM: DaTwinkDaddy -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
Am 2. Feb 2005 um 06:35 Uhr schrieb John Myers: It would keep the connection to the master open, and would also have a consistent PID (unlike a shell script, which, AFAIK, may not). A shell script itself has a consistent PID. However, any command called within a shell script is assigned a new PID. Sebastian -- Sebastian Flothow [EMAIL PROTECTED] Because it reverses the logical flow of conversation. Why is top posting frowned upon? -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
On Wed, February 2, 2005 1:24 am, Sebastian Flothow said: Am 2. Feb 2005 um 06:35 Uhr schrieb John Myers: It would keep the connection to the master open, and would also have a consistent PID (unlike a shell script, which, AFAIK, may not). A shell script itself has a consistent PID. However, any command called within a shell script is assigned a new PID. That's not entirely true. Subshells are not executed in the same process as the script. f.ex.: If my script contains the line foo $(bar baz) quux the 'bar baz' part is executed in a separate shell from the 'foo whatever quux' part. Don't try to make a counterexample involving the $$ variable. It won't work as variables are substituted by the original shell. Try this instead: bash -c '$(echo $(echo $(echo $(echo $(echo $(read))' suspending it, looking in ps, and counting how many shells you now have running. (the bash -c part is so that you _can_ suspend it) -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
On Wednesday 02 February 2005 11:59 am, John Myers [EMAIL PROTECTED] wrote: On Wed, February 2, 2005 1:24 am, Sebastian Flothow said: Am 2. Feb 2005 um 06:35 Uhr schrieb John Myers: It would keep the connection to the master open, and would also have a consistent PID (unlike a shell script, which, AFAIK, may not). A shell script itself has a consistent PID. However, any command called within a shell script is assigned a new PID. That's not entirely true. Subshells are not executed in the same process as the script. Right, but a subshell *is* another command. It can be just as easily run from a prompt. It won't work as variables are substituted by the original shell. Try this instead: bash -c '$(echo $(echo $(echo $(echo $(echo $(read))' suspending it, looking in ps, and counting how many shells you now have running. (the bash -c part is so that you _can_ suspend it) Or, look at a deeply nested recursive make run by emerge in a bash inside su - from bash in an xterm in kde. :) init-kdeinit-xterm-bash-su-bash-emerge-ebuild-make-make-make-make-sh-gcc +as -- Boyd Stephen Smith Jr. [EMAIL PROTECTED] ICQ: 514984 YM/AIM: DaTwinkDaddy -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
On Wed, February 2, 2005 10:01 am, Boyd Stephen Smith Jr. said: On Wednesday 02 February 2005 11:59 am, John Myers [EMAIL PROTECTED] wrote: On Wed, February 2, 2005 1:24 am, Sebastian Flothow said: Am 2. Feb 2005 um 06:35 Uhr schrieb John Myers: It would keep the connection to the master open, and would also have a consistent PID (unlike a shell script, which, AFAIK, may not). A shell script itself has a consistent PID. However, any command called within a shell script is assigned a new PID. That's not entirely true. Subshells are not executed in the same process as the script. Right, but a subshell *is* another command. It can be just as easily run from a prompt. Yes, it is, but it is also part of the original line It won't work as variables are substituted by the original shell. Try this instead: bash -c '$(echo $(echo $(echo $(echo $(echo $(read))' suspending it, looking in ps, and counting how many shells you now have running. (the bash -c part is so that you _can_ suspend it) Or, look at a deeply nested recursive make run by emerge in a bash inside su - from bash in an xterm in kde. :) init-kdeinit-xterm-bash-su-bash-emerge-ebuild-make-make-make-make-sh-gcc +as I know that, but one might wish to be able to use the commands to manipulate the progress data inside a subshell. Thus, we cannot rely on PIDs alone. I can't think of such a situation off the top of my head, but it would be silly to ignore it. Also, one might even need to call the connect command from inside a subshell for some reason. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
On Wednesday 02 February 2005 12:52 pm, John Myers [EMAIL PROTECTED] wrote: On Wed, February 2, 2005 10:01 am, Boyd Stephen Smith Jr. said: On Wednesday 02 February 2005 11:59 am, John Myers [EMAIL PROTECTED] wrote: On Wed, February 2, 2005 1:24 am, Sebastian Flothow said: Am 2. Feb 2005 um 06:35 Uhr schrieb John Myers: It would keep the connection to the master open, and would also have a consistent PID (unlike a shell script, which, AFAIK, may not). A shell script itself has a consistent PID. However, any command called within a shell script is assigned a new PID. That's not entirely true. Subshells are not executed in the same process as the script. Right, but a subshell *is* another command. It can be just as easily run from a prompt. Yes, it is, but it is also part of the original line I know that, but one might wish to be able to use the commands to manipulate the progress data inside a subshell. Thus, we cannot rely on PIDs alone. I can't think of such a situation off the top of my head, but it would be silly to ignore it. Also, one might even need to call the connect command from inside a subshell for some reason. Eh, your project, your call. Just doesn't seem to be that useful to me. However, (unless I misunderstand IPC under linux, and I probably do) you'll have to send a PID to the daemon anyway, if you want to masquerade your PID, you are welcome to. -- Boyd Stephen Smith Jr. [EMAIL PROTECTED] ICQ: 514984 YM/AIM: DaTwinkDaddy -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
On Mon, 31 Jan 2005 22:31:51 -0800, John Myers wrote: eprogress - a general-purpose hierarchical progress reporting system It's a good idea, but you should choose a different name. there is already a script called eprogress floating around the forums. It is a quick and dirty, but useful nonetheless, script to give a rough indication of how far a build has progresses. Your idea sounds more comprehensive, so please continue but try to avoid confusion with the name. -- Neil Bothwick Dolly Parton-- silicone based life pgpGKwPO0WLXD.pgp Description: PGP signature
Re: [gentoo-user] OT: Project Idea
On Tue, 2005-02-01 at 15:02, James Colannino wrote: John Myers wrote: [...]I do, however, really wish that I could find out how far in this build I am.[...] I would absolutely LOVE that ;) Don't mean to be a party popper or anything, but maybe you can lift some code from gentoolkit's genlop However, there's a flaw if one may call it flaw in that this progress will not be available for new and has not been emerged programs. -- Ow Mun Heng Gentoo/Linux on DELL D600 1.4Ghz 98% Microsoft(tm) Free!! Neuromancer 16:31:27 up 7:55, 7 users, load average: 2.41, 2.06, 1.72 -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
John Myers ([EMAIL PROTECTED]) scribbled: [snip to nuts and bolts] eprogress - a general-purpose hierarchical progress reporting system my vision of the architecture has three components: 1) eprogress progress providers (clients?) (perhaps through some sort of libeprogressc). These are programs like emerge, make, gcc, etc. which have some sort of goal, and can report on their progress. They would need to be patched to provide the system with the progress information. I would advise against any development path that required patching many different projects. There would be a lot of resistance. This also seems to be passing off the crux of the problem (determining length of compile time) to the developers of make/gcc/ldd etc. See BUs, below. 2) eprogressd (one for each master task, i.e. if you had an emerge and some other make running at the same time, they would be kept separated). the eprogressd would run in the background and keep track of all the progress data. Why not a single daemon which spawns a thread for each new program, ala sshd (although iirc sshd forks)? 3) eprogress viewers which communicate with eprogressd to display a representation of the progress data. There could be any number of interchangeable viewers, some for console, some for X11. Some may be specialized to a particular task (such as a special one for emerges), but all would use the same protocol to talk to the eprogressd, which would be kept generic (there could (should?) be some sort of libeprogressviewer to help with this) gkrellm? I would suggest that the system should use UNIX domain sockets or TCP/IP sockets for the communication, especially TCP/IP for the viewer connection. It should not be necessary for the viewer to reside on the same machine as the daemon. Nor, for that matter, should it be necessary for individual tasks to be performed on the same machine. Good idea, but there is no need to suggest, it's your project :) Also, progress information does not have to be limited to a percentage (though that is required). The information could contain many things, like compiler command lines, warnings, errors, einfos etc. This is the line item requirement I would drive a truck through, if I were a customer/user. :) Please let me know what you think. This idea has been floated many times before, with little success. Not to rain on the idea. :) The problem is much more complicated than it initially looks. For insight, take a look at the section in linuxfromscratch regarding SBUs (static bash units). http://archive.linuxfromscratch.org/lfs-museum/4.0/LFS-BOOK-4.0-HTML/chapter02/aboutsbus.html Personally, I would do less planning and more writing, some of the best projects started off as a hack to scratch an itch. Keep it simple. A lot of the most useful programs out there are very small. It can always be rewritten later to include TCP/IP or threads. Good luck! Cooper. -- gentoo-user@gentoo.org mailing list
RE: [gentoo-user] OT: Project Idea
eprogress - a general-purpose hierarchical progress reporting system Oh, so you mean something like http://gentoo-wiki.com/TIP_Watch_emerge_progress but hacked into the compiling subsystem? -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
On Tuesday 01 February 2005 01:20, Neil Bothwick wrote: It's a good idea, but you should choose a different name. there is already a script called eprogress floating around the forums. Darn. I just picked it to go along with the whole 'e-prefix' naming scheme. But I will see what I can do. -- t3h 3l3ctr0n3rd [EMAIL PROTECTED] Supermarket Deli Clerk and Student Programmer pgpF7LQRey4cN.pgp Description: PGP signature
Re: [gentoo-user] OT: Project Idea
A clarification: I'm not talking about time estimation here. What I'm talking about is having each provider provide a minimum of two pieces of data: the number of subtasks it will run, and the number of subtasks which have been completed. On Tuesday 01 February 2005 02:47, Jason Cooper wrote: John Myers ([EMAIL PROTECTED]) scribbled: 1) eprogress progress providers (clients?) (perhaps through some sort of libeprogressc). These are programs like emerge, make, gcc, etc. which have some sort of goal, and can report on their progress. They would need to be patched to provide the system with the progress information. I would advise against any development path that required patching many different projects. There would be a lot of resistance. This also seems to be passing off the crux of the problem (determining length of compile time) to the developers of make/gcc/ldd etc. See BUs, below. I almost forgot my (partial) solution to that: For some of the programs (gcc especially), one can write a wrapper which calls GCC and collects its output, possibly supressing it from the console, and passes it up to the framework 2) eprogressd (one for each master task, i.e. if you had an emerge and some other make running at the same time, they would be kept separated). the eprogressd would run in the background and keep track of all the progress data. Why not a single daemon which spawns a thread for each new program, ala sshd (although iirc sshd forks)? I don't think I was clear here: the daemon would be started in the _user's_ name (including root, if root started it (perhaps it should drop that?)), once each time the _user_ invokes such a command. i.e. if a user said emerge -uvD world emerge would start, and then it would start a progress daemon. Child processes of emerge would use the same daemon. I don't want to make a system-wide daemon because most systems' time is not spent with the compiler (or whatever) running Or maybe you understood me and I don't understand you? 3) eprogress viewers which communicate with eprogressd to display a representation of the progress data. There could be any number of interchangeable viewers, some for console, some for X11. Some may be specialized to a particular task (such as a special one for emerges), but all would use the same protocol to talk to the eprogressd, which would be kept generic (there could (should?) be some sort of libeprogressviewer to help with this) gkrellm? a plugin to gkrellm could be one of the several viewers. I do, however, want to write at least console and full-blown graphical viewers I would suggest that the system should use UNIX domain sockets or TCP/IP sockets for the communication, especially TCP/IP for the viewer connection. It should not be necessary for the viewer to reside on the same machine as the daemon. Nor, for that matter, should it be necessary for individual tasks to be performed on the same machine. Good idea, but there is no need to suggest, it's your project :) I guess you're right... I mostly put it in there to see if it was stupid. Also, progress information does not have to be limited to a percentage (though that is required). The information could contain many things, like compiler command lines, warnings, errors, einfos etc. This is the line item requirement I would drive a truck through, if I were a customer/user. :) I think i did not properly articulate myself here either: What I meant was that in addition to the subtask count and subtask completion count, a client could send lots of other data, too. Or, again, perhaps you understand me, and I do not understand you. Please let me know what you think. This idea has been floated many times before, with little success. Darn. Not to rain on the idea. :) The problem is much more complicated than it initially looks. For insight, take a look at the section in linuxfromscratch regarding SBUs (static bash units). http://archive.linuxfromscratch.org/lfs-museum/4.0/LFS-BOOK-4.0-HTML/chapter02/aboutsbus.html I'm not looking to estimate the time left. I realize that that's impossible. Personally, I would do less planning and more writing, some of the best projects started off as a hack to scratch an itch. Keep it simple. A lot of the most useful programs out there are very small. It can always be rewritten later to include TCP/IP or threads. Yeah, I just wanted to make sure the idea wasn't stupid. Good luck! Thanks! I have lots of other ideas too, by the way, but I think I'll focus on this one for now. -- t3h 3l3ctr0n3rd [EMAIL PROTECTED] Supermarket Deli Clerk and Student Programmer pgpbysCNFfsv1.pgp Description: PGP signature
Re: [gentoo-user] OT: Project Idea
John Myers ([EMAIL PROTECTED]) scribbled: A clarification: I'm not talking about time estimation here. What I'm talking about is having each provider provide a minimum of two pieces of data: the number of subtasks it will run, and the number of subtasks which have been completed. So more like make saying I have 33 files to compile, 13 are done? On Tuesday 01 February 2005 02:47, Jason Cooper wrote: John Myers ([EMAIL PROTECTED]) scribbled: [snip clarification] 2) eprogressd (one for each master task, i.e. if you had an emerge and some other make running at the same time, they would be kept separated). the eprogressd would run in the background and keep track of all the progress data. Why not a single daemon which spawns a thread for each new program, ala sshd (although iirc sshd forks)? I don't think I was clear here: the daemon would be started in the _user's_ name (including root, if root started it (perhaps it should drop that?)), once each time the _user_ invokes such a command. i.e. if a user said emerge -uvD world emerge would start, and then it would start a progress daemon. Child processes of emerge would use the same daemon. I don't want to make a system-wide daemon because most systems' time is not spent with the compiler (or whatever) running Or maybe you understood me and I don't understand you? I understand what you're getting at now, it just took me a second or two. :) [snip gkrellm stuff] [snip 05:30 Cooper being a smartass] Also, progress information does not have to be limited to a percentage (though that is required). The information could contain many things, like compiler command lines, warnings, errors, einfos etc. This is the line item requirement I would drive a truck through, if I were a customer/user. :) I think i did not properly articulate myself here either: What I meant was that in addition to the subtask count and subtask completion count, a client could send lots of other data, too. Or, again, perhaps you understand me, and I do not understand you. Sorry, I spend a lot of time focusing in on customer vagueness and nailing it down. If you don't nail it down, you get 15 great 'ideas' that are 'easy' to add two weeks before the delivery date, and contractually, you would have to do it, if you didn't force them to restrict scope in the beginning. In short, I understood what you were saying, I was just highlighting an area needing further clarification, albeit colorfully. :) Please let me know what you think. This idea has been floated many times before, with little success. Darn. Well, the compile time prediction idea has had little success. Not to rain on the idea. :) The problem is much more complicated than it initially looks. For insight, take a look at the section in linuxfromscratch regarding SBUs (static bash units). http://archive.linuxfromscratch.org/lfs-museum/4.0/LFS-BOOK-4.0-HTML/chapter02/aboutsbus.html I'm not looking to estimate the time left. I realize that that's impossible. Not impossible. Just that the potential gains are not worth the work involved to implement. With the adoption of the 2.6.x kernels, it's not as critical for most users to know how long something will take. For me personally, I never watch the 'emerge -uDav world' process. I start it on 4 machines, get a cup of coffee, and proceed with normal computer use. It doesn't interfere with user interaction at all, so I don't care if it takes 2 hours or 2.5. Just let me know when it's finished. Personally, I would do less planning and more writing, some of the best projects started off as a hack to scratch an itch. Keep it simple. A lot of the most useful programs out there are very small. It can always be rewritten later to include TCP/IP or threads. Yeah, I just wanted to make sure the idea wasn't stupid. Now that I understand you're not trying to predict compile times, it seems much more feasible. However, I would most likely not use it simply because it doesn't solve a problem I have. Others opinions are certainly valid also. Is it worth pursuing? Well, if it solves a problem _you_ have, go for it. That's all that really matters. If you find it useful, most likely others will too. Cooper. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
On Tue, 1 Feb 2005 13:24:41 -0500, Jason Cooper wrote: I'm not talking about time estimation here. What I'm talking about is having each provider provide a minimum of two pieces of data: the number of subtasks it will run, and the number of subtasks which have been completed. So more like make saying I have 33 files to compile, 13 are done? that#s what the current eprogress script tries to do. It counts the number of .c and .cpp files in the work directory and compares them with the number of .o files, presenting the latter as a percentage of the former. Sometimes it fails miserably, like with glibc, but most of the time it gives a reasonable approximation of how far through the build progress you have progressed. -- Neil Bothwick HEALTH WARNING: The InterNet can be addictive and cause a lack of real social life. Use with care. Not to be taken more than four times daily. pgpe8h3q3NgE3.pgp Description: PGP signature
Re: [gentoo-user] OT: Project Idea
On Tuesday 01 February 2005 10:24, Jason Cooper wrote: John Myers ([EMAIL PROTECTED]) scribbled: A clarification: I'm not talking about time estimation here. What I'm talking about is having each provider provide a minimum of two pieces of data: the number of subtasks it will run, and the number of subtasks which have been completed. So more like make saying I have 33 files to compile, 13 are done? Yeah, except a whole hierarchy of those. On Tuesday 01 February 2005 02:47, Jason Cooper wrote: Also, progress information does not have to be limited to a percentage (though that is required). The information could contain many things, like compiler command lines, warnings, errors, einfos etc. This is the line item requirement I would drive a truck through, if I were a customer/user. :) I think i did not properly articulate myself here either: What I meant was that in addition to the subtask count and subtask completion count, a client could send lots of other data, too. Or, again, perhaps you understand me, and I do not understand you. Sorry, I spend a lot of time focusing in on customer vagueness and nailing it down. If you don't nail it down, you get 15 great 'ideas' that are 'easy' to add two weeks before the delivery date, and contractually, you would have to do it, if you didn't force them to restrict scope in the beginning. In short, I understood what you were saying, I was just highlighting an area needing further clarification, albeit colorfully. :) Ah, I get what you were saying with the truck thing. Not to rain on the idea. :) The problem is much more complicated than it initially looks. For insight, take a look at the section in linuxfromscratch regarding SBUs (static bash units). http://archive.linuxfromscratch.org/lfs-museum/4.0/LFS-BOOK-4.0-HTML/chapter02/aboutsbus.html I'm not looking to estimate the time left. I realize that that's impossible. Not impossible. Just that the potential gains are not worth the work involved to implement. I agree. With the adoption of the 2.6.x kernels, it's not as critical for most users to know how long something will take. For me personally, I never watch the 'emerge -uDav world' process. I start it on 4 machines, get a cup of coffee, and proceed with normal computer use. It doesn't interfere with user interaction at all, so I don't care if it takes 2 hours or 2.5. Just let me know when it's finished. I usually do the same, but I would rather see a progress indication than compiler command lines. (though I would like them to be available for me to see, hence the extra data fields) Personally, I would do less planning and more writing, some of the best projects started off as a hack to scratch an itch. Keep it simple. A lot of the most useful programs out there are very small. It can always be rewritten later to include TCP/IP or threads. Yeah, I just wanted to make sure the idea wasn't stupid. Now that I understand you're not trying to predict compile times, it seems much more feasible. However, I would most likely not use it simply because it doesn't solve a problem I have. Others opinions are certainly valid also. Is it worth pursuing? Well, if it solves a problem _you_ have, go for it. That's all that really matters. Fine then, don't use it! =) If you find it useful, most likely others will too. I hope so! Thanks for the input! Since I have gotten several voices in favor of it, I will embark upon this great and noble expedition to improve my life (and work on building a reputation), and will report back here (maybe gentoo-dev first) when I have something to show -- t3h 3l3ctr0n3rd [EMAIL PROTECTED] Supermarket Deli Clerk and Student Programmer pgpJmKfW2SBcp.pgp Description: PGP signature
Re: [gentoo-user] OT: Project Idea
John Myers wrote: On Tuesday 01 February 2005 01:20, Neil Bothwick wrote: It's a good idea, but you should choose a different name. there is already a script called eprogress floating around the forums. Darn. I just picked it to go along with the whole 'e-prefix' naming scheme. But I will see what I can do. Maybe you could expand on the already existing codebase of eprogress and then call it eprogress-enhanced ;) James -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
On Tue, 01 Feb 2005 15:29:13 -0800, James Colannino wrote: Maybe you could expand on the already existing codebase of eprogress and then call it eprogress-enhanced ;) There's plenty of room for expansion :) $ wc -l /usr/local/bin/eprogress 19 /usr/local/bin/eprogress -- Neil Bothwick Tribble math: * + * = *** -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
[On in reply to any one message in particular, so not quoting anyone.] Probably the best way to do this is to provide a drop-in replacement for make w/ an optional patch to emerge/ebuild. Make is really what decides how many files will need to be compiled. (For many projects.) Now that's not easy, but it'd probably be easier than asking the gmake and gcc teams to call back into your daemon and/or making and maintaining your own patches to these (and other) projects and having users recompile them to support your stuff. Along the way, you'd might be able to make recursive make's -jjobs option work a bit better. Overview: 1. Connect to daemon (if you can't just do your job and don't worry about progress reporting) 2. Report your pid; daemon uses this to build progress montior heirarchy. 3. Determine how many sub-jobs you have [ant, nant, make, scons, etc. already do something like this while they are building the dependency tree]; report to daemon. 4. Spawn jobs (squentially, in threads, as separate processes, whatever); if they are aware of the daemon they will be able to communicate with it independently. 5. As jobs finish; report to daemon. * Daemon should be light weight, and separate from the UI. * UI could provide different looks: - Progress bar tree (similar to a process tree) - Progress bar stack (for purely/mostly sequential tasks) - Nested progress bars (they'd fill up like eDonkey or GetRight's progress bars; not strictly reading order) * Programs that report to the deamon could also act like UI agents; in the case of make it could show a progress bar that's really the sum progress of it and all it's child makes. -- Boyd Stephen Smith Jr. [EMAIL PROTECTED] ICQ: 514984 YM/AIM: DaTwinkDaddy -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
On Tuesday 01 February 2005 16:03, Boyd Stephen Smith Jr. wrote: [On in reply to any one message in particular, so not quoting anyone.] Probably the best way to do this is to provide a drop-in replacement for make w/ an optional patch to emerge/ebuild. Probably, but in the shorter term, a patch would be more likely, as rewriting make, making it bug-compatible, would be a project all its own. Make is really what decides how many files will need to be compiled. (For many projects.) I know that. Now that's not easy, but it'd probably be easier than asking the gmake and gcc teams to call back into your daemon and/or making and maintaining your own patches to these (and other) projects and having users recompile them to support your stuff. A patch to GCC isn't strictly neccessary, though. A wrapper could be created to run gcc, or it could be done by make just the same as for other processes that aren't progress-aware. I just thought that if it was patched, it could tell what stage of compilation it was on (preprocessing, compiling, assembling, etc.). Also, How often does make actually have any major architectural changes? I don't think a patch to make would be all that hard to maintain. Overview: 1. Connect to daemon (if you can't just do your job and don't worry about progress reporting) 2. Report your pid; daemon uses this to build progress montior heirarchy. 3. Determine how many sub-jobs you have [ant, nant, make, scons, etc. already do something like this while they are building the dependency tree]; report to daemon. 4. Spawn jobs (squentially, in threads, as separate processes, whatever); if they are aware of the daemon they will be able to communicate with it independently. 5. As jobs finish; report to daemon. * Daemon should be light weight, and separate from the UI. Umm, yeah.That was exactly my idea. The UI would be a completely separate component, related only by a common communication protocol. Same with the clients. The daemon just keeps track. * UI could provide different looks: - Progress bar tree (similar to a process tree) - Progress bar stack (for purely/mostly sequential tasks) - Nested progress bars (they'd fill up like eDonkey or GetRight's progress bars; not strictly reading order) Of course. The author of a viewer could make the progress bars any way they want. As an example, I might even write a web-based viewer. * Programs that report to the deamon could also act like UI agents; in the case of make it could show a progress bar that's really the sum progress of it and all it's child makes. Maybe. This would be, in some ways, harder. I would go more for just supressing console output and forking to the background Also, here's an expansion of my concept: a client, bundled in the distribution, to allow shell scripts (ebuilds, anyone?) to use the system. Here's how I see it working: Two parts: 1) a *client* daemon. This would be the process that actually communicates with the master daemon. It would keep the connection to the master open, and would also have a consistent PID (unlike a shell script, which, AFAIK, may not). We don't want the master daemon to think we mysteriously died and report an error. This client daemon would keep track of just the one job's tasks. 2)the API. I see this as two parts as well: A binary to do the actual communication, and a shell script function library, so that it can change environment variables -- t3h 3l3ctr0n3rd [EMAIL PROTECTED] Supermarket Deli Clerk and Student Programmer pgpDqup6enPgU.pgp Description: PGP signature
[gentoo-user] OT: Project Idea
I am looking for a way to make a contribution to Gentoo and open source in general. So I'm thinking to myself, I'm a programmer, why don't I write something? So here I am. I, however, want to get opinions from other Gentoo users before I embark on this quest. I'm not asking for volunteers or contributions or anything, just comments on my idea. If everyone else thinks it's a stupid idea, I won't pursue it. Here it is: So I'm emerging some package, watching the endless stream of compiler commandlines scroll across my screen, when I realize, I really don't care about these command lines all that much. I do, however, really wish that I could find out how far in this build I am. So I do a little thinking, and this is what I came up with: eprogress - a general-purpose hierarchical progress reporting system my vision of the architecture has three components: 1) eprogress progress providers (clients?) (perhaps through some sort of libeprogressc). These are programs like emerge, make, gcc, etc. which have some sort of goal, and can report on their progress. They would need to be patched to provide the system with the progress information. 2) eprogressd (one for each master task, i.e. if you had an emerge and some other make running at the same time, they would be kept separated). the eprogressd would run in the background and keep track of all the progress data. 3) eprogress viewers which communicate with eprogressd to display a representation of the progress data. There could be any number of interchangeable viewers, some for console, some for X11. Some may be specialized to a particular task (such as a special one for emerges), but all would use the same protocol to talk to the eprogressd, which would be kept generic (there could (should?) be some sort of libeprogressviewer to help with this) I would suggest that the system should use UNIX domain sockets or TCP/IP sockets for the communication, especially TCP/IP for the viewer connection. It should not be necessary for the viewer to reside on the same machine as the daemon. Nor, for that matter, should it be necessary for individual tasks to be performed on the same machine. Also, progress information does not have to be limited to a percentage (though that is required). The information could contain many things, like compiler command lines, warnings, errors, einfos etc. Please let me know what you think. -- t3h 3l3ctr0n3rd (John Myers) [EMAIL PROTECTED] Supermarket Deli Clerk and Student Programmer pgp214G7YPSyg.pgp Description: PGP signature
Re: [gentoo-user] OT: Project Idea
John Myers wrote: [...]I do, however, really wish that I could find out how far in this build I am.[...] I would absolutely LOVE that ;) James -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
James Colannino wrote: John Myers wrote: [...]I do, however, really wish that I could find out how far in this build I am.[...] I would absolutely LOVE that ;) James -- gentoo-user@gentoo.org mailing list DITTO! If I had the skill, I would love to help, but maybe I could help with a website or something...? That project would rock! Ian begin:vcard fn:Ian K n:K;Ian email;internet:[EMAIL PROTECTED] note;quoted-printable:Pentium 3=0D=0A= 500mHz=0D=0A= 256MB RAM=0D=0A= 80.0GB HDD=0D=0A= ATI Radeon 7000 Evil Wizard 64MB=0D=0A= Computer name: PentaQuad=0D=0A= x-mozilla-html:TRUE version:2.1 end:vcard -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] OT: Project Idea
I suggest you join the gentoo-dev list and float that... On Mon, 2005-01-31 at 22:31 -0800, John Myers wrote: I am looking for a way to make a contribution to Gentoo and open source in general. So I'm thinking to myself, I'm a programmer, why don't I write something? So here I am. I, however, want to get opinions from other Gentoo users before I embark on this quest. I'm not asking for volunteers or contributions or anything, just comments on my idea. If everyone else thinks it's a stupid idea, I won't pursue it. Here it is: So I'm emerging some package, watching the endless stream of compiler commandlines scroll across my screen, when I realize, I really don't care about these command lines all that much. I do, however, really wish that I could find out how far in this build I am. So I do a little thinking, and this is what I came up with: eprogress - a general-purpose hierarchical progress reporting system my vision of the architecture has three components: 1) eprogress progress providers (clients?) (perhaps through some sort of libeprogressc). These are programs like emerge, make, gcc, etc. which have some sort of goal, and can report on their progress. They would need to be patched to provide the system with the progress information. 2) eprogressd (one for each master task, i.e. if you had an emerge and some other make running at the same time, they would be kept separated). the eprogressd would run in the background and keep track of all the progress data. 3) eprogress viewers which communicate with eprogressd to display a representation of the progress data. There could be any number of interchangeable viewers, some for console, some for X11. Some may be specialized to a particular task (such as a special one for emerges), but all would use the same protocol to talk to the eprogressd, which would be kept generic (there could (should?) be some sort of libeprogressviewer to help with this) I would suggest that the system should use UNIX domain sockets or TCP/IP sockets for the communication, especially TCP/IP for the viewer connection. It should not be necessary for the viewer to reside on the same machine as the daemon. Nor, for that matter, should it be necessary for individual tasks to be performed on the same machine. Also, progress information does not have to be limited to a percentage (though that is required). The information could contain many things, like compiler command lines, warnings, errors, einfos etc. Please let me know what you think. -- Nick Rout [EMAIL PROTECTED] -- gentoo-user@gentoo.org mailing list
