Re: Reengineering X applications
On Tue, 2004-05-11 at 16:44, Gian Filippo Pinzari wrote: > Jim Gettys wrote: > > Lots of useful information can be found in: > > > > http://keithp.com/~keithp/talks/usenix2003/ > > >> If we do everything that should be done, we can eliminate > >> about 90% of the round trips, ultimately. > > Hi Jim, did you have the chance of looking at NX since > the last time we discussed this in the X server ML at > fd.o? That document, that I have seen cited again and > again, says that SSH is enough to ensure that X over low > bandwidth links will work acceptably. Once removed the > roundtrips, of course. The fact you can run X applica- > tions without the round trips today (as I explained > in my previous post) and the result is still -slow X > applications- should be a proof that X over the Internet > needs something more. > > In that docement you say that X doesn't need a specia- > lized proxy system and a dedicated infrastructure. I > not only strongly disagree on that, but I have also > showed with tangible facts (like an OSS software that > everybody can use) that a specialized proxy system and > a dedicated infrastructure can do much better than ZLIB > compression. For example it can disconnect the session > from the display and reconnect it at later time, as > we'll do soon in NX. I don't think the document said that. Fixing in a proxy things that can/should be fixed elsewhere is a bandaid. Bandaids are useful, however; that doesn't mean we shouldn't fix the X and/or how the X protocol is being used. > > In that document you seem to imply that because LBX was > not as good as it was supposed to be, such a proxy layer > can't be written. I think that you should reconsider > that stance. The first step could be looking at the facts. > You told that you didn't ever have a chance to try NX. > Do that, please. I think it could really help people > in the X world to look at X-Window from a different > perspective. You are a X icon and many people seem to > trust you more than they trust data ;-). No, the point of the document (in part) was to put a stake in the heart of LBX; that much less code to maintain. The paper succeeds at that (showing that LBX is never better than SSH). I'm perfectly happy if other proxies do better; we just want to kill LBX off, once and for all... And get data on how/where to work on improving X itself. - Jim -- Jim Gettys <[EMAIL PROTECTED]> HP Labs, Cambridge Research Laboratory ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: Reengineering X applications
Lots of useful information can be found in:
http://keithp.com/~keithp/talks/usenix2003/
On Tue, 2004-05-11 at 02:09, Dr Andrew C Aitchison wrote:
> On Tue, 11 May 2004, Suresh wrote:
>
> > Hi,
> >
> > Has anyone come across X applications re-engineered for low bandwidth networks,
> > This in the context of thin clients.
>
> Low bandwidth or high latency ?
>
> Keith Packard and Jim Gettys have done some work on X clients
> as well as their work on low bandwidth X servers.
> If my memory is reliable, Owen Taylor did some work optimizing
> clients, possibly in the context of Motif.
>
> The low bandwidth case is essentially solved:
> avoid using lots of images
> ssh compression can help a lot
SSH compression ("ssh -C") makes a factor of 300(!)
difference in bandwidth used in a gnome session startup,
(a pretty vanilla set of applications).
> the LBX extension might be worth trying, too
> with this combination, a fast modem connection (say 33Kbps)
> isn't painful in terms of bandwidth.
LBX never does better than "ssh -C". Don't bother with LBX;
you need the security ssh provides these days anyway.
>
> What does kill X performance, even on cable modems (say 512Kbps)
> is latency. Especially at start up many X libraries make lots of
> round trip requests in serial.
> I don't remember the details, but you want to try to batch
> these requests, so that you get lots of answers in each round trip.
> (A way of making them in parallel would be nice too :).
Client side fonts have helped (eliminated more than 25%
of round trips).
Work in the toolkits is also helping. GTK+ now batches
intern atom, for example. Some other egregious bugs in
toolkits have been/are being fixed as a result of
the data.
There are other things we can and should do to help
things further.
If we do everything that should be done, we can eliminate
about 90% of the round trips, ultimately. But the low
hanging fruit is about 50% of the round trips we had
a year or so ago; the remaining will take some more
serious work, as outlined in the paper.
- Jim
--
Jim Gettys <[EMAIL PROTECTED]>
HP Labs, Cambridge Research Laboratory
___
Devel mailing list
[EMAIL PROTECTED]
http://XFree86.Org/mailman/listinfo/devel
X Window System developer's meeting.
Reminder: We have scheduled a three day X Developer's meeting from Wednesday, April 28, through Friday, April 30, 2004, at the Cambridge Research Laboratory (CRL) in Cambridge, Massachusetts. This meeting is intended to cover a wide variety of topics about X Window System technologies, and the as yet unmet needs of the technologies that depend on the X Window System. As a result, this meeting is intended for people working across the entire X Window System stack, including toolkits and window managers and desktop infrastructure. This meeting is not, however, intended to cover general X application development topics or concepts. There will be a number of people giving presentations. These presentations will likely consume no more than half of the overall meeting time so the remaining time can be spent with informal discussions, hacking, breakout sessions, etc. If you wish to volunteer to give a presentation, please send e-mail to [EMAIL PROTECTED] In addition, if you have specific topics you'd like to see discussed in a presentation or during informal sessions, please send e-mail with your suggestions to that address as well. Current presentations for the X Developer's Meeting include: * Keith Packard and Jim Gettys: "The (Re)Architecture of the X Window System" * Looking Glass - Sun Microsystems * Dave Reed & Dave Smith: "Croquet: Integrating X with 3D Immersive Environments" * Eamon Walsh: "Extensible Security for X: Motivation and Design" * Owen Taylor: The Widget Toolkit Developer's Perspective on X Current discussions and/or hacking sessions include: * The big picture of X and a plan to get the necessary pieces done (Stuart Anderson) * Integrating SSH or SSL+zlib into X (Jim McQuillan) If you would like to attend, please send mail to [EMAIL PROTECTED] Having the names of attendees in advance eases building security, as we share the building with other companies. An audio feed will be made available for those attending remotely or unable to attend due to limitations on space. The meeting venue provides networking support with high speed internet access. More information will be at: http://freedesktop.org/Software/XDevConf -- Jim Gettys <[EMAIL PROTECTED]> HP Labs, Cambridge Research Laboratory ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
X Window System developer's meeting.
We have scheduled a three day X Developer's meeting from Wednesday, April 28, through Friday, April 30, 2004, at the Cambridge Research Laboratory (CRL) in Cambridge, Massachusetts. This meeting is intended to cover a wide variety of topics about X Window System technologies, and the as yet unmet needs of the technologies that depend on the X Window System. As a result, this meeting is intended for people working across the entire X Window System stack, including toolkits and window managers and desktop infrastructure. This meeting is not, however, intended to cover general X application development topics or concepts. There will be a number of people giving presentations. These presentations will likely consume no more than half of the overall meeting time so the remaining time can be spent with informal discussions, hacking, breakout sessions, etc. If you wish to volunteer to give a presentation, please send e-mail to [EMAIL PROTECTED] In addition, if you have specific topics you'd like to see discussed in a presentation or during informal sessions, please send e-mail with your suggestions to that address as well. Current presentations for the X Developer's Meeting include: * Keith Packard and Jim Gettys: "The (Re)Architecture of the X Window System" * Looking Glass - Sun Microsystems * Dave Reed & Dave Smith: "Croquet: Integrating X with 3D Immersive Environments" Due to space limitations, attendance will be limited to no more than 40 people. If you would like to attend, please send mail to [EMAIL PROTECTED] Having the names of attendees in advance eases building security, as we share the building with other companies. An audio feed will be made available for those attending remotely or unable to attend due to limitations on space. The meeting venue provides networking support with high speed access outside the facility. The hotels recommended near CRL are The Kendall Hotel, the Cambridge Marriott, and the Residence Inn. All three of these are a short walk from CRL and easily accessible via public transportation from the Kendall Square subway stop next to MIT. We have not negotiated any special rates with these hotels, so you'll need to shop around for the best rates, as they vary. A longer list of hotels is available at http://www.ai.mit.edu/visiting/hotels/hotels.shtml Directions to the lab can be found at: http://www.crl.hpl.hp.com/who/lab/directions.htm -- Jim Gettys <[EMAIL PROTECTED]> HP Labs, Cambridge Research Laboratory ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: memory leak
There is a new utility called "xrestop" available in the freedesktop.org CVS that is very useful in identifying clients that hog server memory. This utility is patterned somewhat after the "top" utility; written by Mathew Allum, using Mark's fine Xres extension. - Jim On Tue, 2004-01-06 at 19:50, Mark Vojkovich wrote: > On Tue, 6 Jan 2004, Petter Reinholdtsen wrote: > > > [Mark Vojkovich] > > > When I see these sort of requests I usually feel pretty comfortable > > > dismissing them as application resource leaks. > > > > Is it possible for dead applications to "use" resources in the X > > server? I mean, if a web browser allocate several resources in the X > > server (shared memory, pixmaps, fonts, I do not know what types of > > resources are available for allication), and they die, is all these > > resources released in the server? > >Server-side resources are released when the connection between > the server and client is severed. Only apps that stick around an > leak cause problems. > >You can check the resource usage with: > > http://www.xfree86.org/~mvojkovi/restest.c > >Be careful running it though. It grabs the server and dumps > data to stdout, which means there's a deadlock situation if you > run it from within X without redirecting the output. So either > run it remotely or redirect to a file "restest >& resout". > > > Mark. > > ___ > Devel mailing list > [EMAIL PROTECTED] > http://XFree86.Org/mailman/listinfo/devel -- Jim Gettys <[EMAIL PROTECTED]> HP Labs, Cambridge Research Laboratory ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: Wrong order of a timeouts processing in WaitForSomething.
I use both USB and PS/2. I did play with the event interface a few hours last year (before my latest encounter with a surgeon), and convinced myself it seemed to be working correctly with USB. I think there are patches (that may or may not have been merged into current 2.4, I don't remember) to add the legacy support. So I'd just open the device and see if PS/2 mice work at this date. A lot of "goodies" developed initially in the development stream of Linux get backported to the stable series. So it is entirely possible everything is already there in current Linux distributions. But I've not had time to check myself... I don't think it is a single event queue: there is an event queue/device, IIRC. I think I initially argued for a single queue, but the flexibility of having different protection on a per device basis made that argument specious in the design discussion, and I acknowledged this was a very bad idea... So you still have to do a select/poll, read all pending input events on all different devices, and merge the events in the server into a single stream in time stamp order. At most a page or two of code, -I believe (to do the merge). But all the different event types go through a single (narrow) interface. Multi-coordinate devices generate multiple events at once (one event/coordinate). - Jim On Wed, 2003-09-24 at 19:35, Juliusz Chroboczek wrote: > JG> In fact, the event interface exists in current 2.4; not > JG> just 2.[45]. Look in /dev/input/event*. > > Some of us still use PS/2 mice... > > JG> Note that this interface is not just for mice, but is generalized > JG> to be usable by pretty arbitrary input devices, > > Yes; it looks like all input devices go through the single event > interface in 2.6. That would seem to guarantee correct ordering of > events from different devices. > > Juliusz > ___ > Devel mailing list > [EMAIL PROTECTED] > http://XFree86.Org/mailman/listinfo/devel -- Jim Gettys <[EMAIL PROTECTED]> HP Labs, Cambridge Research Laboratory ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: Wrong order of a timeouts processing in WaitForSomething.
Yes. In part, the /dev/event interface provides timestamps because I agitated for them a couple years ago, between visits to my surgeons :-(. In fact, the event interface exists in current 2.4; not just 2.[45]. Look in /dev/input/event*. Note that this interface is not just for mice, but is generalized to be usable by pretty arbitrary input devices, and is part of the rewrite of the input system in Linux. - Jim On Wed, 2003-09-24 at 13:21, Juliusz Chroboczek wrote: > Thanks for the explanation, Keith. > > KP> 2)Timestamp events accurately > > [...] > > > KP> Better yet would be to have the kernel timestamp events with a > KP> high-precision timer > > Linux 2.[56] does that on the new /dev/event interface (/dev/psaux > would appear to be deprecated). Is it worth using? > > Juliusz > > > ___ > Devel mailing list > [EMAIL PROTECTED] > http://XFree86.Org/mailman/listinfo/devel -- Jim Gettys <[EMAIL PROTECTED]> HP Labs, Cambridge Research Laboratory ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: Exporting sched_yield to the drivers
Most operating schedulers are much happier to give you the CPU again if you don't monopolize the CPU, and let the other processes get the CPU regularly. Generally, they boost the priority of processes that just use a short amount of CPU, and then give it back. This is typical "interactive" behavior of processes. Having the X server yield if it is busy is a good thing to do. Most workstation based X servers have had a shared queue/pipe with the driver, fill up that queue, and arrange in the driver at interrupt level to set up the next set of commands or image transfers (to keep the hardware busy), and then the OS would typically schedule the X server process again (which would refill the command buffers). In many operating systems, there were ways to tell the operating system in the driver that the process should be woken up. Having the X server busy wait in user mode for any significant time is a bad idea... Note that this equation has been changing in Linux greatly recently: until very recently Linux has had a sucky scheduler and has only recently started to use an O(1) scheduler like most BSD Unix systems have had for aeons (20 years). But also note that Linux has been working very hard the last couple years in general on removing latency. - Jim On Tue, 2003-09-23 at 06:17, Egbert Eich wrote: > Mark Vojkovich writes: > > Can we export to the drivers some function that yields the CPU? > > Currently alot of drivers burn the CPU waiting for fifos, etc... > > usleep(0) is not good for this because it's jiffy based and usually > > never returns in less than 10 msec which has the effect of making > > interactivity worse instead of better. I'm not sure which platforms > > don't export sched_yield() and which will need alternative > > implementations. > > > > I've experimented with sched_yield in drivers when waiting for retrace > on video playback. It turned out that when the system was otherwise > idle the video performance was about the same. However when another > application was using its full time slice the video frame rate would > drop far below the screen refresh rate. > > I've measured the latencies and it turned out that the time elapsed > before the process would be rescheduled was considerably higher than > the average time the process was hanging in the wait loop when the > system was otherwise busy. > Shorter (or variable) time slices may help here, but a better solution > would be to use an event (ie. interrupt) driven rescheduling. > This would require a small kernel support module for each driver, > though. > > Egbert. > ___ > Devel mailing list > [EMAIL PROTECTED] > http://XFree86.Org/mailman/listinfo/devel -- Jim Gettys <[EMAIL PROTECTED]> HP Labs, Cambridge Research Laboratory ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
Re: Audio in X11
An audio server need not be designed to add latency (beyond that of the network itself, of course). With current networks, this is very small, down to a few samples. Existence proof is the AF audio server we did 10 years ago, in which the server design itself did not enforce any latency: if data arrived at the AF server before the sample had been played (and the hardware permitted), it performed cut-through and updated the samples immediately. - Jim On Wed, 2003-09-10 at 17:54, Ross Vandegrift wrote: > On Wed, Sep 10, 2003 at 11:09:54AM -0400, Jim Gettys wrote: > > The other promising work besides MAS is an audio server project > > called "Jack". > > > > It is not clear it currently provides network transparency, but it > > does boast low latency (required for telephony, teleconferencing and > > gaming applications). > > No, jack is intended for apps with much stricter performance > requirements - low latency, sample synchronization, and realtime > transport. These are pretty critical for pro audio work - recording, > production, soundtracking, overdubs, etc. > > It's very doubtful it will ever work over conventional networks - timing > is just too critical to jack. > > Now, a specially designed network with ADAT synchronization could work, > but I doubt anyone would want to port X11 to such a transport... ::-) -- Jim Gettys <[EMAIL PROTECTED]> HP Labs, Cambridge Research Laboratory ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
[Fwd: Re: Fwd: Vblank support in kernel and X server]
Here's Alan Cox's mail about it. In 2.4.20-ac1. - Jim -- Jim Gettys <[EMAIL PROTECTED]> HP Labs, Cambridge Research Laboratory --- Begin Message --- On Sul, 2003-07-27 at 17:26, Soeren Sandmann wrote: > Alan Cox <[EMAIL PROTECTED]> writes: > > > Soeren - do you mind if I hack up the code you sent or do you want me to > > give you a list of stuff that I think wants doing ? > > I definitely don't mind if you want to do the work for me :-). Some test code in the 2.4.22-ac1 tree now. I've switched the logic around a little and also set it up so that you can open then bind to the card you want and multiple opens selecting multiple cards work. I need to fix multiple opens of same card yet. Its not currently keeping timestamps as I wasn't clear if those were also needed. --- End Message ---
Re: Audio in X11
The other promising work besides MAS is an audio server project called "Jack". It is not clear it currently provides network transparency, but it does boast low latency (required for telephony, teleconferencing and gaming applications). The other issue is getting good synchronization with the X server. While the X server has had the XSync extension for a long time, the operating system "hooks" to allow X to synchronize with external events (e.g. vertical sync, sample clock of audio streams, etc) have been absent in open source systems. XSync was developed in the days of engineering workstations 10 years ago, and was debugged with such kernel support. Recently Alan Cox has done some work for Linux support of vertical retrace (and potentially audio sources) to provide the needed kernel hooks for the X server. So a small project for someone with time is now to hook up XSync with that kernel support and/or implement similar support for other open source systems. - Jim On Wed, 2003-09-10 at 10:50, Alan Coopersmith wrote: > There have been several attempts. The latest one, currently sponsored > by X.org, is MAS - http://www.mediaapplicationserver.net/ > > -Alan Coopersmith- [EMAIL PROTECTED] > Sun Microsystems, Inc.- Sun Software Group > User Experience Engineering: G11N: X Window System > > Fred Heitkamp wrote: > > I was wondering. Was there ever an effort to make a > > network independent audio extension for X11? (forgive > > my terminology if it's wrong.) For example, if I am > > logged on from a remote terminal and want to play an > > MP3 from the distant machine on the remote terminal, > > is this possible? Sorry if this is a FAQ, but I didn't > > see one while googling. > > > > Fred > > > > Error Loading Explorer.exe > > You must reinstall Windows. > > > > ___ > > Devel mailing list > > [EMAIL PROTECTED] > > http://XFree86.Org/mailman/listinfo/devel > > > ___ > Devel mailing list > [EMAIL PROTECTED] > http://XFree86.Org/mailman/listinfo/devel -- Jim Gettys <[EMAIL PROTECTED]> HP Labs, Cambridge Research Laboratory ___ Devel mailing list [EMAIL PROTECTED] http://XFree86.Org/mailman/listinfo/devel
