Ross Patterson <[EMAIL PROTECTED]> writes:
> Rob van der Heij <[EMAIL PROTECTED]> writes:
> >I am trying to get VNC working on Linux for S/390, and after
> >browsing the mailing list archives I still need some help.
>
> The echoed characters are
> gibberish, but commands typed into xterm do what they should
> (e.g. "write" sends intelligble messages to other users,
> "exit" closes the window, etc.).
I know replying to one's own mail is gauche (and probably disgusting in some Wired
JargonWatch list), but I've found what seems to be the major cause of trouble. IBM
System/390 boxes are big-endian, and of course Intel x86'es are little-endian. There
are at least three cases in the VNC server source where the attempt to be
variously-endian fails.
In vnc_unixsrc/Xvnc/programs/Xserver/hw/vnc, rfb.h defines macros for mixed-endian
byteswapping (Swap16IfLE() and Swap32IfLE()) but hard-codes the flag that determines
the server's endianness to "1" (meaning little-endian). I think there should be some
conditional definition, or at least it should be called out in the README etc.
In the same directory, init.c defines the PF_EQ() macro assuming that byteswapping
isn't necessary for 8-bit pixels, but in fact seems to be (although this doesn't make
any sense to me). Running vncserver with "-depth 32" produces a non-garbled display
(xterm contains readable characters, etc.). Either it's an endian-ness issue or
there's some side effect of the translation functions in translate.c that this
configuration requires and that rfbTranslateNone() (which is used with -depth 8)
doesn't supply.
Lastly, in the same directory, translate.c defines the BGR233 pixel format with a
constant 0 for rfbPixelFormat.bigEndian (meaning little-endian). If the evidence
above is correct that there is actually some endian effect for 8-bit pixels, this
needs to be conditional as well.
VNC isn't working perfectly on System/390 yet, but now it's basically usable. There
are still some pixel artifacts to clean up, and that may be the last of it.
Once all is well, would portability patches be welcome?
Ross Patterson
Computer Associates Int'l, Inc.
---------------------------------------------------------------------
To unsubscribe, send a message with the line: unsubscribe vnc-list
to [EMAIL PROTECTED]
See also: http://www.uk.research.att.com/vnc/intouch.html
---------------------------------------------------------------------
