Jumbo frames are the use of larger than normal MTU (Maximum Transmission
Unit) settings on gigabit Ethernet links.  The traditional limit for
Ethernet frames is 1500 bytes, which was fine for 10 and 100 megabit
Ethernet links.  With gigabit, however, since you lose a certain minimum
amount of bandwidth to signaling overhead (preamble, postamble, header
info, etc.) and that the Ethernet card has to do a certain minimum
processing for each Ethernet frame it receives, a huge amount of CPU
overhead can be spent on trying to fill a gigabit pipe.  The other
problem is that if the host(s) are sending/receiving data larger than
1500 bytes, the data packet has to be fragmented into multiple, smaller
packets, which then have to be reassembled on the far side.  Since this
all has to be done on the host CPU rather than the Ethernet card, it
increases both bus overhead and CPU time.

With jumbo frames, you use a >1500 byte MTU - the exact amount varies by
implementation, but they're generally in the 9000-9200 byte range.
That's a 6x improvement in the amount of data per ethernet frame, plus
there's less reassembly.  Unfortunately, Sun never really embraced it as
a technology, so unless you're running one of a couple of third-party
gigabit cards, I think you're probably out of luck.  

The specific relevance to RAC, which I somehow managed to mention, is
that data blocks being shuttled 'tween nodes (depending on the
blocksize) can be placed into a smaller number of ethernet frames,
reducing both latency and overhead.  Ideally, each block will fit into
one ethernet frame, but as always, YMMV.

Thanks,
Matt

--
Matthew Zito
GridApp Systems
Email: [EMAIL PROTECTED]
Cell: 646-220-3551
Phone: 212-358-8211 x 359
http://www.gridapp.com

> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On 
> Behalf Of Ravi Kulkarni
> Sent: Thursday, July 10, 2003 2:49 PM
> To: Multiple recipients of list ORACLE-L
> Subject: RE: RAC system Calls
> 
> 
> Matt,
> What are jumbo frames? Are these assigning private
> network IPs to cluster_interconnects parameter?
> -Ravi.
> 
> 
> --- Matthew Zito <[EMAIL PROTECTED]> wrote:
> > 
> > And are you using jumbo frames on your interconnect?
> >  That can make a
> > significant contribution to reducing overhead from a
> > system standpoint.
> > 
> > Thanks,
> > Matt
> > 
> > --
> > Matthew Zito
> > GridApp Systems
> > Email: [EMAIL PROTECTED]
> > Cell: 646-220-3551
> > Phone: 212-358-8211 x 359
> > http://www.gridapp.com
> > 
> > > -----Original Message-----
> > > From: [EMAIL PROTECTED]
> > [mailto:[EMAIL PROTECTED] On
> > > Behalf Of K Gopalakrishnan
> > > Sent: Thursday, July 10, 2003 11:44 AM
> > > To: Multiple recipients of list ORACLE-L
> > > Subject: RE: RAC system Calls
> > > 
> > > 
> > > Ravi:
> > > 
> > > Do you have a statspack report? I would like to
> > see that. But
> > > in any case, 45% kernel is just too much?
> > > 
> > > BTW have you verified the private interconnect is
> > used
> > > for cache fusion transfer.. Make sure the cache
> > fusion
> > > is not going thru the public network.
> > > 
> > > 
> > > 
> > > Best Regards,
> > > K Gopalakrishnan
> > > 
> > >  
> > > 
> > > 
> > > -----Original Message-----
> > > Ravi Kulkarni
> > > Sent: Thursday, July 10, 2003 9:30 AM
> > > To: Multiple recipients of list ORACLE-L
> > > 
> > > 
> > > Hello List,
> > > 
> > > We are running Benchmark tests on Solaris 2-Node
> > RAC.
> > > Consistently noticed the following :
> > > - Very high Kernel usage (averaging 45%) on TOP
> > > - Statspack has "IPC Send Completion sync" waits
> > (70%
> > > Total ela time)
> > > - On trussing top process, found Oracle to be
> > issuing
> > > huge number of "times" system calls in addition to
> > 
> > > read/writes(which I think are select/inserts). Has
> > anyone
> > > noticed this in your environment. I am guessing
> > these to be
> > > inter-instance pings, but could not get any hits
> > in
> > > Doc/Metalink to confirm this. "times" call is
> > clocking lot of
> > > CPU. Is this normal ?
> > > Any pointers would be helpful ? If this is out of
> > > context, is there a separate list for RAC?
> > > 
> > > Thanks,
> > > Ravi.
> > > 
> > > 
> > > 
> > > __________________________________
> > > Do you Yahoo!?
> > > SBC Yahoo! DSL - Now only $29.95 per month!
> > http://sbc.yahoo.com
> > > --
> > > Please see the official ORACLE-L FAQ:
> > http://www.orafaq.net
> > > --
> > > Author: Ravi Kulkarni
> > >   INET: [EMAIL PROTECTED]
> > > 
> > > Fat City Network Services    -- 858-538-5051
> > http://www.fatcity.com
> > > San Diego, California        -- Mailing list and
> > web hosting services
> > >
> >
> ---------------------------------------------------------------------
> > > To REMOVE yourself from this mailing list, send an
> > E-Mail message
> > > to: [EMAIL PROTECTED] (note EXACT spelling of
> > 'ListGuru')
> > > and in the message BODY, include a line
> > containing: UNSUB
> > > ORACLE-L (or the name of mailing list you want to
> > be removed
> > > from).  You may also send the HELP command for
> > other
> > > information (like subscribing).
> > > 
> > > 
> > > --
> > > Please see the official ORACLE-L FAQ:
> > http://www.orafaq.net
> > > --
> > > Author: K Gopalakrishnan
> > >   INET: [EMAIL PROTECTED]
> > > 
> > > Fat City Network Services    -- 858-538-5051
> > http://www.fatcity.com
> > > San Diego, California        -- Mailing list and
> > web hosting services
> > >
> >
> ---------------------------------------------------------------------
> > > To REMOVE yourself from this mailing list, send an
> > E-Mail message
> > > to: [EMAIL PROTECTED] (note EXACT spelling of
> > 'ListGuru')
> > > and in the message BODY, include a line
> > containing: UNSUB
> > > ORACLE-L (or the name of mailing list you want to
> > be removed
> > > from).  You may also send the HELP command for
> > other
> > > information (like subscribing).
> > > 
> > 
> > --
> > Please see the official ORACLE-L FAQ:
> > http://www.orafaq.net
> > -- 
> > Author: Matthew Zito
> >   INET: [EMAIL PROTECTED]
> > 
> > Fat City Network Services    -- 858-538-5051
> > http://www.fatcity.com
> > San Diego, California        -- Mailing list and web
> > hosting services
> >
> ---------------------------------------------------------------------
> > To REMOVE yourself from this mailing list, send an
> > E-Mail message
> > to: [EMAIL PROTECTED] (note EXACT spelling of
> > 'ListGuru') and in
> > the message BODY, include a line containing: UNSUB
> > ORACLE-L
> > (or the name of mailing list you want to be removed
> > from).  You may
> > also send the HELP command for other information
> > (like subscribing).
> 
> 
> __________________________________
> Do you Yahoo!?
> SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com
> -- 
> Please see the official ORACLE-L FAQ: http://www.orafaq.net
> -- 
> Author: Ravi Kulkarni
>   INET: [EMAIL PROTECTED]
> 
> Fat City Network Services    -- 858-538-5051 http://www.fatcity.com
> San Diego, California        -- Mailing list and web hosting services
> ---------------------------------------------------------------------
> To REMOVE yourself from this mailing list, send an E-Mail message
> to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') 
> and in the message BODY, include a line containing: UNSUB 
> ORACLE-L (or the name of mailing list you want to be removed 
> from).  You may also send the HELP command for other 
> information (like subscribing).
> 

-- 
Please see the official ORACLE-L FAQ: http://www.orafaq.net
-- 
Author: Matthew Zito
  INET: [EMAIL PROTECTED]

Fat City Network Services    -- 858-538-5051 http://www.fatcity.com
San Diego, California        -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from).  You may
also send the HELP command for other information (like subscribing).

Reply via email to