SYSTEM ARCHITECTURE COUNCIL
Platform Software ARC
---------------------------------
PSARC Regular Meeting time: Wednesdays 10:00-1:00pm in MPK17-3507.
07-08-2009 MEETING MINUTES
============================================================================
Send CORRECTIONS, additions, deletions to psarc-coord at sun.com.
Minutes are archived in sac.Eng:/sac/export/sac/Minutes/PSARC.
Co-Chair(s):
Garrett D'Amore: Yes
Tim Marsland: no
ATTENDEES - Members: (6 active members)
Kais Belgaied: Yes
Mark Carlson: Yes
Richard Matthews: Yes
Darren Moffat: Yes (on sabbatical)
Sebastien Roy: Yes
Glenn Skinner: Yes
Bill Sommerfeld: no (on sabbatical)
Gary Winiger: no (on sabbatical)
STAFF -
Asa Romberger (PM): Yes
ATTENDEES - Interns:
Frank Che no
Charles Debardeleben: no
James Falkner: no (on sabbatical)
Daniel Hain: Yes
Michael Haines: no
Alan Hargreaves: no
Phil Harman: no
Cecilia Hu: no
Wyllys Ingersoll: no
Alec Muffett: no (on sabbatical)
Darren Reed: no
Dean Roehrich no
Ienup Sung: no
Phi Tran Yes
Brian Utterback: no
James Walker Yes
Mark Martin Yes (external)
Don Cragun Yes (external)
Guests:
-- GUESTS --
Sangeeta Misra Yes
Sharon Liu Yes
Michael Schuster Yes
Not all names are captured. Please send email to Asa.Romberger at Sun.com, if
you attended the meeting and your name is missing from the list.
---------------------------------------------------------------------------
MEETING SUMMARY:
================
AGENDA
10:00-10:10 Open ARC Business (use open dial in above)
10:10-10:55 Open Commitment: ILB: Integrated L3/L4 Load
balancer
(2008/575)
Submitter: Sangeeta Misra
Owner: Sebastien Roy
Intern: Mark Martin
Exposure: open
11:00-11:10 Closed ARC Business (use closed dial in above)
-if needed
---------------------------------------------------------------------------
Case Anchors: <br>
<A HREF="#case1">ILB: Integrated L3/L4 Load balancer (2008/575)</A> <br>
===========================================================================
Fast Tracks:
============
Case (Timeout) Exposure Title
2005/425 (07/08/09) unknown EOF /usr/ucb/cc & /usr/ucb/lint
approved
2009/374 (07/08/09) open libxmlsec
extend to 07/15/09
2009/377 (07/10/09) open In-kernel pfexec implementation.
approved
2009/378 (07/10/09) open Basic File Privileges
approved
2009/380 (07/13/09) open STREAMS and Character Device Coexistence
let run
2009/381 (07/13/09) open Support off-line files on SAM-QFS for
Samba in Solaris
let run
---------------------------------------------------------------------------
ILB: Integrated L3/L4 Load (2008/575)
IAM
====
Name: ILB: Integrated L3/L4 Load balancer
Submitter: Sangeeta Misra
Owner: Sebastien Roy
Intern: Mark Martin
Exposure: open
SUMMARY
=======
ILB project will deliver the basic features needed for
Solaris to be used as a L3/L4 load balancing solution on SPARC
and X86 platforms. This solution can also be used by third party
load balancing solutions.
ISSUES
=======
djr-00 Why does load balancing need to be a part of a core part of IP?
This is a non-essential and non-compulsory function and should
only need to be present on systems where it is configured to run.
Having this inside of IP adds bloat to IP and the base kernel size.
Architecturally, there should be no reason why it needs to be a
part
of IP. If it does then it suggests that there are deeper faults in
the project's use of interfaces and architecture. Further, if this
module needs access to "internal" data structures and functions
then
documenting those as an external module helps us to understand what
functionality we need to export from IP to support developers that
wish to develop kernel level modules that provide value added
functionality in appliances.
Response:
Nothing that prevents ilb from being in a seperate module than IP, we
simply chose to do it this way as it fits well with the forwarding
code. We are planning to deliver ILB kernel and userland component as
s seperate package from the core stack The project includes kernel and
userland components with the following names.( see ans to Q1 of 20 q)
SUNWilbr ILB kernel component
SUNWilb components delivered in /usr which are:
o ilbadm
o libilb
o ilbd
djr-01 Is there an architectural reason why you cannot support correct
load balancing of IP fragments or is it just an implementation
issue? If it is architecture, doesn't this suggest the project
needs to rethink its solution? If it is implementation, why
should the project be let off the hook for partial interoperability
with IP?
Response:
There is no architectural reasons but we have decided to consider this
item as a "good to have" instead of "must have" because it is not
considered critical item by the interested customers that we have
spoken to. While itlimit the ILB phase 1 UDP use cases somewhat, it
will in no way prevent deployment- so there is no harm in pushing out
as a post Phase 1 RFE. If we can addresss all the other items on the
must do list in ample time in Phase 1 we will consider implementing
this
djr-02 Are standard sockets being used or will a ilb-specific socket be
used and thus a new sockmod delivered? If it is the former, details
on what type of sockets (AF_INET,SOCK_RAW), etc, would be useful.
If it is the latter, details on the new sockmod are required.
Response:
Yes standard sockets
djr-03 Why is this project implementing yet another private hook inside
of IP's forwarding path rather than implementing and using a public
hook using PSARC/2005/334 and PSARC/2008/219? Whilst the exisiting
forwarding hook may not export the correct data for load balancing,
if this project were to publish a new packet event, even if it were
uncommited, it would encourage external projects/developers to use
it and hopefully contribute to building community involvement.
Response:
We have chosen this design instead of extending IP FIlter hooks to
ensure that the order of ILB processing and that of IP Filter is
correct. Furthermore, should we in the future need an ILB hook on the
transmit side, that hook wouldn't belong where pfhooks sits on the
transmit side; we'd need the transmit hook to be before IRE lookup and
fragmentation, instead of at the bottom of the IP output code(where
pfhook transmit is).
djr-04 When a process writes out the RTT to stdout, what should the RTT
be measured in? Further, is it a decimal number or an Integer?
Further, is a range of 1-254 sufficiently large enough for the
user process to provide adequate coverage of the acceptable
RTT values? One suggestion is for the RTT emitted by a user
task to be in nano-seconds (ping already reports these) and
to use -1 and 0 as "special values" and to support reading in
an RTT of [1,MAXINT] as "normal."
Response:
in millisec ( but we will consider the suggestion on nanosec)
djr-05 To elaborate further on 10.3, the use of ILB NAT with IPFilter will
not interoperate well with stateful filtering. Current stateful
filtering on a system that routes packets only requires one state
entry per "flow" (i.e. TCP connection) even though IPFilter's NAT
may be used on either side because IPFilter is able to match up
packet details in its NAT table. The use of an external NAT, such
as ILB's, will require 2 state table entries per connection because
IPFilter is unable to map the changed headers back.
Response:
Not sure what question is. I think he is a possible complication in
IP FIlter ILB NAT interaction here. We will have to work with him
offline on this.
djr-06 In 9, I'm guessing a typo at the end of the paragraph, unless
PSARC is 1800+ years old (PSARC/200/517)
djr-07 An interface table is required.
Response:
Agreed will be provided for PSARC committment.
Commitment Issues
=================
djr-08 See djr-07 from inception. Unless I'm blind (quite possible), I
don't
see an interface table in this design document. Are you
deliverying no
interfaces at all?
Response:
The interface table is the answer to the questionnaire question #3.
djr-09 ILB-design.txt (1) - the filenames by themselves aren't enough.
Full pathnames and an interface table are required.
Response:
Will update the design doc.
djr-10 Where does the unix socket for ilbd live?
Response:
The document states, "The /var/run/daemon directory will hold the
AF_UNIX rendezvous file". So it will be /var/run/daemon/ilb_sock.
ged-01 Committed. The rationale in the 20q states that 3rd party apps
might
need to be supported. Can you provide more detail? Would
Uncommitted
be adequate to this? Committed interfaces are night impossible to
change, so I'd prefer to see new APIs take conservative approaches
in their use, at least until some implementation experience is
gained.
Response:
It is unclear if making these API Uncommitted will deter folks from safely
using these APIs. We plan on bumping up a library version and shipping
multiple versions to ensure backward compatibility.
ged-02 persist=<netmask>. I don't understand why a netmask would be useful
with this feature. It seems like you either want session
persistence,
or you don't, regardless of the originating network. What am I
missing?
Response:
ILB provides stickiness per load balancing rule (a rule represents a virtual
service) . A user just specifies whether a rule is sticky or not and what
netmask to use. After a persistent mapping is created, subsequent
requests/packets to this virtual service with a matching source IP address
of the client (after the persistent netmask is applied) will
be forwarded to the same back end server
ged-03 Port range collapsing. What's the use case for this feature?
Response:
This is a feature that was requested by one of the interested customers.
According to them, there are protocols out there, like RMI that
make an initial connection over a specific port but then set up a port
in a particular range. Handling this with their particular LB software,
would involve the creation of a large number virtual servers, one for each
of the ports that may be used.
ged-04 Health check. Will the implementation any sample or default health
check helper programs?
Response:
By default ILB will not run any health checks. The user has to specify a
health check object and specify one of these health check types:
ping, tcp probe, udp probe, or user supplied test
and associate it to a rule during rule creation. ILB provides built-in ping,
tcp probe, udp probes.
ged-05 Performance concerns. It sounds like ILB adds another "if" branch
on the per-packet hot code path. This can have quite detrimental
effects to per-packet overhead, and seriously impact the maximum
packets-per-second that can be processed (e.g. for IP forwarding).
There are benchmarks that test for this, and packet-per-second
numbers
are important to certain market segments. How will the ILB project
ensure this component of performance is not negatively impacted?
Response:
We plan to test forwarding performance (with ILB disabled which is its
default state) with Ixia and make sure there no severe regression in
forwarding perf against onnv. Forwarding performance with ILB enabled,
should be around 90% of the plain forwarding performance
kb-01 IPFilter offers NAT. This project also uses / offers NAT.
Any thoughts on the expected behavior when both IPFilter's NAT and
this project are active on a the same system?
Can inadvertant activating of both lead to unintended results?
Both are SMF based services, so can some dependency be
expressed here?
VOTE
====
Approve - 5
Deny -
Abstain -
Not Participating (NP) -
THE NEXT STEP
=============
Approved
