zstreamdump [PSARC/2009/491 Self Review]
On the existing zfs man page, I see the text: The format of the stream is evolving. No backwards compati- bility is guaranteed. You may not be able to receive your streams on future versions of ZFS. But I see no similar warning for zstreamdump, nor an explanation as to why that warning does not also apply to it. -- This message posted from opensolaris.org
zstreamdump [PSARC/2009/491 Self Review]
Nicolas Williams Nicolas.Williams at sun.com wrote: It'd be nice if zfs recv could do this too, no? Alternatively, it'd be nice if there was a way to slot zstreamdump into a zfs send | ... | zfs recv pipe. E.g., an option to zstreamdump to send its output to a file or to stderr, passing the stream through to stdout for piping elsewhere. It looks a bit inconsistent to put a lot of functionality into programs like fpool and zfs and on the other side create new programs for minor features. J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de(uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
zstreamdump [PSARC/2009/491 Self Review]
Joerg Schilling wrote: Nicolas Williams Nicolas.Williams at sun.com wrote: It'd be nice if zfs recv could do this too, no? Alternatively, it'd be nice if there was a way to slot zstreamdump into a zfs send | ... | zfs recv pipe. E.g., an option to zstreamdump to send its output to a file or to stderr, passing the stream through to stdout for piping elsewhere. It looks a bit inconsistent to put a lot of functionality into programs like fpool and zfs and on the other side create new programs for minor features. This is a debugging tool only. -- Darren J Moffat
zstreamdump [PSARC/2009/491 Self Review]
I know this case was Self Review and I agree with it being self review. It is after all a debugging tool. As a ZFS developer I've been using this tool already and it has been a huge help to me debugging problems with ZFS Crypto and send|recv. I see very little use for it by normal admins and users because unless you know how the stream is formatted and know how ZFS really works (especially how send|recv work) its output won't be that understandable. Traditionally tools like this wouldn't have been packaged at all but would have been available in bfu archives only. The plan is that bfu will go way sometime after the transition of ON building SVR4 to IPS packages so internal tools now need to be packaged up. With my ARC hat on +1 to this case - even though it doesn't strictly need it. -- Darren J Moffat
Python Routes [LSARC/2009/481 FastTrack timeout 09/17/2009]
Mark, I really don't have the bandwidth to do cherrypy and since is is already available in /release, I would not bother...sorry.. -manjunath Mark Martin wrote: On Tue, Sep 15, 2009 at 12:23 AM, Manjunath Basappa Manjunath.Basappa at sun.com wrote: Mark Martin wrote Can we get an updated spec, please? Summary === Routes is a Python re-implementation of the Rails routes system for mapping URL's to Controllers/Actions and generating URL's. Routes makes it easy to create pretty and concise URL's that are RESTful with little effort. snip Rooutes is delivered as a set of python modules and classes. It works with in a python based web framework. This package delivers the latest version of routes (1.10.3) which works with python versions 2.4 and 2.6. The packages that will be delivered are: SUNWpython26-routes SUNWpython24-routes Dependencies SUNWPython at 2.4.4,5.11-0.111:20090508T152730Z SUNWPython26 at 2.6.1,5.11-0.111:20090508T152901Z Interfaces == Man pages are included. The SUNWpy-routes package is Uncommitted. The remaining interfaces are Volatile. Exported Interfaces (python modules) --- routes Provides common classes and functions most users will want access to. routes.base Route and Mapper core classes routes.middleware Routes WSGI Middleware routes.threadinglocal routes.util Utility functions for use in templates / controllers Manjunath, Aside from the following two minor issues, this looks fine to me. 1) I'd prefer to see package dependencies not marked up as IPS dependencies. Hopefully that's a small enough request and we won't have to go into things here. 2) Why volatile on these interfaces? Why can't they be uncommitted? P.S. Why not run a fast track on CherryPy if you've already done the work on that? :) -- next part -- An embedded and charset-unspecified text was scrubbed... Name: routes.txt URL: http://mail.opensolaris.org/pipermail/opensolaris-arc/attachments/20090916/a9b603d0/attachment-0001.txt
Copy Reduction Interfaces [PSARC/2009/478 FastTrack timeout 09/16/2009]
Filesystems might have some blocksize and alignment constraints
conditioning their ability to loan up buffers (for writes).
If that is so, we could use an API to query the FS about
those values. For a copy on write variable block size
filesystem, that natural blocksize might also depend on the
vnode being targetted. Do we know if ZFS will ever be able to
loan up buffers for writes that are not aligned full records ?
-r
Rich.Brown at Sun.COM writes:
I'm sponsoring this case on behalf of Mahesh Siddheshwar and Chunli Zhang.
This case proposes new interfaces to support copy reduction in the I/O path
especially for file sharing services.
Minor binding is requested.
This times out on Wednesday, 16 September, 2009.
Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI
This information is Copyright 2009 Sun Microsystems
1. Introduction
1.1. Project/Component Working Name:
Copy Reduction Interfaces
1.2. Name of Document Author/Supplier:
Author: Mahesh Siddheshwar, Chunli Zhang
1.3 Date of This Document:
09 September, 2009
4. Technical Description
== Introduction/Background ==
Zero-copy (copy avoidance) is essentially buffer sharing
among multiple modules that pass data between the modules.
This proposal avoids the data copy in the READ/WRITE path
of filesystems, by providing a mechanism to share data buffers
between the modules. It is intended to be used by network file
sharing services like NFS, CIFS or others.
Although the buffer sharing can be achieved through a few different
solutions, any such solution must work with File Event Monitors
(FEM monitors)[1] installed on the files. The solution must
allow the underlying filesystem to maintain any existing file
range locking in the filesystem.
The proposed solution provides extensions to the existing VOP
interface to request and return buffers from a filesystem. The
buffers are then used with existing VOP_READ/VOP_WRITE calls with
minimal changes.
== Proposed Changes ==
VOP Extensions for Zero-Copy Support
a. Extended struct uio, xuio_t
The following proposes an extensible uio structure that can be extended for
multiple purposes. For example, an immediate extension, xu_zc, is to be
used by the proposed VOP_REQZCBUF/VOP_RETZCBUF interfaces to pass loaned
zero-copy buffers, as well as to be passed to the existing
VOP_READ/VOP_WRITE
calls for normal read/write operations. Another example of extension,
xu_aio, is intended to replace uioa_t for async I/O.
This new structure, xuio_t, contains the following:
- the existing uio structure (embedded) as the first member
- additional fields to support extensibility
- a union of all the defined extensions
The following uio_extflag is added to indicate that an uio structure is
indeed an xuio_t:
#defineUIO_XUIO0x004 /* Structure is xuio_t */
The following uio_extflag will be removed after uioa_t has been converted
to xuio_t:
#defineUIO_ASYNC 0x002 /* Structure is xuio_t */
The project team has commitment from the networking team to remove
the current use of uioa_t and use the proposed extensions (CR 6880095).
The definition of xuio_t is:
typedef struct xuio {
uio_t xu_uio;/* Embedded UIO structure */
/* Extended uio fields */
enum xuio_type xu_type; /* What kind of uio structure? */
union {
/* Async I/O Support */
struct {
uint32_t xu_a_state; /* state of async i/o */
uint32_t xu_a_state; /* state of async i/o */
ssize_t xu_a_mbytes; /* bytes that have been uioamove()ed */
uioa_page_t *xu_a_lcur; /* pointer into uioa_locked[] */
void **xu_a_lppp;/* pointer into
lcur-uioa_ppp[] */
void *xu_a_hwst[4]; /* opaque hardware state */
uioa_page_t xu_a_locked[UIOA_IOV_MAX]; /* Per iov locked pages
*/
} xu_aio;
/* Zero Copy Support */
struct {
enum uio_rw xu_zc_rw;/* the use of the buffer */
void *xu_zc_priv;/* fs specific */
} xu_zc;
} xu_ext;
} xuio_t;
where xu_type is currently defined as:
typedef enum xuio_type {
UIOTYPE_ASYNCIO,
UIOTYPE_ZEROCOPY
} xuio_type_t;
New uio extensions can be added by defining a new xuio_type_t, and adding a
new member to the xu_ext union.
b. Requesting zero-copy buffers
#define VOP_REQZCBUF(vp, rwflag, uiozcp, cr, ct) \
fop_reqzcbuf(vp, rwflag, uiozcp, cr, ct)
int fop_reqzcbuf(vnode_t *, enum uio_rw, xuio_t *, cred_t *,
caller_context_t *);
This function requests buffers
PSARC 2009/493 ld -z wrap option
I am sponsoring the following self-reviewed case for myself.
It adds a new option (-z wrap=symbol) to the Solaris link-editor (ld)
along with two synonyms: -wrap=symbol and --wrap=symbol.
I believe it qualifies for self review:
- This is a GNU familiarity case --- the GNU ld used by Linux accepts
the -wrap/--wrap options and it is used by software of importance to
Solaris (Xorg).
- We have established precedent for providing GNU ld options to ease
building of open source software on our platform. One example is:
PSARC 2008/583 add gld options to ld(1)
- There are no backward compatibility issues with the proposed syntax,
and the form of the syntax does not break new ground.
The wrap option tells the link-editor to rename function references
in a systematic way that allows the user to link wrapper functions
against already compiled code. The user of this feature supplies the
wrapper functions. This feature will be integrated as:
6850768 ld option to autogenerate wrappers/interposers similar to
GNU ld --wrap
Documentation for the original GNU ld feature can be found at:
http://sourceware.org/binutils/docs-2.19/ld/Options.html#index-g_t_002d_002dwrap-249
Diffs for the changes to the proposed new ld(1) manpage are shown below.
-
Release Binding:Patch/Micro
New ld options:
-z wrap Committed
-wrap Committed
--wrap Committed
-
*** ld.orig Mon Aug 24 14:38:24 2009
--- ld.new Tue Sep 15 13:28:59 2009
***
*** 30,36
[-z preinitarray=function] [-z redlocsym] [-z relaxreloc]
[-z rescan-now] [-z recan] [-z rescan-start ... -z rescan-end]]
[-z target=sparc|x86] [-z text | textwarn | textoff]
! [-z verbose] filename...
DESCRIPTION
--- 30,36
[-z preinitarray=function] [-z redlocsym] [-z relaxreloc]
[-z rescan-now] [-z recan] [-z rescan-start ... -z rescan-end]]
[-z target=sparc|x86] [-z text | textwarn | textoff]
! [-z verbose] [-z wrap=symbol] filename...
DESCRIPTION
***
*** 1114,1119
--- 1114,1152
are deemed too noisy to be generated by default.
+ -z wrap=symbol
+ -wrap=symbol
+ --wrap=symbol
+
+ Rename undefined references to symbol in order to allow wrapper
+ code to be linked into the output object without having to modify
+ source code. When -z wrap is specified, all undefined references
+ to symbol are modified to reference __wrap_symbol, and all
+ references to __real_symbol are modified to reference symbol. The
+ user is expected to provide an object containing the __wrap_symbol
+ function. This wrapper function can call __real_symbol in order to
+ reference the actual function being wrapped.
+
+ The following is an example of a wrapper for the malloc() function:
+
+ void *
+ __wrap_malloc (size_t c)
+ {
+ printf (malloc called with %zu\n, c);
+ return __real_malloc (c);
+ }
+
+ If you link other code with this file using -z wrap=malloc to
+ compile all the objects, then all calls to malloc will call the
+ function __wrap_malloc instead. The call to __real_malloc in
+ __wrap_malloc will call the real malloc function.
+
+ The real and wrapped functions should be maintained in separate
+ source files. Otherwise, the compiler or assembler may resolve
+ the call instead of leaving that operation for the link-editor
+ to carry out, and prevent the wrap from occurring.
+
+
ENVIRONMENT VARIABLES
LD_ALTEXEC
PSARC 2009/493 ld -z wrap option
This looks like a potentially handy feature, and the architecture seems
fairly straight-forward. However, I'm not sure it qualifies for
self-review. (Although the bar for self review seems to be moving
somewhat...) If only because it introduces new interfaces and behavior
for consumers.
I'd suggest that this maybe should be a fast-track instead of self
review, unless there is some particular rush to get this case through?
Please consider promoting it to a fast track. In any case, I'll happily
give it a +1.
Btw, you failed to specify a release binding. Is this patch or minor
(or something else)? (It looks like this could be Patch binding fairly
easily...)
- Garrett
Ali Bahrami wrote:
I am sponsoring the following self-reviewed case for myself.
It adds a new option (-z wrap=symbol) to the Solaris link-editor (ld)
along with two synonyms: -wrap=symbol and --wrap=symbol.
I believe it qualifies for self review:
- This is a GNU familiarity case --- the GNU ld used by Linux accepts
the -wrap/--wrap options and it is used by software of
importance to
Solaris (Xorg).
- We have established precedent for providing GNU ld options to ease
building of open source software on our platform. One example is:
PSARC 2008/583 add gld options to ld(1)
- There are no backward compatibility issues with the proposed
syntax,
and the form of the syntax does not break new ground.
The wrap option tells the link-editor to rename function references
in a systematic way that allows the user to link wrapper functions
against already compiled code. The user of this feature supplies the
wrapper functions. This feature will be integrated as:
6850768 ld option to autogenerate wrappers/interposers similar to
GNU ld --wrap
Documentation for the original GNU ld feature can be found at:
http://sourceware.org/binutils/docs-2.19/ld/Options.html#index-g_t_002d_002dwrap-249
Diffs for the changes to the proposed new ld(1) manpage are shown below.
-
Release Binding:Patch/Micro
New ld options:
-z wrapCommitted
-wrapCommitted
--wrapCommitted
-
*** ld.origMon Aug 24 14:38:24 2009
--- ld.newTue Sep 15 13:28:59 2009
***
*** 30,36
[-z preinitarray=function] [-z redlocsym] [-z relaxreloc]
[-z rescan-now] [-z recan] [-z rescan-start ... -z rescan-end]]
[-z target=sparc|x86] [-z text | textwarn | textoff]
! [-z verbose] filename...
DESCRIPTION
--- 30,36
[-z preinitarray=function] [-z redlocsym] [-z relaxreloc]
[-z rescan-now] [-z recan] [-z rescan-start ... -z rescan-end]]
[-z target=sparc|x86] [-z text | textwarn | textoff]
! [-z verbose] [-z wrap=symbol] filename...
DESCRIPTION
***
*** 1114,1119
--- 1114,1152
are deemed too noisy to be generated by default.
+ -z wrap=symbol
+ -wrap=symbol
+ --wrap=symbol
+
+ Rename undefined references to symbol in order to allow
wrapper
+ code to be linked into the output object without having to
modify
+ source code. When -z wrap is specified, all undefined
references
+ to symbol are modified to reference __wrap_symbol, and all
+ references to __real_symbol are modified to reference
symbol. The
+ user is expected to provide an object containing the
__wrap_symbol
+ function. This wrapper function can call __real_symbol in
order to
+ reference the actual function being wrapped.
+
+ The following is an example of a wrapper for the malloc()
function:
+
+ void *
+ __wrap_malloc (size_t c)
+ {
+ printf (malloc called with %zu\n, c);
+ return __real_malloc (c);
+ }
+
+ If you link other code with this file using -z wrap=malloc to
+ compile all the objects, then all calls to malloc will call
the
+ function __wrap_malloc instead. The call to __real_malloc in
+ __wrap_malloc will call the real malloc function.
+
+ The real and wrapped functions should be maintained in
separate
+ source files. Otherwise, the compiler or assembler may resolve
+ the call instead of leaving that operation for the link-editor
+ to carry out, and prevent the wrap from occurring.
+
+
ENVIRONMENT VARIABLES
LD_ALTEXEC
PSARC 2009/493 ld -z wrap option
Garrett D'Amore wrote: This looks like a potentially handy feature, and the architecture seems fairly straight-forward. However, I'm not sure it qualifies for self-review. (Although the bar for self review seems to be moving somewhat...) If only because it introduces new interfaces and behavior for consumers. I'd suggest that this maybe should be a fast-track instead of self review, unless there is some particular rush to get this case through? Please consider promoting it to a fast track. In any case, I'll happily give it a +1. Given this is being included explicitly for compatibility with GNU ld there is nothing ARC can change anyway. It is only a +1 or rederail and vote no type of case. So I really don't see the point in adding pointless extra process for a case like this. -- Darren J Moffat
PSARC 2009/493 ld -z wrap option
Garrett D'Amore wrote: This looks like a potentially handy feature, and the architecture seems fairly straight-forward. However, I'm not sure it qualifies for self-review. (Although the bar for self review seems to be moving somewhat...) If only because it introduces new interfaces and behavior for consumers. I'd suggest that this maybe should be a fast-track instead of self review, unless there is some particular rush to get this case through? Please consider promoting it to a fast track. In any case, I'll happily give it a +1. Btw, you failed to specify a release binding. Is this patch or minor (or something else)? (It looks like this could be Patch binding fairly easily...) - Garrett I'm not opposed to making it a fast track. However, I'd like to ask you to reconsider. It's a trivial option being implemented purely for Linux compatibility. Software we care about (Xorg) uses it, which is the sole reason for introducing it. It seems well below the line to me... I did specify the release binding, in the middle: - Release Binding:Patch/Micro New ld options: -z wrapCommitted -wrapCommitted --wrapCommitted - Thanks... - Ali
PSARC 2009/493 ld -z wrap option
Darren J Moffat wrote: Garrett D'Amore wrote: This looks like a potentially handy feature, and the architecture seems fairly straight-forward. However, I'm not sure it qualifies for self-review. (Although the bar for self review seems to be moving somewhat...) If only because it introduces new interfaces and behavior for consumers. I'd suggest that this maybe should be a fast-track instead of self review, unless there is some particular rush to get this case through? Please consider promoting it to a fast track. In any case, I'll happily give it a +1. Given this is being included explicitly for compatibility with GNU ld there is nothing ARC can change anyway. It is only a +1 or rederail and vote no type of case. So I really don't see the point in adding pointless extra process for a case like this. Actually, as a fast track, it doesn't really add any process *as long as someone reviews it*. Obviously both of us have reviewed it, so that burden doesn't add anything to this caes. It *does* incur a little extra delay to wait for a fast track timeout. I don't think that is too onerous. Note that I'm *not* suggesting that this should be a full case. :-) IMO familiarity doesn't automatically make it self-review. Otherwise we might as well just abdicate all architecture and allow anything from GNU/Linux to come in. That doesn't seem to have been the precedent. - Garrett
PSARC 2009/493 ld -z wrap option
Ali Bahrami wrote: Garrett D'Amore wrote: This looks like a potentially handy feature, and the architecture seems fairly straight-forward. However, I'm not sure it qualifies for self-review. (Although the bar for self review seems to be moving somewhat...) If only because it introduces new interfaces and behavior for consumers. I'd suggest that this maybe should be a fast-track instead of self review, unless there is some particular rush to get this case through? Please consider promoting it to a fast track. In any case, I'll happily give it a +1. Btw, you failed to specify a release binding. Is this patch or minor (or something else)? (It looks like this could be Patch binding fairly easily...) - Garrett I'm not opposed to making it a fast track. However, I'd like to ask you to reconsider. It's a trivial option being implemented purely for Linux compatibility. Software we care about (Xorg) uses it, which is the sole reason for introducing it. It seems well below the line to me... I guess I see the line differently than you do. To me the line is set for things that: a) introduce or modify no non-Private interfaces or b) backed by well established precedent (e.g. drivers for new hardware that follow established architecture for devices of the given type) Admittedly, this case probably is pretty close to the line on b. Really, apart from the incursion of a little extra delay (probably one week in this case), there is nothing else needed at this point if your case is turned into a fast track, except in the unlikely event that someone takes exception with the case. I did specify the release binding, in the middle: - Release Binding:Patch/Micro Ah I must have missed that. - Garrett New ld options: -z wrapCommitted -wrapCommitted --wrapCommitted - Thanks... - Ali
IP_DONTFRAG socket option [PSARC/2009/494 FastTrack timeout 09/23/2009]
I'm submitting this fast-track for Erik Nordmark, it times out on 09/23/2009. The release binding is Patch. Background: -- Busy DNS servers, and other servers that do UDP request/response protocols, typically want to avoid Path MTU discovery since path MTU discovery both adds latency (a packet would be dropped by routers instead of being fragmented and forward) and adds state on the server (Path MTU state would be created for the destination IP address). That is counterproductive when there are lots of clients that do a single UDP request/response to the server. Details: --- For IPv6 we have had two socket options to control this (from RFC 3542) IPv6_USE_MIN_MTU and IPV6_DONTFRAG. But there is no standard for IPv4. However, FreeBSD implements an IP_DONTFRAG socket option, which is used by the BIND DNS server software. This case is to introduce IP_DONTFRAG in Solaris. Exported Interfaces - InterfaceClassification Comments - IP_DONTFRAG Committed ip(7P) - Man page updates: Add this text to ip(7P) after IP_TOS: IP_DONTFRAG If enabled (the default) then the Don't Fragment flag is set on IP packets. Disabling the option means that Don't Fragment will not be set which will result in not creating any Path MTU state due to this socket.
PSARC 2009/493 ld -z wrap option
Garrett D'Amore wrote: b) backed by well established precedent (e.g. drivers for new hardware that follow established architecture for devices of the given type) There's at least 10 years of precedent of introducing new public ld flags in self-review cases. -- -Alan Coopersmith- alan.coopersmith at sun.com Sun Microsystems, Inc. - X Window System Engineering
netstat -r flags for blackhole and reject routes [PSARC/2009/495 FastTrack timeout 09/23/2009]
I'm submitting this fast-track for Erik Nordmark. It times out on 09/23/2009. The release binding is Minor due to the change in semantics of the netstat -r 'B' route flag. Background: -- Using route(1m) the administrator can add reject or blackhole routes. A routing daemon can add them using route(7p). Such routes, if matched by a packet, have the effect of dropping the packet. A reject route would send an ICMP error when matched, and a blackhole would silently drop the packet. It is possible to see whether these flags are set on a route using route get ipaddr. But the flags are not reported in netstat -r. That is odd given that their effect is completely different than other routes. BSD shows the setting of those flags in netstat -r. In NetBSD the letters are B RTF_BLACKHOLEJust discard pkts (during updates). R RTF_REJECT Host or net unreachable. In FreeBSD the letters are B RTF_BLACKHOLEJust discard pkts (during updates) b RTF_BROADCASTThe route represents a broadcast address R RTF_REJECT Host or net unreachable Solaris currently uses 'B' for broadcast (in the netstat -ra output; does not appear without the 'a' option). Proposal: Switch the current Solaris broadcast letter from 'B' to 'b' i.e. use the FreeBSD choice of letters above. (An alternative would be to keep 'B' for broadcast, and use the lower-case 'b' and 'r' for blackhole and reject, but that makes Solaris unique, which isn't necessarily a positive statement in this case.) Exported Interfaces - InterfaceClassificationComments - netstat -r outputUncommitted(*)netstat(1m) (*) PSARC/2001/355 has some background on this. Man page changes: netstat(1m) is changed as follows: *** ipd.netstat.origTue Sep 15 23:15:55 2009 --- ipd.netstat.new Tue Sep 15 23:17:14 2009 *** *** 692,699 --- 692,702 DRoute was created dynamically by a redirect. + BPackets will be silently dropped (RTF_BLACKHOLE set) + RPackets will be dropped with ICMP error sent (RTF_REJECT set) + If the -a option is specified, there will be routing entries with the following flags: *** *** 700,706 ACombined routing and address resolution entries. ! BBroadcast addresses. LLocal addresses for the host. --- 703,709 ACombined routing and address resolution entries. ! bBroadcast addresses. LLocal addresses for the host.
PSARC 2009/477 Addition of NE_IFINDEX_CHANGE to sys/neti.h
+1 on this case. - Garrett
IP_DONTFRAG socket option [PSARC/2009/494 FastTrack timeout 09/23/2009]
+1 Kais, On 09/16/09 09:03, Sebastien Roy wrote: I'm submitting this fast-track for Erik Nordmark, it times out on 09/23/2009. The release binding is Patch. Background: -- Busy DNS servers, and other servers that do UDP request/response protocols, typically want to avoid Path MTU discovery since path MTU discovery both adds latency (a packet would be dropped by routers instead of being fragmented and forward) and adds state on the server (Path MTU state would be created for the destination IP address). That is counterproductive when there are lots of clients that do a single UDP request/response to the server. Details: --- For IPv6 we have had two socket options to control this (from RFC 3542) IPv6_USE_MIN_MTU and IPV6_DONTFRAG. But there is no standard for IPv4. However, FreeBSD implements an IP_DONTFRAG socket option, which is used by the BIND DNS server software. This case is to introduce IP_DONTFRAG in Solaris. Exported Interfaces - InterfaceClassification Comments - IP_DONTFRAG Committed ip(7P) - Man page updates: Add this text to ip(7P) after IP_TOS: IP_DONTFRAG If enabled (the default) then the Don't Fragment flag is set on IP packets. Disabling the option means that Don't Fragment will not be set which will result in not creating any Path MTU state due to this socket.
netstat -r flags for blackhole and reject routes [PSARC/2009/495 FastTrack timeout 09/23/2009]
+1 Kais. On 09/16/09 09:10, Sebastien Roy wrote: I'm submitting this fast-track for Erik Nordmark. It times out on 09/23/2009. The release binding is Minor due to the change in semantics of the netstat -r 'B' route flag.
PSARC 2009/493 ld -z wrap option
Rod Evans wrote: Garrett D'Amore wrote: Really, apart from the incursion of a little extra delay (probably one week in this case), there is nothing else needed at this point if your case is turned into a fast track, except in the unlikely event that someone takes exception with the case. I still don't see the point. We don't decide upon submitting a fast-track, or a self-review case based on the time we wish to wait for the process to complete, we always wait a day or two after submitting a self-review to ensure we give folks time to comment. We make the distinction based upon whether we feel there's any architectural issues that might concern or be of interest to the ARC. In this case, as with most linker cases, we don't think there are. Thus, we're going to use the lightest weight process. Alright, I yield. I thought that there might be something here of architectural interest. I guess nobody else thinks so, and I'm happy with the case as specified, so let it go as self-review. - Garrett
Copy Reduction Interfaces [PSARC/2009/478 FastTrack timeout 09/16/2009]
Are there instances where an assigned zero-copy buffer could be orphaned?
If so, should there be a recovery list associated with this addition?
Perhaps off
the designated vnode.
This comment shouldn't block fast-track approval. Just a question.
--
Rick
On 09/ 9/09 04:02 PM, Rich.Brown at Sun.COM wrote:
I'm sponsoring this case on behalf of Mahesh Siddheshwar and Chunli Zhang.
This case proposes new interfaces to support copy reduction in the I/O path
especially for file sharing services.
Minor binding is requested.
This times out on Wednesday, 16 September, 2009.
Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI
This information is Copyright 2009 Sun Microsystems
1. Introduction
1.1. Project/Component Working Name:
Copy Reduction Interfaces
1.2. Name of Document Author/Supplier:
Author: Mahesh Siddheshwar, Chunli Zhang
1.3 Date of This Document:
09 September, 2009
4. Technical Description
== Introduction/Background ==
Zero-copy (copy avoidance) is essentially buffer sharing
among multiple modules that pass data between the modules.
This proposal avoids the data copy in the READ/WRITE path
of filesystems, by providing a mechanism to share data buffers
between the modules. It is intended to be used by network file
sharing services like NFS, CIFS or others.
Although the buffer sharing can be achieved through a few different
solutions, any such solution must work with File Event Monitors
(FEM monitors)[1] installed on the files. The solution must
allow the underlying filesystem to maintain any existing file
range locking in the filesystem.
The proposed solution provides extensions to the existing VOP
interface to request and return buffers from a filesystem. The
buffers are then used with existing VOP_READ/VOP_WRITE calls with
minimal changes.
== Proposed Changes ==
VOP Extensions for Zero-Copy Support
a. Extended struct uio, xuio_t
The following proposes an extensible uio structure that can be extended for
multiple purposes. For example, an immediate extension, xu_zc, is to be
used by the proposed VOP_REQZCBUF/VOP_RETZCBUF interfaces to pass loaned
zero-copy buffers, as well as to be passed to the existing
VOP_READ/VOP_WRITE
calls for normal read/write operations. Another example of extension,
xu_aio, is intended to replace uioa_t for async I/O.
This new structure, xuio_t, contains the following:
- the existing uio structure (embedded) as the first member
- additional fields to support extensibility
- a union of all the defined extensions
The following uio_extflag is added to indicate that an uio structure is
indeed an xuio_t:
#define UIO_XUIO0x004 /* Structure is xuio_t */
The following uio_extflag will be removed after uioa_t has been converted
to xuio_t:
#define UIO_ASYNC 0x002 /* Structure is xuio_t */
The project team has commitment from the networking team to remove
the current use of uioa_t and use the proposed extensions (CR 6880095).
The definition of xuio_t is:
typedef struct xuio {
uio_t xu_uio; /* Embedded UIO structure */
/* Extended uio fields */
enum xuio_type xu_type; /* What kind of uio structure? */
union {
/* Async I/O Support */
struct {
uint32_t xu_a_state; /* state of async i/o */
uint32_t xu_a_state; /* state of async i/o */
ssize_t xu_a_mbytes; /* bytes that have been uioamove()ed */
uioa_page_t *xu_a_lcur; /* pointer into uioa_locked[] */
void **xu_a_lppp; /* pointer into lcur-uioa_ppp[] */
void *xu_a_hwst[4]; /* opaque hardware state */
uioa_page_t xu_a_locked[UIOA_IOV_MAX]; /* Per iov locked pages
*/
} xu_aio;
/* Zero Copy Support */
struct {
enum uio_rw xu_zc_rw; /* the use of the buffer */
void *xu_zc_priv; /* fs specific */
} xu_zc;
} xu_ext;
} xuio_t;
where xu_type is currently defined as:
typedef enum xuio_type {
UIOTYPE_ASYNCIO,
UIOTYPE_ZEROCOPY
} xuio_type_t;
New uio extensions can be added by defining a new xuio_type_t, and adding a
new member to the xu_ext union.
b. Requesting zero-copy buffers
#define VOP_REQZCBUF(vp, rwflag, uiozcp, cr, ct) \
fop_reqzcbuf(vp, rwflag, uiozcp, cr, ct)
int fop_reqzcbuf(vnode_t *, enum uio_rw, xuio_t *, cred_t *,
caller_context_t *);
This function requests buffers associated with file vp in preparation for
a
subsequent zero copy read or write. The extended uio_t -- xuio_t is used
to pass the parameters and results. Only the following fields of xuio_t
are
relevant to this call.
uiozcp-xu_uio.uio_resid: used by the caller to specify the total
PSARC 2009/493 ld -z wrap option
Garrett D'Amore wrote: Really, apart from the incursion of a little extra delay (probably one week in this case), there is nothing else needed at this point if your case is turned into a fast track, except in the unlikely event that someone takes exception with the case. I still don't see the point. We don't decide upon submitting a fast-track, or a self-review case based on the time we wish to wait for the process to complete, we always wait a day or two after submitting a self-review to ensure we give folks time to comment. We make the distinction based upon whether we feel there's any architectural issues that might concern or be of interest to the ARC. In this case, as with most linker cases, we don't think there are. Thus, we're going to use the lightest weight process. -- Rod.
Copy Reduction Interfaces [PSARC/2009/478 FastTrack timeout 09/16/2009]
Roch wrote:
Filesystems might have some blocksize and alignment constraints
conditioning their ability to loan up buffers (for writes).
If that is so, we could use an API to query the FS about
those values. For a copy on write variable block size
filesystem, that natural blocksize might also depend on the
vnode being targetted.
Yes. The provider can fail the VOP_REQZCBUF() call if it determines
that it is inefficient to take the zero-copy path. Depending on the
provider implementation, this could be blocksize aligned. In such cases,
the consumer could use VFSNAME_STATVFS() call to determine
'f_bsize' value. But as you note, certain implementations may have
different values for individual files. In such cases if the VOP_REQZCBUF()
fails, the consumer then uses the traditional non zero-copy path.
An additional API to find the such constraints/requirements may
be useful in future, but is out-of-scope for this project. However, the
project team will open an RFE for this issue and put you on the
interest list.
Do we know if ZFS will ever be able to
loan up buffers for writes that are not aligned full records ?
No, not planned currently. It has to be block size aligned.
Also note that currently, from an implementation perspective,
zero-copy WRITEs are efficient only in case network-based
filesystems like NFS over RDMA transports.
Mahesh
-r
Rich.Brown at Sun.COM writes:
I'm sponsoring this case on behalf of Mahesh Siddheshwar and Chunli Zhang.
This case proposes new interfaces to support copy reduction in the I/O path
especially for file sharing services.
Minor binding is requested.
This times out on Wednesday, 16 September, 2009.
Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI
This information is Copyright 2009 Sun Microsystems
1. Introduction
1.1. Project/Component Working Name:
Copy Reduction Interfaces
1.2. Name of Document Author/Supplier:
Author: Mahesh Siddheshwar, Chunli Zhang
1.3 Date of This Document:
09 September, 2009
4. Technical Description
== Introduction/Background ==
Zero-copy (copy avoidance) is essentially buffer sharing
among multiple modules that pass data between the modules.
This proposal avoids the data copy in the READ/WRITE path
of filesystems, by providing a mechanism to share data buffers
between the modules. It is intended to be used by network file
sharing services like NFS, CIFS or others.
Although the buffer sharing can be achieved through a few different
solutions, any such solution must work with File Event Monitors
(FEM monitors)[1] installed on the files. The solution must
allow the underlying filesystem to maintain any existing file
range locking in the filesystem.
The proposed solution provides extensions to the existing VOP
interface to request and return buffers from a filesystem. The
buffers are then used with existing VOP_READ/VOP_WRITE calls with
minimal changes.
== Proposed Changes ==
VOP Extensions for Zero-Copy Support
a. Extended struct uio, xuio_t
The following proposes an extensible uio structure that can be extended
for
multiple purposes. For example, an immediate extension, xu_zc, is to be
used by the proposed VOP_REQZCBUF/VOP_RETZCBUF interfaces to pass loaned
zero-copy buffers, as well as to be passed to the existing
VOP_READ/VOP_WRITE
calls for normal read/write operations. Another example of extension,
xu_aio, is intended to replace uioa_t for async I/O.
This new structure, xuio_t, contains the following:
- the existing uio structure (embedded) as the first member
- additional fields to support extensibility
- a union of all the defined extensions
The following uio_extflag is added to indicate that an uio structure is
indeed an xuio_t:
#define UIO_XUIO0x004 /* Structure is xuio_t */
The following uio_extflag will be removed after uioa_t has been
converted
to xuio_t:
#define UIO_ASYNC 0x002 /* Structure is xuio_t */
The project team has commitment from the networking team to remove
the current use of uioa_t and use the proposed extensions (CR 6880095).
The definition of xuio_t is:
typedef struct xuio {
uio_t xu_uio; /* Embedded UIO structure */
/* Extended uio fields */
enum xuio_type xu_type;/* What kind of uio structure? */
union {
/* Async I/O Support */
struct {
uint32_t xu_a_state; /* state of async i/o */
uint32_t xu_a_state; /* state of async i/o */
ssize_t xu_a_mbytes; /* bytes that have been uioamove()ed */
uioa_page_t *xu_a_lcur;/* pointer into uioa_locked[] */
Copy Reduction Interfaces [PSARC/2009/478 FastTrack timeout 09/16/2009]
Rick Matthews wrote:
Are there instances where an assigned zero-copy buffer could be orphaned?
No. The consumer Must release the buffers through VOP_RETZCBUF().
Mahesh
If so, should there be a recovery list associated with this addition?
Perhaps off
the designated vnode.
This comment shouldn't block fast-track approval. Just a question.
--
Rick
On 09/ 9/09 04:02 PM, Rich.Brown at Sun.COM wrote:
I'm sponsoring this case on behalf of Mahesh Siddheshwar and Chunli
Zhang.
This case proposes new interfaces to support copy reduction in the
I/O path
especially for file sharing services.
Minor binding is requested.
This times out on Wednesday, 16 September, 2009.
Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI
This information is Copyright 2009 Sun Microsystems
1. Introduction
1.1. Project/Component Working Name:
Copy Reduction Interfaces
1.2. Name of Document Author/Supplier:
Author: Mahesh Siddheshwar, Chunli Zhang
1.3 Date of This Document:
09 September, 2009
4. Technical Description
== Introduction/Background ==
Zero-copy (copy avoidance) is essentially buffer sharing
among multiple modules that pass data between the modules. This
proposal avoids the data copy in the READ/WRITE path of filesystems,
by providing a mechanism to share data buffers
between the modules. It is intended to be used by network file
sharing services like NFS, CIFS or others.
Although the buffer sharing can be achieved through a few different
solutions, any such solution must work with File Event Monitors
(FEM monitors)[1] installed on the files. The solution must
allow the underlying filesystem to maintain any existing file range
locking in the filesystem.
The proposed solution provides extensions to the existing VOP
interface to request and return buffers from a filesystem. The
buffers are then used with existing VOP_READ/VOP_WRITE calls with
minimal changes.
== Proposed Changes ==
VOP Extensions for Zero-Copy Support
a. Extended struct uio, xuio_t
The following proposes an extensible uio structure that can be
extended for
multiple purposes. For example, an immediate extension, xu_zc, is
to be used by the proposed VOP_REQZCBUF/VOP_RETZCBUF interfaces to
pass loaned
zero-copy buffers, as well as to be passed to the existing
VOP_READ/VOP_WRITE
calls for normal read/write operations. Another example of extension,
xu_aio, is intended to replace uioa_t for async I/O.
This new structure, xuio_t, contains the following:
- the existing uio structure (embedded) as the first member
- additional fields to support extensibility
- a union of all the defined extensions
The following uio_extflag is added to indicate that an uio
structure is
indeed an xuio_t:
#defineUIO_XUIO0x004/* Structure is xuio_t */
The following uio_extflag will be removed after uioa_t has been
converted to xuio_t:
#defineUIO_ASYNC0x002/* Structure is xuio_t */
The project team has commitment from the networking team to remove
the current use of uioa_t and use the proposed extensions (CR
6880095).
The definition of xuio_t is:
typedef struct xuio {
uio_t xu_uio;/* Embedded UIO structure */
/* Extended uio fields */
enum xuio_type xu_type;/* What kind of uio structure? */
union {
/* Async I/O Support */
struct {
uint32_t xu_a_state;/* state of async i/o */
uint32_t xu_a_state;/* state of async i/o */
ssize_t xu_a_mbytes;/* bytes that have been
uioamove()ed */
uioa_page_t *xu_a_lcur;/* pointer into uioa_locked[] */
void **xu_a_lppp;/* pointer into lcur-uioa_ppp[] */
void *xu_a_hwst[4];/* opaque hardware state */
uioa_page_t xu_a_locked[UIOA_IOV_MAX]; /* Per iov
locked pages */
} xu_aio;
/* Zero Copy Support */
struct {
enum uio_rw xu_zc_rw;/* the use of the buffer */
void *xu_zc_priv;/* fs specific */
} xu_zc;
} xu_ext;
} xuio_t;
where xu_type is currently defined as:
typedef enum xuio_type {
UIOTYPE_ASYNCIO,
UIOTYPE_ZEROCOPY
} xuio_type_t;
New uio extensions can be added by defining a new xuio_type_t, and
adding a
new member to the xu_ext union.
b. Requesting zero-copy buffers
#define VOP_REQZCBUF(vp, rwflag, uiozcp, cr, ct) \
fop_reqzcbuf(vp, rwflag, uiozcp, cr, ct)
int fop_reqzcbuf(vnode_t *, enum uio_rw, xuio_t *, cred_t *,
caller_context_t *);
This function requests buffers associated with file vp in
preparation for a
subsequent zero copy read or write. The extended uio_t -- xuio_t
is used
to pass the parameters and results. Only the following fields of
xuio_t are
zpool recovery support [PSARC/2009/479 FastTrack timeout 09/16/2009]
This case was approved in today's PSARC meeting. -tim
Obsolescence of /usr/X11 [PSARC/2009/482 FastTrack timeout 09/17/2009]
Alan Coopersmith wrote: I am sponsoring this fast-track for myself, with a timeout of Sept. 17, and a release binding of patch (though delivery is only planned for OpenSolaris Solaris Next at this time). This was approved today during PSARC business. -- -Alan Coopersmith- alan.coopersmith at sun.com Sun Microsystems, Inc. - X Window System Engineering
PSARC 2009/480 Add bootpath into Solaris Sparc BootArchive for iSCSI boot
This case has failed to meet the obviousness requirements for self-review. At PSARC today, the agreement was to turn this into a fast track with a timer set for Sept 23. I've updated the IAM file to reflect this. - Garrett
Add bootpath into Solaris Sparc BootArchive for iSCSI boot [PSARC/2009/480 Self Review]
David Kahn wrote: Bart Smaalders wrote: Tarl Neustaedter wrote: The knowledge of which driver to use within Solaris isn't needed until after this stage - so I don't think it belongs in the boot command in the first place. This is correct... and storing such driver info somewhere where a special program needs to update it is just plain broken. Solaris should be passed the _external_ info about its boot drive - and then should dynamically figure out how to talk to that external device. Any other approach will break when drivers change, paths to devices change, etc. That's what the FWARC case does today, so if you are agreeing with that, then I agree with you. Is that what you mean, Bart? Yes I'm working on the new packaging system for Solaris next, and we don't use an installer or upgrade program; instead, changes are made to the delivered files as specified in the packages. There is no release-specific code that gets run to perform arbitrary fixups to config files, such as determining kernel-internal boot paths, etc. Today, w/ ZFS root (mandatory in Solaris next for a variety of reasons) this works well on both SPARC and x86 although USB connected storage devices still need some help. The first iscsi proposal required an installer program (running on the previous release being upgraded via zfs clone) to compute the new kernel's internal boot paths so that the new kernel could boot. Needless to say, this would create significant technical and management issues; since we manage to boot a ubiquitous CD image, dynamically discovering paths to that device after initial image load, I believe that we can and should do the same for iscsi boot devices. - Bart -- Bart Smaalders Solaris Kernel Performance barts at cyber.eng.sun.com http://blogs.sun.com/barts You will contribute more with mercurial than with thunderbird.
Copy Reduction Interfaces [PSARC/2009/478 FastTrack timeout 09/16/2009]
My issues have been resolved. Thanks Mahesh.
-r
Rich.Brown at Sun.COM writes:
I'm sponsoring this case on behalf of Mahesh Siddheshwar and Chunli Zhang.
This case proposes new interfaces to support copy reduction in the I/O path
especially for file sharing services.
Minor binding is requested.
This times out on Wednesday, 16 September, 2009.
Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI
This information is Copyright 2009 Sun Microsystems
1. Introduction
1.1. Project/Component Working Name:
Copy Reduction Interfaces
1.2. Name of Document Author/Supplier:
Author: Mahesh Siddheshwar, Chunli Zhang
1.3 Date of This Document:
09 September, 2009
4. Technical Description
== Introduction/Background ==
Zero-copy (copy avoidance) is essentially buffer sharing
among multiple modules that pass data between the modules.
This proposal avoids the data copy in the READ/WRITE path
of filesystems, by providing a mechanism to share data buffers
between the modules. It is intended to be used by network file
sharing services like NFS, CIFS or others.
Although the buffer sharing can be achieved through a few different
solutions, any such solution must work with File Event Monitors
(FEM monitors)[1] installed on the files. The solution must
allow the underlying filesystem to maintain any existing file
range locking in the filesystem.
The proposed solution provides extensions to the existing VOP
interface to request and return buffers from a filesystem. The
buffers are then used with existing VOP_READ/VOP_WRITE calls with
minimal changes.
== Proposed Changes ==
VOP Extensions for Zero-Copy Support
a. Extended struct uio, xuio_t
The following proposes an extensible uio structure that can be extended for
multiple purposes. For example, an immediate extension, xu_zc, is to be
used by the proposed VOP_REQZCBUF/VOP_RETZCBUF interfaces to pass loaned
zero-copy buffers, as well as to be passed to the existing
VOP_READ/VOP_WRITE
calls for normal read/write operations. Another example of extension,
xu_aio, is intended to replace uioa_t for async I/O.
This new structure, xuio_t, contains the following:
- the existing uio structure (embedded) as the first member
- additional fields to support extensibility
- a union of all the defined extensions
The following uio_extflag is added to indicate that an uio structure is
indeed an xuio_t:
#defineUIO_XUIO0x004 /* Structure is xuio_t */
The following uio_extflag will be removed after uioa_t has been converted
to xuio_t:
#defineUIO_ASYNC 0x002 /* Structure is xuio_t */
The project team has commitment from the networking team to remove
the current use of uioa_t and use the proposed extensions (CR 6880095).
The definition of xuio_t is:
typedef struct xuio {
uio_t xu_uio;/* Embedded UIO structure */
/* Extended uio fields */
enum xuio_type xu_type; /* What kind of uio structure? */
union {
/* Async I/O Support */
struct {
uint32_t xu_a_state; /* state of async i/o */
uint32_t xu_a_state; /* state of async i/o */
ssize_t xu_a_mbytes; /* bytes that have been uioamove()ed */
uioa_page_t *xu_a_lcur; /* pointer into uioa_locked[] */
void **xu_a_lppp;/* pointer into
lcur-uioa_ppp[] */
void *xu_a_hwst[4]; /* opaque hardware state */
uioa_page_t xu_a_locked[UIOA_IOV_MAX]; /* Per iov locked pages
*/
} xu_aio;
/* Zero Copy Support */
struct {
enum uio_rw xu_zc_rw;/* the use of the buffer */
void *xu_zc_priv;/* fs specific */
} xu_zc;
} xu_ext;
} xuio_t;
where xu_type is currently defined as:
typedef enum xuio_type {
UIOTYPE_ASYNCIO,
UIOTYPE_ZEROCOPY
} xuio_type_t;
New uio extensions can be added by defining a new xuio_type_t, and adding a
new member to the xu_ext union.
b. Requesting zero-copy buffers
#define VOP_REQZCBUF(vp, rwflag, uiozcp, cr, ct) \
fop_reqzcbuf(vp, rwflag, uiozcp, cr, ct)
int fop_reqzcbuf(vnode_t *, enum uio_rw, xuio_t *, cred_t *,
caller_context_t *);
This function requests buffers associated with file vp in preparation
for a
subsequent zero copy read or write. The extended uio_t -- xuio_t is used
to pass the parameters and results. Only the following fields of xuio_t
are
relevant to this call.
uiozcp-xu_uio.uio_resid: used by the caller to specify the total length
of the buffer.
uiozcp-xu_uio.uio_loffset:
PSARC 2009/476 Public linked lists
This case was approved at PSARC today. - Garrett
OpenSolaris ARC Minutes - 09/15/2009
SYSTEM ARCHITECTURE COUNCIL Layered Software ARC - 09-15-2009 MEETING MINUTES Send CORRECTIONS, additions, deletions to lsarc-coord at sun.com. Minutes are archived in /shared/sac/Minutes/LSARC. ATTENDEES: Chair Mark CarlsonYes Members (8 active members) Lloyd Chambers no Aniruddh Dikhit no Danek Duvall(on sabbatical) John FischerYes Ed Hunter Yes (Acted as Chair) Chris Kasso (on sabbatical) Arieh Markel(on sabbatical) Margot Miller Yes Jyri Virkki Yes Michael Kearney Yes Tom Childersno (external) Mark Martin Yes (external) Interns Brian Cameron Yes James Gates no Matthias Huetschno Linda Schneider no Rahul Shah no Krishna Tallapaneni no David Tong no James Walkerno Jeff Caino Coordinator Asa Romberger (PM) Yes Guest Not all names are captured. Please send email to Asa.Romberger at Sun.com, if you attended the meeting and your name was not captured. == AGENDA == Case Anchors: br A HREF=#case1GutenPrint Ver 5.2.4 (2009/469)/A br == ARC Business Case (Timeout) Exposure Title 2009/472 (09/04/09) open Crypt-CBC Perl Module approved 2009/475 (09/15/09) open GNOME 2.28 approved 2009/481 (09/17/09) open Python Routes let it run 2009/489 (09/21/09) open Synergy - Mouse/Keyboard sharing let it run Opinions Other stuff --- Next Meeting -- 09/22/2009 No meeting --- --- 2009/469 Name: Gutenprint Ver 5.2.4 Submitter: Gowtham Thommandra Owner: James Gates Exposure: open SUMMARY === This project replaces gimp-print (version 4.2.3) with the latest version, now called Gutenprint (version 5.2.4). ISSUES == THE NEXT STEP = The Submitter and Owner did not show. The case at least needs a note that the missing 64-bit libraries are an exception and do not represent a precident. In addition, there is a .a file that needs to be removed or a defect logged to have it removed
Copy Reduction Interfaces [PSARC/2009/478 FastTrack timeout 09/16/2009]
This case was approved at today's PSARC meeting. I put an updated final_spec.txt in the case directory which corrects a typo that Mahesh found. On behalf of the team, thank you for your time and assistance on this case. Rich
OpenSolaris ARC Minutes - 09/16/2009
SYSTEM ARCHITECTURE COUNCIL Platform Software ARC - PSARC Regular Meeting time: Wednesdays 10:00-1:00pm in MPK17-3507. 09-16-2009 MEETING MINUTES Send CORRECTIONS, additions, deletions to psarc-coord at sun.com. Minutes are archived in sac.Eng:/sac/export/sac/Minutes/PSARC. Co-Chair(s): Garrett D'Amore:Yes Tim Marsland: Yes ATTENDEES - Members: (6 active members) Kais Belgaied: Yes Mark Carlson: no Richard Matthews: Yes Darren Moffat: no (on sabbatical) Sebastien Roy: Yes Glenn Skinner: Yes Bill Sommerfeld:no (on sabbatical) Gary Winiger: Yes (on sabbatical) STAFF - Asa Romberger (PM): Yes ATTENDEES - Interns: Frank Che no James Falkner: no (on sabbatical) Daniel Hain:no Michael Haines: no Alan Hargreaves:no Phil Harman:no Wyllys Ingersoll: no Darren Reed:no Dean Roehrich no Ienup Sung: no Phi TranYes Brian Utterback:no James WalkerYes Suhasini PeddadaYes Calum MackayYes Mark Martin no (external) Don Cragun Yes (external) Guests: -- GUESTS -- Rich Brown Siddheshwar Mahesh Tim Haley Nicolas Droux Alan Coopersmith Not all names are captured. Please send email to Asa.Romberger at Sun.com, if you attended the meeting and your name is missing from the list. --- MEETING SUMMARY: AGENDA --- Case Anchors: br A HREF=#case1Public GLDv3 Interfaces (2009/473)/A br Case withdrawn === Fast Tracks: Case (Timeout) Exposure Title 2009/476 (09/15/09) open Public linked lists approved 2009/477 (09/15/09) open Addition of NE_IFINDEX_CHANGE to sys/neti.h approved 2009/478 (09/16/09) open Copy Reduction Interfaces approved 2009/479 (09/16/09) open zpool recovery support approved 2009/482 (09/17/09) open Obsolescence of /usr/X11 approved 2009/483 (09/18/09) open libxklavier re-integration approved 2009/488 (09/21/09) open flowadm(1m) remote_port flow attribute let it run 2009/490 (09/22/09) open x86 Generic FMA Topology Enumerator let it run 2009/480open Add bootpath into Solaris Sparc BootArchive for iSCSI boot derail - turn it into a fast track timing out 09/23/2009 Next Meeting: = 09/23/2009 10:00-10:10 Open ARC Business (use open dial in above) 10:10-10:55 Commitment: S10C (2009/253) Submitter: Gerald Jelinek Owner: Rick Matthews Intern: Dean Roehrich Exposure: open 10:55-11:40 Inception Solaris ATCA IPMI Driver (2009/467) Submitter: Kevin Song Owner: Garrett D'Amore Intern: Jim Walker Exposure: open 11:45-11:55 Closed ARC Business (use closed dial in above)
64-bit only projects?
If an unbundled project wants to deliver only 64-bit binaries on sparc, should the 64-bit binaries go in the top-level bin directory? Or into bin/sparcv9 with a symlink pointing down to it? Or something else? I didn't see any examples of this in /usr/bin on a sparc lab machine with S10. Has anyone done this yet? The SPARC compilers might want to do this at some point. Pointers to past ARC cases with an email log would be great. --chris
zstreamdump [PSARC/2009/491 Self Review]
On 09/15/09 18:08, Nicolas Williams wrote: On Tue, Sep 15, 2009 at 06:06:13PM -0600, Tim Haley wrote: NAME zstreamdump - dump the metadata in a ZFS send stream SYNOPSIS zstreamdump [-vC] DESCRIPTION The zstreamdump command reads a ZFS send stream from stdin and prints out selected fields from the metadata to stdout. By default, it also validates the checksums for each of the dataset streams embedded in the overall send stream. A summary of the stream contents is printed at the end. OPTIONS The following options are supported: -v Verbose mode. All records are dumped. Default is to only dump the metadata from the BEGIN and END records. -C Suppress the validation of checksums. Causes the tool to run faster. It'd be nice if zfs recv could do this too, no? Alternatively, it'd be nice if there was a way to slot zstreamdump into a zfs send | ... | zfs recv pipe. E.g., an option to zstreamdump to send its output to a file or to stderr, passing the stream through to stdout for piping elsewhere. This is a good idea, but I'd rather leave it as is. This is just a rather simple and crude dumping tool. I don't want anyone assuming that this is more of a 'stream verification' tool than it really is. lori
64-bit only projects?
Chris Quenelle wrote: If an unbundled project wants to deliver only 64-bit binaries on sparc, should the 64-bit binaries go in the top-level bin directory? Or into bin/sparcv9 with a symlink pointing down to it? Or something else? I didn't see any examples of this in /usr/bin on a sparc lab machine with S10. Has anyone done this yet? The SPARC compilers might want to do this at some point. It's not in /usr/bin yet (the ARC case to move it there was just approved this morning), but /usr/X11/bin/Xorg is a simple 64-bit binary on SPARC. (On x86, it's isaexec'ed.) -- -Alan Coopersmith- alan.coopersmith at sun.com Sun Microsystems, Inc. - X Window System Engineering
EOF of XRESOLV [PSARC/2009/496 FastTrack timeout 09/23/2009]
I'm submitting this fast-track for Sowmini Varadhan. It times out on 09/23/2009. The release binding is Minor. This case proposes to announce the obsolescence of support for code associated with the IFF_XRESOLV interface flag in the next patch or micro release, and remove it and associated kernel code from the next minor release. Details: The IFF_XRESOLV flag was introduced as part of PSARC 1999/446 to support external address resolvers for ATM devices. The Interfaces proposed by PSARC 1999/446 were subsequently modified by PSARC 2001/023, so that the IFF_XRESOLV flag was marked Evolving, and had to be set by the ATM device using private ioctls (SIOCSLIFNAME) interface as opposed to ifconfig(1m). The ATM driver was EOL'ed as part of PSARC 2006/272 removing the sole consumer of the Evolving IFF_XRESOLV interface, and leaving behind a mass of untestable, possibly buggy code in the Solaris kernel TCP/IP modules. As a result, the current state of the code is that nothing in Solaris can set IFF_XRESOLV, though ifconfig(1m) has the facility to display it as part of the output for interface flags. The proposal is to remove all code associated with IFF_XRESOLV, and adjust the documentation to reflect this removal. Affected Interfaces: Interfaces removed by this case - Interface Classification Comments - IFF_XRESOLV Committed 2001/023, 2006/272 - Man page updates: --- ifconfig.oldTue Sep 15 15:05:38 2009 +++ ifconfig.newTue Sep 15 15:05:52 2009 @@ -1551,12 +1551,7 @@ option.) - XRESOLV -Indicates that the interface uses an IPv6 external -resolver. - - LOGICAL INTERFACES Solaris TCP/IP allows multiple logical interfaces to be associated with a physical network interface. This allows a
EOF of XRESOLV [PSARC/2009/496 FastTrack timeout 09/23/2009]
There's also a reference to IFF_XRESOLV in if_tcp(7P) that will need to be removed. -- meem
64-bit only projects? (and /usr/lib/locale)
Thanks guys! I've got a follow-up. What about /usr/lib/locale vs /usr/share/locale? Is /usr/lib/locale only for ON and /usr/share/locale for everyone else? Is that documented anywhere? I've seen the filesystem(5) man page and the SAC best practices document. (http://sac.sfbay/cgi-bin/bp.cgi?NAME=install_locations.bp) If anyone else knows of guidelines for the right place to install different kinds of files, please send along some pointers. --chris James Carlson wrote: I'd just put them in /usr/bin without fanfare. SPARC no longer has a supported platform where 32-bit kernels run, so there's no real need for isaexec shenanigans there. Alan Coopersmith wrote: It's not in /usr/bin yet (the ARC case to move it there was just approved this morning), but /usr/X11/bin/Xorg is a simple 64-bit binary on SPARC. (On x86, it's isaexec'ed.)
EOF of XRESOLV [PSARC/2009/496 FastTrack timeout 09/23/2009]
Do we believe that there will never be another media layer which requires this sort of external resolver? (Or that ATM is dead and gone from Solaris forever?) If so, then I'll be happy to support this since I hate leaving around dead/untested code. But I'm worried that this one might be a tad premature. Of course, if all the world is 802, then it probably is indeed not needed. - Garrett Sebastien Roy wrote: I'm submitting this fast-track for Sowmini Varadhan. It times out on 09/23/2009. The release binding is Minor. This case proposes to announce the obsolescence of support for code associated with the IFF_XRESOLV interface flag in the next patch or micro release, and remove it and associated kernel code from the next minor release. Details: The IFF_XRESOLV flag was introduced as part of PSARC 1999/446 to support external address resolvers for ATM devices. The Interfaces proposed by PSARC 1999/446 were subsequently modified by PSARC 2001/023, so that the IFF_XRESOLV flag was marked Evolving, and had to be set by the ATM device using private ioctls (SIOCSLIFNAME) interface as opposed to ifconfig(1m). The ATM driver was EOL'ed as part of PSARC 2006/272 removing the sole consumer of the Evolving IFF_XRESOLV interface, and leaving behind a mass of untestable, possibly buggy code in the Solaris kernel TCP/IP modules. As a result, the current state of the code is that nothing in Solaris can set IFF_XRESOLV, though ifconfig(1m) has the facility to display it as part of the output for interface flags. The proposal is to remove all code associated with IFF_XRESOLV, and adjust the documentation to reflect this removal. Affected Interfaces: Interfaces removed by this case - Interface Classification Comments - IFF_XRESOLV Committed 2001/023, 2006/272 - Man page updates: --- ifconfig.oldTue Sep 15 15:05:38 2009 +++ ifconfig.newTue Sep 15 15:05:52 2009 @@ -1551,12 +1551,7 @@ option.) - XRESOLV -Indicates that the interface uses an IPv6 external -resolver. - - LOGICAL INTERFACES Solaris TCP/IP allows multiple logical interfaces to be associated with a physical network interface. This allows a
64-bit only projects? (and /usr/lib/locale)
There is no specific restrictions or guidelines but usually localization files also go to the same file system where your software goes and be populated underneath the top directory of your software's file system hierarchy. If you're delivering into /usr/, for instance, you can populate your localization files under /usr/lib/locale/locale/ and, for 64-bit specific items, under /usr/lib/locale/locale/native inst set name/. For the message files, you just populate one set for both 32-bit and 64-bit executables. Message/L10N files from FOSS including GNOME usually go to /usr/share/locale/locale/LC_MESSAGES/ but not always. For more details on the locations, please see man pages for gettext(3C), catopen(3C), environ(5), and localedef(1). If you have any further questions or need assistance, please contact i18n-discuss at opensolaris.org mailing list. Ienup Chris Quenelle wrote at 09/16/09 15:23: Thanks guys! I've got a follow-up. What about /usr/lib/locale vs /usr/share/locale? Is /usr/lib/locale only for ON and /usr/share/locale for everyone else? Is that documented anywhere? I've seen the filesystem(5) man page and the SAC best practices document. (http://sac.sfbay/cgi-bin/bp.cgi?NAME=install_locations.bp) If anyone else knows of guidelines for the right place to install different kinds of files, please send along some pointers. --chris James Carlson wrote: I'd just put them in /usr/bin without fanfare. SPARC no longer has a supported platform where 32-bit kernels run, so there's no real need for isaexec shenanigans there. Alan Coopersmith wrote: It's not in /usr/bin yet (the ARC case to move it there was just approved this morning), but /usr/X11/bin/Xorg is a simple 64-bit binary on SPARC. (On x86, it's isaexec'ed.)
zfs checksum ereport payload additions [PSARC/2009/497 Self Review]
I am sponsoring the following self review case for Jonathan Adams. This case adds some fields to the ZFS checksum ereports. An FMA portfolio for the changes has been submitted and approved. This case seeks micro/patch binding. As this is a small backwards-compatible modification to the ZFS FMA ereports, and has been reviewed by the FMA events C-team and the ZFS team, the case is being filed as self-review. If any ARC members have questions or believe the case is above the automatic approval threshold, please let us know. Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: zfs checksum ereport payload additions 1.2. Name of Document Author/Supplier: Author: Jonathan Adams 1.3 Date of This Document: 16 September, 2009 4. Technical Description These changes will allow us to improve future diagnosis of and recovery from data corruption. FMA Portfolio Case -- 2009/019 zfs_cksum http://wikihome.sfbay.sun.com/fma-portfolio/Wiki.jsp?page=2009.019.zfs_cksum Exported Interface Stability --- --- ereport.fs.zfs.checksum Sun Private The ercheck summary detailing the error report is also included in the case directory. See also PSARC/2006/139 FMA for ZFS PSARC/2007/283 FMA for ZFS Phase 2 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: Automatic 6.6. ARC Exposure: open
EOF of XRESOLV [PSARC/2009/496 FastTrack timeout 09/23/2009]
Garrett D'Amore wrote: Do we believe that there will never be another media layer which requires this sort of external resolver? (Or that ATM is dead and gone from Solaris forever?) If so, then I'll be happy to support this since I hate leaving around dead/untested code. But I'm worried that this one might be a tad premature. I think it is similar to the Mobile IPv4 removal we did a while back. While it might come back, the fact that nothing uses it, it can't be tested any more, it adds significant complexity to the code. Thus if we need something form of external resolver for a new IP-over-foo, then we have to built it from scratch anyway. Erik Of course, if all the world is 802, then it probably is indeed not needed. - Garrett Sebastien Roy wrote: I'm submitting this fast-track for Sowmini Varadhan. It times out on 09/23/2009. The release binding is Minor. This case proposes to announce the obsolescence of support for code associated with the IFF_XRESOLV interface flag in the next patch or micro release, and remove it and associated kernel code from the next minor release. Details: The IFF_XRESOLV flag was introduced as part of PSARC 1999/446 to support external address resolvers for ATM devices. The Interfaces proposed by PSARC 1999/446 were subsequently modified by PSARC 2001/023, so that the IFF_XRESOLV flag was marked Evolving, and had to be set by the ATM device using private ioctls (SIOCSLIFNAME) interface as opposed to ifconfig(1m). The ATM driver was EOL'ed as part of PSARC 2006/272 removing the sole consumer of the Evolving IFF_XRESOLV interface, and leaving behind a mass of untestable, possibly buggy code in the Solaris kernel TCP/IP modules. As a result, the current state of the code is that nothing in Solaris can set IFF_XRESOLV, though ifconfig(1m) has the facility to display it as part of the output for interface flags. The proposal is to remove all code associated with IFF_XRESOLV, and adjust the documentation to reflect this removal. Affected Interfaces: Interfaces removed by this case - Interface Classification Comments - IFF_XRESOLV Committed 2001/023, 2006/272 - Man page updates: --- ifconfig.oldTue Sep 15 15:05:38 2009 +++ ifconfig.newTue Sep 15 15:05:52 2009 @@ -1551,12 +1551,7 @@ option.) - XRESOLV -Indicates that the interface uses an IPv6 external -resolver. - - LOGICAL INTERFACES Solaris TCP/IP allows multiple logical interfaces to be associated with a physical network interface. This allows a
EOF of XRESOLV [PSARC/2009/496 FastTrack timeout 09/23/2009]
Erik Nordmark wrote: Garrett D'Amore wrote: Do we believe that there will never be another media layer which requires this sort of external resolver? (Or that ATM is dead and gone from Solaris forever?) If so, then I'll be happy to support this since I hate leaving around dead/untested code. But I'm worried that this one might be a tad premature. I think it is similar to the Mobile IPv4 removal we did a while back. While it might come back, the fact that nothing uses it, it can't be tested any more, it adds significant complexity to the code. Thus if we need something form of external resolver for a new IP-over-foo, then we have to built it from scratch anyway. Okay, with that, +1. - Garrett Erik Of course, if all the world is 802, then it probably is indeed not needed. - Garrett Sebastien Roy wrote: I'm submitting this fast-track for Sowmini Varadhan. It times out on 09/23/2009. The release binding is Minor. This case proposes to announce the obsolescence of support for code associated with the IFF_XRESOLV interface flag in the next patch or micro release, and remove it and associated kernel code from the next minor release. Details: The IFF_XRESOLV flag was introduced as part of PSARC 1999/446 to support external address resolvers for ATM devices. The Interfaces proposed by PSARC 1999/446 were subsequently modified by PSARC 2001/023, so that the IFF_XRESOLV flag was marked Evolving, and had to be set by the ATM device using private ioctls (SIOCSLIFNAME) interface as opposed to ifconfig(1m). The ATM driver was EOL'ed as part of PSARC 2006/272 removing the sole consumer of the Evolving IFF_XRESOLV interface, and leaving behind a mass of untestable, possibly buggy code in the Solaris kernel TCP/IP modules. As a result, the current state of the code is that nothing in Solaris can set IFF_XRESOLV, though ifconfig(1m) has the facility to display it as part of the output for interface flags. The proposal is to remove all code associated with IFF_XRESOLV, and adjust the documentation to reflect this removal. Affected Interfaces: Interfaces removed by this case - Interface Classification Comments - IFF_XRESOLV Committed 2001/023, 2006/272 - Man page updates: --- ifconfig.oldTue Sep 15 15:05:38 2009 +++ ifconfig.newTue Sep 15 15:05:52 2009 @@ -1551,12 +1551,7 @@ option.) - XRESOLV -Indicates that the interface uses an IPv6 external -resolver. - - LOGICAL INTERFACES Solaris TCP/IP allows multiple logical interfaces to be associated with a physical network interface. This allows a
64-bit only projects? (and /usr/lib/locale)
Ienup, Thanks very much for the explanation and the pointers. I'll read the man pages, and followup to i18n-discuss if I have more questions. --chris Ienup Sung wrote: There is no specific restrictions or guidelines but usually localization files also go to the same file system where your software goes and be populated underneath the top directory of your software's file system hierarchy. If you're delivering into /usr/, for instance, you can populate your localization files under /usr/lib/locale/locale/ and, for 64-bit specific items, under /usr/lib/locale/locale/native inst set name/. For the message files, you just populate one set for both 32-bit and 64-bit executables. Message/L10N files from FOSS including GNOME usually go to /usr/share/locale/locale/LC_MESSAGES/ but not always. For more details on the locations, please see man pages for gettext(3C), catopen(3C), environ(5), and localedef(1). If you have any further questions or need assistance, please contact i18n-discuss at opensolaris.org mailing list. Ienup Chris Quenelle wrote at 09/16/09 15:23: Thanks guys! I've got a follow-up. What about /usr/lib/locale vs /usr/share/locale? Is /usr/lib/locale only for ON and /usr/share/locale for everyone else? Is that documented anywhere? I've seen the filesystem(5) man page and the SAC best practices document. (http://sac.sfbay/cgi-bin/bp.cgi?NAME=install_locations.bp) If anyone else knows of guidelines for the right place to install different kinds of files, please send along some pointers. --chris James Carlson wrote: I'd just put them in /usr/bin without fanfare. SPARC no longer has a supported platform where 32-bit kernels run, so there's no real need for isaexec shenanigans there. Alan Coopersmith wrote: It's not in /usr/bin yet (the ARC case to move it there was just approved this morning), but /usr/X11/bin/Xorg is a simple 64-bit binary on SPARC. (On x86, it's isaexec'ed.)
EOF of XRESOLV [PSARC/2009/496 FastTrack timeout 09/23/2009]
On (09/16/09 14:14), Peter Memishian wrote: There's also a reference to IFF_XRESOLV in if_tcp(7P) that will need to be removed. Good point. I'll make sure it's addressed. For the record, the diffs would be --- if_tcp.old Wed Sep 16 21:04:13 2009 +++ if_tcp.new Wed Sep 16 21:04:32 2009 @@ -723,8 +723,6 @@ used */ #define IFF_OFFLINE 0x008000/* Interface is offline */ - #define IFF_XRESOLV 0x01/* IPv6 external resolver */ #define IFF_COS_ENABLED 0x02/* If CoS marking is supported */
Add bootpath into Solaris Sparc BootArchive for iSCSI boot [PSARC/2009/480 Self Review]
I just submitted FWARC/2009/498 for Tarl, which adds a new /chosen property, iscsi-tpgt. Apparently, the information was already available to OF, so it was easy enough to publish it, even though we don't need it, and in theory anybody could do the same thing that OF does to get it. Nonetheless, OF can easily publish it for the OS if they need it. It doesn't affect the OF boot path or encode any OS implementation specific info in the property, so we're fine with it. Does that solve the issues that were in this case? Do they still need this case? -David
Add bootpath into Solaris Sparc BootArchive for iSCSI boot [PSARC/2009/480 Self Review]
I never said that the OBP should know which driver Solaris should use. I said that the choice of driver should: a) be defaulted, b) be settable via boot archive and/or boot option. Administering a choice of driver via boot archive seems reasonable to me. Storing the full boot path in the boot archive is not (since it is redundant, causing yet one more place to update when a system's root migrates). I don't know if you solved the last piece of this or not. We (the firmware) simply export a pseudo path for iscsi boot, which is /iscsi-hba/disk the disk node has a compatible property, which I'm told currently contains the value sd. We could probably change that to something like SUNW,iscsi-disk, and the OS can bind that string to whatever driver you want to in /etc/driver_aliases. (either bundled or via add_drv). If you want us to change that, we can ask Tarl to do it, and we'll submit another fast-track case for that change. But that is not something that is or should be settable in the firmware at least to the normal used. I can't imagine a user friendly system where the user has to configure the name of the OS driver in the firmware that they want to use. How would they know that? Nonetheless, it seems like the wrong thing to want to do if you ask me. We don't specify OS driver bindings in the firmware. We do provide compatible property value definitions with default values as specified in specific bus bindings, based on standards. The driver binding is always left to the OS. That's worked for us for a long time, and it's hard to envision a case now where it doesn't work for us, as long as devices provide some unique string that the OS can use to bind to a device driver name. If you want something besides sd in the /iscsi-hba/disk compatible propval, we can do that. Other than that, I'm not sure we can help with the driver binding issue. -David
