[CFR] Forward RTO recovery algorithm (rfc5682) patch
This patch is *VERY* experimental patch to implement rfc5862 which is on
the IETF standard tracks so it could be completely wrong or has a lot of
bugs on it or wrong approaches because I'm a really newbie on TCP stack.
This patch includes two features to support `Basic FRTO algorithm' and
`SACK-Enhanced FRTO algorithm' but not including a feature to support
things mentioned at `Appendix A'.
I'm looking for reviewers teaching me where it's wrong or what the
approach was bad on line by line. However any comments are welcome.
The patch is available at:
http://people.freebsd.org/~weongyo/patch_tcpfrto_20110205.diff
regards,
Weongyo Jeong
Index: netinet/tcp_input.c
===
--- netinet/tcp_input.c (revision 218148)
+++ netinet/tcp_input.c (working copy)
@@ -161,6 +161,11 @@
VNET_NAME(tcp_abc_l_var), 2,
Cap the max cwnd increment during slow-start to this number of segments);
+VNET_DEFINE(int, tcp_do_rfc5682) = 1;
+SYSCTL_VNET_INT(_net_inet_tcp, OID_AUTO, rfc5682, CTLFLAG_RW,
+VNET_NAME(tcp_do_rfc5682), 0,
+Enable RFC 5682 (Forward RTO-Recovery: F-RTO));
+
SYSCTL_NODE(_net_inet_tcp, OID_AUTO, ecn, CTLFLAG_RW, 0, TCP ECN);
VNET_DEFINE(int, tcp_do_ecn) = 0;
@@ -254,6 +259,212 @@
}
}
+/* revert to the conventional RTO recovery. */
+#define TCP_FRTO_REVERT(tp) do { \
+ (tp)-frto_flags = 0; \
+} while (0)
+
+static int
+tcp_frto_send2mss(struct tcpcb *tp, struct tcphdr *th, uint16_t type)
+{
+ struct inpcb *inp = tp-t_inpcb;
+ struct socket *so = inp-inp_socket;
+ u_long oldcwnd;
+ tcp_seq onxt;
+
+ /*
+ * XXXWG If the TCP sender does not have any new data to send, OR the
+ * advertised window prohibits new transmissions, the recommended
+ * action is to skip step 3 of this algorithm and continue with
+ * slow-start retransmissions, following the conventional RTO recovery
+ * algorithm. However, alternative ways of handling the window-limited
+ * cases that could result in better performance are discussed in
+ * Appendix A.
+ */
+ if (so-so_snd.sb_cc == 0 || tp-snd_wnd == 0)
+ /* XXXWG skip step 3 OR do alternative ways. */
+ return (1);
+ oldcwnd = tp-snd_cwnd;
+ onxt = tp-snd_nxt;
+ /*
+ * XXXWG transmit up to two new (previously unsent) segments and enter
+ * step 3 of this algorithm. If the TCP sender does not have enough
+ * unsent data, it can send only one segment.
+ */
+ tp-snd_cwnd = 2 * tp-t_maxseg;
+ tp-snd_nxt = tp-snd_max;
+ /*
+ * XXXWG in addition, the TCP sender MAY override the Nagle algorithm.
+ */
+ tp-t_flags |= TF_ACKNOW;
+ (void) tcp_output(tp);
+ tp-snd_nxt = onxt;
+ tp-snd_cwnd = oldcwnd;
+ return (0);
+}
+
+static void inline
+tcp_frto_ack_received(struct tcpcb *tp, struct tcphdr *th, uint16_t type)
+{
+
+ /* SACK-Enhanced F-RTO */
+
+ if (tp-t_flags TF_SACK_PERMIT) {
+ if ((tp-frto_flags FRTO_INSTEP1) != 0
+ (tp-frto_flags FRTO_CANGOSTEP2) != 0) {
+ /* SACK-Enhanced F-RTO - step 2 */
+ if (type == CC_DUPACK)
+/*
+ * 2) If duplicate ACKs arrive before the
+ * cumulative acknowledgment for retransmitted
+ * data, adjust the scoreboard according to
+ * the incoming SACK information. Stay in
+ * step 2 and wait for the next new
+ * acknowledgment.
+ */
+return;
+ KASSERT(type == CC_ACK,
+ (%s: expected ACK (but %d), __func__, type));
+
+ /*
+ * 2) When a new acknowledgment arrives, set variable
+ * RecoveryPoint to indicate the highest sequence
+ * number transmitted so far.
+ */
+ tp-frto_flags |= FRTO_INSTEP2;
+ tp-snd_recover = tp-snd_max;
+
+ /*
+ * 2a) If the Cumulative Acknowledgement field covers
+ * RecoveryPoint but not more than
+ * RecoveryPoint, revert to the conventional RTO
+ * recovery and set the congestion window to no
+ * more than 2 * MSS, like a regular TCP would do.
+ * Do not enter step 3 of this algorithm.
+ */
+ if (th-th_ack == tp-snd_recover) {
+tp-snd_cwnd = 2 * tp-t_maxseg;
+TCP_FRTO_REVERT(tp);
+return;
+ /*
+ * 2b) If the Cumulative Acknowledgment field does not
+ * cover RecoveryPoint but is larger than
+ * SND.UNA.
+ */
+ } else if (SEQ_LT(th-th_ack, tp-snd_recover)
+ SEQ_GT(th-th_ack, tp-snd_una)
+ !tcp_frto_send2mss(tp, th, type)) {
+tp-frto_flags |= FRTO_CANGOSTEP3;
+tp-frto_fack = tp-snd_fack;
+ }
+ }
+ if ((tp-frto_flags FRTO_INSTEP2) != 0
+ (tp-frto_flags FRTO_CANGOSTEP3) != 0) {
+ struct sackhole *q = TAILQ_FIRST(tp-snd_holes);
+ /* the Cumulative Acknowledgment */
+ tcp_seq cack = SEQ_MAX(th-th_ack, tp-snd_una);
+
+ /*
+ * XXXWG 3a) If the Cumulative Acknowledgment field or
+ * the SACK information covers more than
+ * RecoveryPoint.
+ */
+ if (SEQ_GEQ(cack, tp-snd_recover) ||
+ (q != NULL SEQ_GT(q-start, tp-snd_recover)))
+goto frto_revert;
+ /*
+ * XXXWG 3a) take this branch also when the
+ * acknowledgment is a duplicate ACK and
libdialog wide charcter support (Was: Request for testing/comments -- import of new dialog/libdialog)
Nathan Whitehorn nwhiteh...@freebsd.org writes: As part of work on a new installer, I would like to update the base system dialog and libdialog to the newer one provided by Thomas Dickey (http://invisible-island.net/dialog/, ports as devel/cdialog). This is a much nicer, fuller featured version of dialog that simplifies the creation of new dialog-using tools (a longstanding impediment to a new versions of sade, sysinstall, etc.), and is under a marginally better license (LGPL2 instead of GPL2). Patches to effect the import can be found at: - http://people.freebsd.org/~nwhitehorn/libdialog-update.diff Unlike devel/cdialog your diff (and r217309) has wide character support disabled. Was this intentional? For example, without it for each UTF-8 character there'd be extra space padding in editobox, inputbox, fselect, dselect, etc. %% Index: gnu/lib/libdialog/dlg_config.h === --- gnu/lib/libdialog/dlg_config.h (revision 218313) +++ gnu/lib/libdialog/dlg_config.h (working copy) @@ -89,3 +89,5 @@ #define SYSTEM_NAME freebsd9.0 #define TIME_WITH_SYS_TIME 1 #define TYPE_CHTYPE_IS_SCALAR 1 +#define USE_WIDE_CURSES 1 +#define _XOPEN_SOURCE_EXTENDED 1 %% ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] removing broken includes
On Sat Feb 5 11, Roman Divacky wrote:
alex,
I think you are the kind of person to try out
http://code.google.com/p/include-what-you-use/
thanks. this is the first time i hear of this project. i'll try to get it up
and running.
with fbsd :)
On Thu, Feb 03, 2011 at 10:12:57PM +, Alexander Best wrote:
hi everybody,
i've started to check the source for broken includes, such as the one fixed
in
r218189. so far i've run through arch=amd64 and found only two broken
includes,
which should be fixed by the attached patch. the commands i'm using are:
for target=buildkernel:
make SRCCONF=/dev/null __MAKE_CONF=/dev/null NO_WARNS= CWARNFLAGS=-w
-Werror -Wmissing-include-dirs buildkernel
for target=buildworld:[*]
make SRCCONF=/dev/null __MAKE_CONF=/dev/null WITHOUT_CDDL=true NO_WARNS=
CWARNFLAGS=-w -Wmissing-include-dirs buildworld
next up are the other archs. also i'll try to run both buildworld and
installworld with CC=clang -v and CXX=clang++ -v. although clang turns
-Wmissing-include-dirs into a noop, using the -v flag one can still detect
broken (and even duplicate) includes. thanks to pluknet@ for the info.
cheers.
alex
[*] the reason i'm setting WITHOUT_CDDL=true is that my installed world was
compiled with that option. trying to buildworld with CDDL support thus
fails.
also at first i was trying to use the same command for buildworld, as i used
for buildkernel. however it seems CWARNFLAGS gets set in some places, so
gcc(1)
will also treat non -Wmissing-include-dirs warnings as errors.
--
a13x
Index: sys/conf/files
===
--- sys/conf/files (revision 218217)
+++ sys/conf/files (working copy)
@@ -960,7 +960,7 @@
dev/e1000/e1000_82571.coptional em | igb \
compile-with ${NORMAL_C} -I$S/dev/e1000
dev/e1000/e1000_82575.coptional em | igb \
-compile-with ${NORMAL_C} -I$S/dev/igb
+ compile-with ${NORMAL_C} -I$S/dev/e1000
dev/e1000/e1000_ich8lan.c optional em | igb \
compile-with ${NORMAL_C} -I$S/dev/e1000
dev/e1000/e1000_api.c optional em | igb \
Index: sys/modules/netgraph/atm/ccatm/Makefile
===
--- sys/modules/netgraph/atm/ccatm/Makefile (revision 218217)
+++ sys/modules/netgraph/atm/ccatm/Makefile (working copy)
@@ -12,6 +12,6 @@
SRCS= ng_ccatm.c cc_conn.c cc_data.c cc_dump.c cc_port.c cc_sig.c
\
cc_user.c unisap.c
-CFLAGS+= -I${LIBBASE} -I${LIBBASE}/netnatm/ccatm -DCCATM_DEBUG
+CFLAGS+= -I${LIBBASE} -DCCATM_DEBUG
.include bsd.kmod.mk
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
--
a13x
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] removing broken includes
On Fri Feb 4 11, John Baldwin wrote:
On Thursday, February 03, 2011 5:12:57 pm Alexander Best wrote:
hi everybody,
i've started to check the source for broken includes, such as the one fixed
in
r218189. so far i've run through arch=amd64 and found only two broken
includes,
which should be fixed by the attached patch. the commands i'm using are:
The sys/conf/files fix should definitely go in.
after sys/modules/netgraph/atm/ccatm/Makefile has been fixed, maybe we could
extend the warnings at least for the kernel. that way new commits would need
to make sure that no broken includes are being added to the src.
any thoughts on this patch?
cheers.
alex
ps: -W was renamed to -Werrer. see the gcc(1) manual page.
--
John Baldwin
--
a13x
Index: sys/conf/kern.mk
===
--- sys/conf/kern.mk(revision 218319)
+++ sys/conf/kern.mk(working copy)
@@ -3,7 +3,7 @@
#
# Warning flags for compiling the kernel and components of the kernel.
#
-# Note that the newly added -Wcast-qual is responsible for generating
+# Note that the newly added -Wcast-qual is responsible for generating
# most of the remaining warnings. Warnings introduced with -Wall will
# also pop up, but are easier to fix.
.if ${CC:T:Micc} == icc
@@ -12,17 +12,18 @@
.else
CWARNFLAGS?= -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes \
-Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual \
- -Wundef -Wno-pointer-sign -fformat-extensions
+ -Wundef -Wno-pointer-sign -fformat-extensions \
+ -Wmissing-include-dirs
.endif
#
# The following flags are next up for working on:
-# -W
+# -Wextra
#
# On the i386, do not align the stack to 16-byte boundaries. Otherwise GCC
# 2.95 adds code to the entry and exit point of every function to align the
# stack to 16-byte boundaries -- thus wasting approximately 12 bytes of stack
-# per function call. While the 16-byte alignment may benefit micro
benchmarks,
+# per function call. While the 16-byte alignment may benefit micro benchmarks,
# it is probably an overall loss as it makes the code bigger (less efficient
# use of code cache tag lines) and uses more stack (less efficient use of data
# cache tag lines). Explicitly prohibit the use of SSE and other SIMD
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Intermittent pauses copying from one usb drive to another
On Saturday 05 February 2011 14:27:56 Donald Allen wrote: I've discussed problems with FreeBSD and usb sata drives on this list in the past (as recently as last September), and have given up on FreeBSD as a result of the usb disk problems (I do my backups with usb drives) and other usb-related issues. But I've now replaced the usb enclosures I was using at that time and have acquired some Toshiba usb drives, so with this new equipment, I thought I'd give FreeBSD another try, since generally I prefer it to Linux. But I'm having problems again. Plugging in the Toshiba drives produces entries like this in /var/log/message: Hi, Errors happen because the USB firmware of your USB hardware does not support all the SCSI commands issued by the CAM/SCSI layer. This is a well known problem. Probably Linux is better at filtering the SCSI commands passed to the various devices. Proof: I have a custom USB test software suite actually running under FreeBSD which can stress test mass storage devices. So far very few USB mass storage devices I've seen pass all the tests and recover properly. Most disks/memory sticks simply die at the first non-supported SCSI command and/or error scenario even though the SCSI and USB mass storage wrapper for SCSI commands define error codes to be returned in case of failure. Contact me off-list if you are interested in this. Solution: USB mass storage devices needs to be crippled down to a few SCSI commands like READ_12 and WRITE_12. This is not a USB problem. It needs to be done in the CAM/SCSI layer. What you can do: Ask the vendor to test their USB devices with FreeBSD 8.2 before shipping. --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Intermittent pauses copying from one usb drive to another
On Saturday 05 February 2011 16:18:56 Donald Allen wrote: Does whoever is responsible for CAM/SCSI know about this and do you know if there are plans to fix it? What is the point of supporting USB devices (and we aren't talking about an odd-ball device here; these are USB disks), when the support is partially broken? As far as I know there are no ongoing plans to fix this issue. Yes, we need to support the oddballs too, of course. Currently a lot of quirks have been pushed into the umass driver, but I guess that we need to add more quirks, and probably switch around from black-listing into white-listing, so that the quirks are turned on by default. Thanks for your understanding! --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [PATCH] OpenSolaris/ZFS: C++ compatibility
On Fri, Feb 04, 2011 at 11:03:53AM -0700, Justin T. Gibbs wrote: The attached patch is sufficient to allow a C++ program to use libzfs. The motivation for these changes is work I'm doing on a ZFS fault handling daemon that is written in C++. SpectraLogic's intention is to return this work to the FreeBSD project once it is a bit more complete. Since these changes modify files that come from OpenSolaris, I want to be sure I understand the project's policies regarding divergence from the vendor before I check them in. All of the changes save one should be trivial to merge with vendor changes and I will do that work for the v28 import. Is there any reason I should not commit these changes? Now that OpenSolaris is dead we don't have to be so strict with keeping the diff against vendor small at all cost. I'd prefer not to modify vendor code whenever possible so it is easier for us to cooperate with IllumOS (we already took ome code from them). Me and my company are also interested in fault management daemon (although not restricted to ZFS, but a more general purpose mechanism like FMA in Solaris). My question would be are there any chances you may be convinced to use plain C? With C we might be able to help, but not with C++. -- Pawel Jakub Dawidek http://www.wheelsystems.com p...@freebsd.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! pgphkmODt5wu8.pgp Description: PGP signature
Re: Intermittent pauses copying from one usb drive to another
On Sat, Feb 5, 2011 at 9:34 AM, Hans Petter Selasky hsela...@c2i.netwrote: On Saturday 05 February 2011 14:27:56 Donald Allen wrote: I've discussed problems with FreeBSD and usb sata drives on this list in the past (as recently as last September), and have given up on FreeBSD as a result of the usb disk problems (I do my backups with usb drives) and other usb-related issues. But I've now replaced the usb enclosures I was using at that time and have acquired some Toshiba usb drives, so with this new equipment, I thought I'd give FreeBSD another try, since generally I prefer it to Linux. But I'm having problems again. Plugging in the Toshiba drives produces entries like this in /var/log/message: Hi, Errors happen because the USB firmware of your USB hardware does not support all the SCSI commands issued by the CAM/SCSI layer. This is a well known problem. Probably Linux is better at filtering the SCSI commands passed to the various devices. Proof: I have a custom USB test software suite actually running under FreeBSD which can stress test mass storage devices. So far very few USB mass storage devices I've seen pass all the tests and recover properly. Most disks/memory sticks simply die at the first non-supported SCSI command and/or error scenario even though the SCSI and USB mass storage wrapper for SCSI commands define error codes to be returned in case of failure. Contact me off-list if you are interested in this. Solution: USB mass storage devices needs to be crippled down to a few SCSI commands like READ_12 and WRITE_12. This is not a USB problem. It needs to be done in the CAM/SCSI layer. I understand what you are saying. Does whoever is responsible for CAM/SCSI know about this and do you know if there are plans to fix it? What is the point of supporting USB devices (and we aren't talking about an odd-ball device here; these are USB disks), when the support is partially broken? You don't see problems like this with Windows or Linux, and this whole area of USB devices has been a problem since the 7.x series and, I suspect, before, regardless of whether it's the fault of the USB layer or some other part of the system. The user doesn't care which layer is at fault; they care about whether their devices are supported or not by the system. What you can do: Ask the vendor to test their USB devices with FreeBSD 8.2 before shipping. That's going to be like talking to a wall. I have a Toshiba netbook that does not run Linux properly, at least not Linux out of the box. Why? The BIOS is broken and doesn't support MSI correctly (interrupts are lost). Windows (at least the one delivered with the machine -- XP) doesn't use MSI, but Linux does by default. So you try installing Linux on this thing and the installation periodically grinds to a halt until you generate some interrupts by touching the touchpad or pressing a key on the keyboard, etc. You have to turn off MSI in the Linux kernel at boot time to get things to work. I explained all this to Toshiba Customer Service (an oxymoron) last year and they couldn't have cared less and it is still not fixed. They don't give a damn about Linux, so I'm sure they'll get right on it when I ask them to test their USB drives with FreeBSD :-) /Don --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: libdialog wide charcter support (Was: Request for testing/comments -- import of new dialog/libdialog)
On 02/05/11 03:04, Anonymous wrote: Nathan Whitehornnwhiteh...@freebsd.org writes: As part of work on a new installer, I would like to update the base system dialog and libdialog to the newer one provided by Thomas Dickey (http://invisible-island.net/dialog/, ports as devel/cdialog). This is a much nicer, fuller featured version of dialog that simplifies the creation of new dialog-using tools (a longstanding impediment to a new versions of sade, sysinstall, etc.), and is under a marginally better license (LGPL2 instead of GPL2). Patches to effect the import can be found at: - http://people.freebsd.org/~nwhitehorn/libdialog-update.diff Unlike devel/cdialog your diff (and r217309) has wide character support disabled. Was this intentional? For example, without it for each UTF-8 character there'd be extra space padding in editobox, inputbox, fselect, dselect, etc. Thanks for the report! This is fixed in r218331. -Nathan ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Intermittent pauses copying from one usb drive to another
On 2/5/11 8:28 AM, Donald Allen wrote: On Sat, Feb 5, 2011 at 10:28 AM, Hans Petter Selaskyhsela...@c2i.netwrote: On Saturday 05 February 2011 16:18:56 Donald Allen wrote: Does whoever is responsible for CAM/SCSI know about this and do you know if there are plans to fix it? What is the point of supporting USB devices (and we aren't talking about an odd-ball device here; these are USB disks), when the support is partially broken? As far as I know there are no ongoing plans to fix this issue. Yes, we need to support the oddballs too, of course. Actually, I'm arguing that you (FreeBSD, not necessarily the USB layer) need to better support the *main-stream* devices better (I care much less about the outliers, because by definition, the quality of their support affects far fewer people). These Toshiba drives are vanilla stuff, used by many for backups Currently a lot of quirks have been pushed into the umass driver, but I guess that we need to add more quirks, and probably switch around from black-listing into white-listing, so that the quirks are turned on by default. Thanks for your understanding! I do understand, but in some ways I don't. FreeBSD is a great system, superior to Linux in many ways that have been discussed ad nauseum, but USB devices, particularly disks and flash-based devices, are ubiquitous these days, and FreeBSD's support for these devices is weak, weaker than Linux and even OpenBSD (in my experience). This seems like a very odd place for a gaping hole in the system, particularly after all the work you did to re-implement the USB layer. Hans Petter, Could the USB mass storage layer not refuse to pass down some commands and just return the proscribed error? /Don --HPS ___ freebsd-...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-usb To unsubscribe, send any mail to freebsd-usb-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Intermittent pauses copying from one usb drive to another
Hans Petter, Could the USB mass storage layer not refuse to pass down some commands and just return the proscribed error? Yes, that's what I'm thinking would be the simplest solution. I just need an overview which SCSI commands we should pass and not pass. Do you have an idea? --hps ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Intermittent pauses copying from one usb drive to another
On Sat, Feb 5, 2011 at 10:28 AM, Hans Petter Selasky hsela...@c2i.netwrote: On Saturday 05 February 2011 16:18:56 Donald Allen wrote: Does whoever is responsible for CAM/SCSI know about this and do you know if there are plans to fix it? What is the point of supporting USB devices (and we aren't talking about an odd-ball device here; these are USB disks), when the support is partially broken? As far as I know there are no ongoing plans to fix this issue. Yes, we need to support the oddballs too, of course. Actually, I'm arguing that you (FreeBSD, not necessarily the USB layer) need to better support the *main-stream* devices better (I care much less about the outliers, because by definition, the quality of their support affects far fewer people). These Toshiba drives are vanilla stuff, used by many for backups Currently a lot of quirks have been pushed into the umass driver, but I guess that we need to add more quirks, and probably switch around from black-listing into white-listing, so that the quirks are turned on by default. Thanks for your understanding! I do understand, but in some ways I don't. FreeBSD is a great system, superior to Linux in many ways that have been discussed ad nauseum, but USB devices, particularly disks and flash-based devices, are ubiquitous these days, and FreeBSD's support for these devices is weak, weaker than Linux and even OpenBSD (in my experience). This seems like a very odd place for a gaping hole in the system, particularly after all the work you did to re-implement the USB layer. /Don --HPS ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Non-SCSI cam layer based device noise (was Re: Intermittent pauses copying from one usb drive to another)
On Sat, Feb 5, 2011 at 6:34 AM, Hans Petter Selasky hsela...@c2i.net wrote: On Saturday 05 February 2011 14:27:56 Donald Allen wrote: I've discussed problems with FreeBSD and usb sata drives on this list in the past (as recently as last September), and have given up on FreeBSD as a result of the usb disk problems (I do my backups with usb drives) and other usb-related issues. But I've now replaced the usb enclosures I was using at that time and have acquired some Toshiba usb drives, so with this new equipment, I thought I'd give FreeBSD another try, since generally I prefer it to Linux. But I'm having problems again. Plugging in the Toshiba drives produces entries like this in /var/log/message: Hi, Errors happen because the USB firmware of your USB hardware does not support all the SCSI commands issued by the CAM/SCSI layer. This is a well known problem. Probably Linux is better at filtering the SCSI commands passed to the various devices. Proof: I have a custom USB test software suite actually running under FreeBSD which can stress test mass storage devices. So far very few USB mass storage devices I've seen pass all the tests and recover properly. Most disks/memory sticks simply die at the first non-supported SCSI command and/or error scenario even though the SCSI and USB mass storage wrapper for SCSI commands define error codes to be returned in case of failure. Contact me off-list if you are interested in this. Solution: USB mass storage devices needs to be crippled down to a few SCSI commands like READ_12 and WRITE_12. This is not a USB problem. It needs to be done in the CAM/SCSI layer. I know I'm sidetracking a bit, but ATACAM is in a similar boat because I still run into corner cases like leaving a blank DVD+R in my burner and something is polling the drive and it's coming back as follows multiple times a second (snippets from /var/log/messages): Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): READ TOC/PMA/ATIP. CDB: 43 0 0 0 0 0 0 0 4 0 Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): CAM status: SCSI Status Error Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): SCSI status: Check Condition Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid field in CDB): Command byte 6 is invalid Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): READ TOC/PMA/ATIP. CDB: 43 0 0 0 0 0 0 0 4 0 Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): CAM status: SCSI Status Error Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): SCSI status: Check Condition Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid field in CDB): Command byte 6 is invalid Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): READ TOC/PMA/ATIP. CDB: 43 0 0 0 0 0 0 0 4 0 Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): CAM status: SCSI Status Error Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): SCSI status: Check Condition Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid field in CDB): Command byte 6 is invalid Feb 2 03:00:01 bayonetta kernel: (cd0:ahcich0:0:0:0): READ TOC/PMA/ATIP. CDB: 43 0 0 0 0 0 0 0 4 0 On a more relevant topic it would also be nice if the following noise (note the Medium not present errors) didn't end up in the syslog every time I turned on my monitor (the monitor has built in card readers and I rarely populate them with real flash devices), but I don't know if that's doable because it might break some helpful output in syslog when failures do occur with real SCSI enabled devices: ugen7.2: vendor 0x0424 at usbus7 uhub8: vendor 0x0424 product 0x2502, class 9/0, rev 2.00/0.01, addr 2 on usbus7 uhub8: 2 ports with 1 removable, self powered ugen7.3: vendor 0x0424 at usbus7 uhub9: vendor 0x0424 product 0x2602, class 9/0, rev 2.00/0.00, addr 3 on usbus7 uhub9: 4 ports with 3 removable, self powered ugen7.4: Generic at usbus7 umass0: Generic Flash Card Reader, class 0/0, rev 2.00/4.44, addr 4 on usbus7 (probe0:umass-sim0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe0:umass-sim0:0:0:0): CAM status: SCSI Status Error (probe0:umass-sim0:0:0:0): SCSI status: Check Condition (probe0:umass-sim0:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present) da0 at umass-sim0 bus 0 scbus7 target 0 lun 0 da0: Generic Flash HS-CF 4.44 Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: Attempt to query device size failed: NOT READY, Medium not present (probe0:umass-sim0:0:0:1): TEST UNIT READY. CDB: 0 20 0 0 0 0 (probe0:umass-sim0:0:0:1): CAM status: SCSI Status Error (probe0:umass-sim0:0:0:1): SCSI status: Check Condition (probe0:umass-sim0:0:0:1): SCSI sense: NOT READY asc:3a,0 (Medium not present) da1 at umass-sim0 bus 0 scbus7 target 0 lun 1 da1: Generic Flash HS-COMBO 4.44 Removable Direct Access SCSI-0 device da1: 40.000MB/s transfers da1: Attempt to query device size failed: NOT READY,
Re: Non-SCSI cam layer based device noise (was Re: Intermittent pauses copying from one usb drive to another)
On Saturday 05 February 2011 19:32:59 Garrett Cooper wrote: On a more relevant topic it would also be nice if the following noise (note the Medium not present errors) didn't end up in the syslog every time I turned on my monitor (the monitor has built in card readers and I rarely populate them with real flash devices), but I don't know if that's doable because it might break some helpful output in syslog when failures do occur with real SCSI enabled devices: There was a discussion about this a few months ago - see http://lists.freebsd.org/pipermail/freebsd-scsi/2010-November/004650.html . -- Bruce Cran ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [PATCH] OpenSolaris/ZFS: C++ compatibility
On 2/5/2011 8:39 AM, Pawel Jakub Dawidek wrote: On Fri, Feb 04, 2011 at 11:03:53AM -0700, Justin T. Gibbs wrote: The attached patch is sufficient to allow a C++ program to use libzfs. The motivation for these changes is work I'm doing on a ZFS fault handling daemon that is written in C++. SpectraLogic's intention is to return this work to the FreeBSD project once it is a bit more complete. Since these changes modify files that come from OpenSolaris, I want to be sure I understand the project's policies regarding divergence from the vendor before I check them in. All of the changes save one should be trivial to merge with vendor changes and I will do that work for the v28 import. Is there any reason I should not commit these changes? Now that OpenSolaris is dead we don't have to be so strict with keeping the diff against vendor small at all cost. I'd prefer not to modify vendor code whenever possible so it is easier for us to cooperate with IllumOS (we already took ome code from them). Perhaps IllumOS will accept these changes back? As I mentioned in the change descriptions included with the patch, the header files already show the intention of providing C++ support (extern C blocks), they just don't quite deliver. The changes shouldn't be controversial. Me and my company are also interested in fault management daemon (although not restricted to ZFS, but a more general purpose mechanism like FMA in Solaris). We have talked internally about this at Spectra too. Since we don't have BSD licensed nvpair code, we've thought of using Google protocol buffers to allow extensible encoding of fault data. The GP implementation is MIT licensed and looks like it might be less cumbersome to use than nvpairs. For the first release of our product, however, we are just making due with the string data that devctl provides. My question would be are there any chances you may be convinced to use plain C? With C we might be able to help, but not with C++. The core FMA support needs to be reasonably accessible from C code of course (fully functional and not cumbersome to use). But we should allow FMA agents to be coded in whatever language is convenient to the developer. The project may only be able to accept agents in C (and I'm voting for C++ too) into it's distribution, but that policy should not drive us to make the FMA architecture hard to access from shell, python, ruby, or some other language. The reason I chose C++ for this task is that devd, the source of the events I process, already requires C++ so using C++ in zfsd doesn't impose any new requirements on the system. Zfsd, like even the C kernel of FreeBSD is coded in an object oriented fashion, but its much cleaner to implement this type of design in a language that inherently supports object oriented concepts. Could I rewrite all that I have in C? Sure, but there would have to be some compelling reasons to offset the reduction in clarity and maintainability such a change would cause. Is your inability to help on a C++ version of this code due to distaste for C++ or just a lack of experience with it? Thanks, Justin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] removing broken includes
On Fri, 4 Feb 2011 20:19:48 +, Alexander Best arun...@freebsd.org wrote:
On Fri Feb 4 11, John Baldwin wrote:
On Thursday, February 03, 2011 5:12:57 pm Alexander Best wrote:
hi everybody,
i've started to check the source for broken includes, such as the one fixed
in
r218189. so far i've run through arch=amd64 and found only two broken
includes,
which should be fixed by the attached patch. the commands i'm using are:
The sys/conf/files fix should definitely go in.
as a side note: while checking the tinderbox output i saw a few gas related
warnings, which complained about VCS IDs being spammed to object files.
Maybe it's intentional for these objects to _have_ the VCS id in them,
like some of the @(#) strings that are passed down to the ELF binary.
This sort of object data is often used to determine what bits have gone
into the binary, so we should check if they are intentional before
ifdef'ing them all out.
% Index: sbin/routed/rtquery/rtquery.c
% ===
% --- sbin/routed/rtquery/rtquery.c (revision 218217)
% +++ sbin/routed/rtquery/rtquery.c (working copy)
% @@ -49,6 +49,7 @@
% #include bstring.h
% #endif
%
% +#if 0
% #define UNUSED __attribute__((unused))
% #ifndef __RCSID
% #define __RCSID(_s) static const char rcsid[] UNUSED = _s
% @@ -67,6 +68,7 @@
% __RCSID($Revision: 2.26 $);
% #ident $Revision: 2.26 $
% #endif
% +#endif
We can avoid this by building rtquery.c with -D NO__RCSID. See below
for the same trick about lukemftpd...
% #ifndef sgi
% #define _HAVE_SIN_LEN
% Index: contrib/lukemftp/src/main.c
% ===
% --- contrib/lukemftp/src/main.c (revision 218217)
% +++ contrib/lukemftp/src/main.c (working copy)
% @@ -96,8 +96,10 @@
%
% #include sys/cdefs.h
% #ifndef lint
% +#if 0
% __COPYRIGHT(@(#) Copyright (c) 1985, 1989, 1993, 1994\n\
% The Regents of the University of California. All rights
reserved.\n);
% +#endif
% #endif /* not lint */
This looks 'harmless' to include in the resulting binary, but maybe we
should just define NO_COPYRIGHT while building lukemftp? This will
disable the spamming of the object code with the static string and we
don't have to modify vendor code at all.
So, how about this instead?
% diff -r bd0305f6cb2b libexec/lukemftpd/Makefile
% --- a/libexec/lukemftpd/MakefileSat Feb 05 02:39:55 2011 +0100
% +++ b/libexec/lukemftpd/MakefileSat Feb 05 22:28:12 2011 +0100
% @@ -13,6 +13,7 @@ SRCS= cmds.c conf.c ftpd.c ftpcmd.y pope
% SRCS+= strsuftoll.c
%
% WFORMAT= 0
% +CFLAGS+= -DNO_COPYRIGHT
%
% .if ${MK_INET6_SUPPORT} != no
% CFLAGS+= -DINET6
also i noticed a lot of these warnings:
usr/subversion-src/lib/libc/i386/string/memcpy.S:7: Warning: ignoring
incorrect section type for .note.GNU-stack
i think i've seen some recent commits dealing with that new section.
That looks like something more useful to fix now, rather than later.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [PATCH] OpenSolaris/ZFS: C++ compatibility
On Sat, Feb 05, 2011 at 02:36:40PM -0700, Justin T. Gibbs wrote:
On 2/5/2011 8:39 AM, Pawel Jakub Dawidek wrote:
On Fri, Feb 04, 2011 at 11:03:53AM -0700, Justin T. Gibbs wrote:
The attached patch is sufficient to allow a C++ program to use libzfs.
The motivation for these changes is work I'm doing on a ZFS fault
handling daemon that is written in C++. SpectraLogic's intention
is to return this work to the FreeBSD project once it is a bit more
complete.
Since these changes modify files that come from OpenSolaris, I want to be
sure I understand the project's policies regarding divergence from
the vendor before I check them in. All of the changes save one should
be trivial to merge with vendor changes and I will do that work for the
v28 import. Is there any reason I should not commit these changes?
Now that OpenSolaris is dead we don't have to be so strict with keeping
the diff against vendor small at all cost. I'd prefer not to modify
vendor code whenever possible so it is easier for us to cooperate with
IllumOS (we already took ome code from them).
Perhaps IllumOS will accept these changes back? As I mentioned in the
change descriptions included with the patch, the header files already
show the intention of providing C++ support (extern C blocks), they
just don't quite deliver. The changes shouldn't be controversial.
Sure. To be clear: I'm not against those changes, I think they are worth
it. And getting IllumOS to accept them back is definitely a good idea.
Me and my company are also interested in fault management daemon
(although not restricted to ZFS, but a more general purpose mechanism
like FMA in Solaris).
We have talked internally about this at Spectra too. Since we don't have
BSD licensed nvpair code, we've thought of using Google protocol buffers
to allow extensible encoding of fault data. The GP implementation is
MIT licensed and looks like it might be less cumbersome to use than
nvpairs. For the first release of our product, however, we are just
making due with the string data that devctl provides.
I've developed similar API during HAST work, maybe it is a good starting
point? src/sbin/hastd/nv.{c,h}.
My question would be are there any chances you may
be convinced to use plain C? With C we might be able to help, but not
with C++.
The core FMA support needs to be reasonably accessible from C code of
course (fully functional and not cumbersome to use). But we should
allow FMA agents to be coded in whatever language is convenient to the
developer. The project may only be able to accept agents in C (and I'm
voting for C++ too) into it's distribution, but that policy should not
drive us to make the FMA architecture hard to access from shell, python,
ruby, or some other language.
Yes, agents should not be limited to one language. I wouldn't be
surprised is the majority of agents will be shell scripts.
The reason I chose C++ for this task is that devd, the source of the
events I process, already requires C++ so using C++ in zfsd doesn't
impose any new requirements on the system. Zfsd, like even the C
kernel of FreeBSD is coded in an object oriented fashion, but its
much cleaner to implement this type of design in a language that
inherently supports object oriented concepts. Could I rewrite all
that I have in C? Sure, but there would have to be some compelling
reasons to offset the reduction in clarity and maintainability such
a change would cause.
Hmm, so zfsd will receive events from devd? I'm in opinion that we
should let devd alone. In my initial port I used devd, because it was
closest match, but if we want to clean it up, we shouldn't go through
devd. For example ZFS v28 can report whole binary blocks where checksum
doesn't match and passing those through devd would be cumbersome.
Is your inability to help on a C++ version of this code due to distaste
for C++ or just a lack of experience with it?
The latter. I'm sure there are many committers that are fluent in C++,
but all of them know C. I was under impression that Warner implemented
devd in C++ also as a kind of experiment, which nobody really followed.
--
Pawel Jakub Dawidek http://www.wheelsystems.com
p...@freebsd.org http://www.FreeBSD.org
FreeBSD committer Am I Evil? Yes, I Am!
pgpQQMrZ5Hwdv.pgp
Description: PGP signature
Re: [PATCH] OpenSolaris/ZFS: C++ compatibility
On 2/5/2011 3:06 PM, Pawel Jakub Dawidek wrote:
On Sat, Feb 05, 2011 at 02:36:40PM -0700, Justin T. Gibbs wrote:
Perhaps IllumOS will accept these changes back? As I mentioned in the
change descriptions included with the patch, the header files already
show the intention of providing C++ support (extern C blocks), they
just don't quite deliver. The changes shouldn't be controversial.
Sure. To be clear: I'm not against those changes, I think they are worth
it. And getting IllumOS to accept them back is definitely a good idea.
Ok.
We have talked internally about this at Spectra too. Since we don't have
BSD licensed nvpair code, we've thought of using Google protocol buffers
to allow extensible encoding of fault data. The GP implementation is
MIT licensed and looks like it might be less cumbersome to use than
nvpairs. For the first release of our product, however, we are just
making due with the string data that devctl provides.
I've developed similar API during HAST work, maybe it is a good starting
point? src/sbin/hastd/nv.{c,h}.
Sure. If the decision is made to use nvpairs, I would advocate for
emulating the Solaris API. There's no reason to be slightly different
from an established implementation.
The reason I chose C++ for this task is that devd, the source of the
events I process, already requires C++ so using C++ in zfsd doesn't
impose any new requirements on the system. Zfsd, like even the C
kernel of FreeBSD is coded in an object oriented fashion, but its
much cleaner to implement this type of design in a language that
inherently supports object oriented concepts. Could I rewrite all
that I have in C? Sure, but there would have to be some compelling
reasons to offset the reduction in clarity and maintainability such
a change would cause.
Hmm, so zfsd will receive events from devd? I'm in opinion that we
should let devd alone. In my initial port I used devd, because it was
closest match, but if we want to clean it up, we shouldn't go through
devd. For example ZFS v28 can report whole binary blocks where checksum
doesn't match and passing those through devd would be cumbersome.
zfsd parses devctl event data (via devd's unix domain socket) into an
internal event class representation. The data representation for the
event class as well as the source for the event data can be easily
changed out once a more complete solution is available. For the policies
SpectraLogic needs for its first product launch, the data coming through
devctl is sufficient even if it is not ideal.
Is your inability to help on a C++ version of this code due to distaste
for C++ or just a lack of experience with it?
The latter. I'm sure there are many committers that are fluent in C++,
but all of them know C. I was under impression that Warner implemented
devd in C++ also as a kind of experiment, which nobody really followed.
FreeBSD has lots of examples of really well written C code and shell code,
but almost no examples written in more modern languages (C++ isn't even
that modern!). That's too bad. My hope is that, by submitting another
example of (dare I hope well written) C++ use in FreeBSD, that this will
change. At least review the code (when I release it in a few weeks) before
begging me to rewrite it! :-)
--
Justin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [PATCH] OpenSolaris/ZFS: C++ compatibility
On Sat, Feb 5, 2011 at 7:29 PM, Justin T. Gibbs gi...@scsiguy.com wrote:
On 2/5/2011 3:06 PM, Pawel Jakub Dawidek wrote:
On Sat, Feb 05, 2011 at 02:36:40PM -0700, Justin T. Gibbs wrote:
Perhaps IllumOS will accept these changes back? As I mentioned in the
change descriptions included with the patch, the header files already
show the intention of providing C++ support (extern C blocks), they
just don't quite deliver. The changes shouldn't be controversial.
Sure. To be clear: I'm not against those changes, I think they are worth
it. And getting IllumOS to accept them back is definitely a good idea.
Ok.
We have talked internally about this at Spectra too. Since we don't have
BSD licensed nvpair code, we've thought of using Google protocol buffers
to allow extensible encoding of fault data. The GP implementation is
MIT licensed and looks like it might be less cumbersome to use than
nvpairs. For the first release of our product, however, we are just
making due with the string data that devctl provides.
I've developed similar API during HAST work, maybe it is a good starting
point? src/sbin/hastd/nv.{c,h}.
Sure. If the decision is made to use nvpairs, I would advocate for
emulating the Solaris API. There's no reason to be slightly different
from an established implementation.
The reason I chose C++ for this task is that devd, the source of the
events I process, already requires C++ so using C++ in zfsd doesn't
impose any new requirements on the system. Zfsd, like even the C
kernel of FreeBSD is coded in an object oriented fashion, but its
much cleaner to implement this type of design in a language that
inherently supports object oriented concepts. Could I rewrite all
that I have in C? Sure, but there would have to be some compelling
reasons to offset the reduction in clarity and maintainability such
a change would cause.
Hmm, so zfsd will receive events from devd? I'm in opinion that we
should let devd alone. In my initial port I used devd, because it was
closest match, but if we want to clean it up, we shouldn't go through
devd. For example ZFS v28 can report whole binary blocks where checksum
doesn't match and passing those through devd would be cumbersome.
zfsd parses devctl event data (via devd's unix domain socket) into an
internal event class representation. The data representation for the
event class as well as the source for the event data can be easily
changed out once a more complete solution is available. For the policies
SpectraLogic needs for its first product launch, the data coming through
devctl is sufficient even if it is not ideal.
Is your inability to help on a C++ version of this code due to distaste
for C++ or just a lack of experience with it?
The latter. I'm sure there are many committers that are fluent in C++,
but all of them know C. I was under impression that Warner implemented
devd in C++ also as a kind of experiment, which nobody really followed.
FreeBSD has lots of examples of really well written C code and shell code,
but almost no examples written in more modern languages (C++ isn't even
that modern!). That's too bad. My hope is that, by submitting another
example of (dare I hope well written) C++ use in FreeBSD, that this will
change. At least review the code (when I release it in a few weeks) before
begging me to rewrite it! :-)
The patch looks reasonable. It's kind of funny that the
OpenSolaris folks use variable names that conflict with reserved
keywords in C++ (class, private).
Thanks!
-Garrett
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
