Re: Low latency High Frequency Trading
On Thu, Nov 8, 2012 at 8:55 PM, Diana Eichert wrote: > take a look at Tilera TileGX boards > (you better hire a s/w developer.) > Some company is already working on that http://mail-index.netbsd.org/netbsd-users/2012/10/31/msg011803.html
Re: Low latency High Frequency Trading
On Thu, Nov 08, 2012 at 08:08:05PM +0200, Dan Shechter wrote: > For unrelated reasons, I can't directly receive the TCP stream. > > I must copy the TCP data from a running stream to another server. I > can use tap or just port-mirroring on the switch. So I can't use any > network stack or leverage any offloading. > > I also need to modify the received data, and add few application > headers before sending it as a multicast udp stream. > > Winsock is userland. What I want to do is in the kernel, even before > ip_input. I guess it should be faster. > > I am looking at netFPGA too, but prefer to do this in software. > You might want to try this: http://info.iet.unipi.it/~luigi/netmap/ It's FreeBSD and Linux only, though. The emerging solution for high performance traffic routers like this is to have one or more threads loop in userspace over a memory mapped NIC buffer. Most of these interfaces are highly proprietary. Netmap provides the relative programmatic simplicity of a TAP-type interface with the zero-copy performance of the mapped buffering.
Re: Building OpenConnect with libintl
On Thu, Nov 08, 2012 at 03:38:18PM +, Woodhouse, David wrote: > On Thu, 2012-11-08 at 14:36 +0100, Marc Espie wrote: > > > > Pass LIBTOOL=/usr/bin/libtool on make's command line. > > > > Trying to get through the spaghetti of gnu autocrap only leads to > > insanity. > > > > That falls under the "don't fight that shit, it's hopeless". > > Hm, OpenBSD libtool doesn't seem to honour the argument '-version-number > 2:1', so my resulting library is 'libopenconnect.so.0.0' instead of > 'libopenconnect.so.2.1'. > > Should I be specifying that differently? Found this in the GNU libtool documentation: "New projects should use the -version-info flag instead." It looks like -version-number is an accepted option (thus no error message), but no further processing is done on it. May be an oversight in our libtool? Note that the ports system overrides the version specified on the command line, to fit into the OpenBSD-specific versioning scheme. > > -- >Sent with MeeGo's ActiveSync support. > > David WoodhouseOpen Source Technology Centre > david.woodho...@intel.com Intel Corporation > > [demime 1.01d removed an attachment of type application/x-pkcs7-signature > which had a name of smime.p7s]
Re: Low latency High Frequency Trading
take a look at Tilera TileGX boards (you better hire a s/w developer.)
Re: Low latency High Frequency Trading
They are all available with PCI Express interface, no worries, so you will be able of plug them straight into your server. Alternatively, how about going for the second option of making living in this business :-) ? On Thu, Nov 8, 2012 at 7:09 PM, Dan Shechter wrote: > When I was saying money is not a problem, it was related to server > component costs... :) > Best regards, > Dan > > > On Thu, Nov 8, 2012 at 8:07 PM, Ariel Burbaickij > wrote: > > I know that you have an impression I am getting caustic :-) but these > > ideas are pretty obvious once money is not a problem field, so: > > > > http://en.wikipedia.org/wiki/Netronome > > > > IXPs on steroids. > > > > > > > > On Thu, Nov 8, 2012 at 7:01 PM, Johan Beisser wrote: > >> > >> On Thu, Nov 8, 2012 at 9:58 AM, Ariel Burbaickij > >> wrote: > >> > If money is not a problem -- go buy high-trading on the chip solutions > >> > and > >> > have sub-microsecond resolution. > >> > > >> > http://lmgtfy.com/?q=high+frequency+trading+FPGA > >> > >> I'd love to see PF offloading on to something like that. Not that I > >> can justify the expense for my work, but it'd be useful.
Re: Low latency High Frequency Trading
For unrelated reasons, I can't directly receive the TCP stream. I must copy the TCP data from a running stream to another server. I can use tap or just port-mirroring on the switch. So I can't use any network stack or leverage any offloading. I also need to modify the received data, and add few application headers before sending it as a multicast udp stream. Winsock is userland. What I want to do is in the kernel, even before ip_input. I guess it should be faster. I am looking at netFPGA too, but prefer to do this in software. Best regards, Dan On Thu, Nov 8, 2012 at 7:36 PM, Johan Beisser wrote: > On Thu, Nov 8, 2012 at 4:12 AM, Dan Shechter wrote: >> Hi All, >> >> >> A windows 2008 server is receiving TCP traffic from a stock exchange >> and sends it, almost as is, using UDP multicast to automated high >> frequancy traders. >> >> StockExchange --TCP---> windows2008 ---MCAST-UDP> >> >> On average, the time it take to do the TCP to UDP translation, using >> winsock, is 240 micro seconds. It can even be as high as 60,000 micro >> seconds. >> >> >> >> 1. Use port mirroring to get the TCP data sent to a dedicated OpenBSD >> box with two NICs. One for the TCP, the other for the multicast UDP. > > You'll incur an extra penalty offloading to the kernel. Winsock is > already doing that, though. > >> 2. Put the TCP port in a promiscuous mode. > > Why? You can just set up the right bits to listen to on the network, > and pull raw frames to be processed. Or, just let the network stack > behave as it should. > >> 3. Write my TCP->UDP logic directly into ether_input.c > > Any reason to not use pf for this translation? > >> >> >> Now for the questions: >> 1. Am I on the right track? or in other words how crazy is my idea? > > Pretty crazy. You may want to see if there's hardware accelerated or > on NIC TCP off-load options instead. > >> 2. What would be the latency? Can I achieve 50 microseconds between >> getting the interrupt and until sending the new packet through the >> NIC? > > See above. You'll end up having to do some tuning. > >> 3. Which NIC/CPU/Memory should I use? Money is not a problem. > > Custom order a few NICs, hire a developer to write a driver to offload > TCP/UDP on the NIC, and enable as little kernel interference as > possible. > > Money's not a problem, right?
Re: Low latency High Frequency Trading
When I was saying money is not a problem, it was related to server component costs... :) Best regards, Dan On Thu, Nov 8, 2012 at 8:07 PM, Ariel Burbaickij wrote: > I know that you have an impression I am getting caustic :-) but these > ideas are pretty obvious once money is not a problem field, so: > > http://en.wikipedia.org/wiki/Netronome > > IXPs on steroids. > > > > On Thu, Nov 8, 2012 at 7:01 PM, Johan Beisser wrote: >> >> On Thu, Nov 8, 2012 at 9:58 AM, Ariel Burbaickij >> wrote: >> > If money is not a problem -- go buy high-trading on the chip solutions >> > and >> > have sub-microsecond resolution. >> > >> > http://lmgtfy.com/?q=high+frequency+trading+FPGA >> >> I'd love to see PF offloading on to something like that. Not that I >> can justify the expense for my work, but it'd be useful.
Re: Low latency High Frequency Trading
I know that you have an impression I am getting caustic :-) but these ideas are pretty obvious once money is not a problem field, so: http://en.wikipedia.org/wiki/Netronome IXPs on steroids. On Thu, Nov 8, 2012 at 7:01 PM, Johan Beisser wrote: > On Thu, Nov 8, 2012 at 9:58 AM, Ariel Burbaickij > wrote: > > If money is not a problem -- go buy high-trading on the chip solutions > and > > have sub-microsecond resolution. > > > > http://lmgtfy.com/?q=high+frequency+trading+FPGA > > I'd love to see PF offloading on to something like that. Not that I > can justify the expense for my work, but it'd be useful.
Re: Low latency High Frequency Trading
On Thu, Nov 8, 2012 at 9:58 AM, Ariel Burbaickij wrote: > If money is not a problem -- go buy high-trading on the chip solutions and > have sub-microsecond resolution. > > http://lmgtfy.com/?q=high+frequency+trading+FPGA I'd love to see PF offloading on to something like that. Not that I can justify the expense for my work, but it'd be useful.
Re: Low latency High Frequency Trading
If money is not a problem -- go buy high-trading on the chip solutions and have sub-microsecond resolution. http://lmgtfy.com/?q=high+frequency+trading+FPGA On Thu, Nov 8, 2012 at 6:36 PM, Johan Beisser wrote: > On Thu, Nov 8, 2012 at 4:12 AM, Dan Shechter wrote: > > Hi All, > > > > > > A windows 2008 server is receiving TCP traffic from a stock exchange > > and sends it, almost as is, using UDP multicast to automated high > > frequancy traders. > > > > StockExchange --TCP---> windows2008 ---MCAST-UDP> > > > > On average, the time it take to do the TCP to UDP translation, using > > winsock, is 240 micro seconds. It can even be as high as 60,000 micro > > seconds. > > > > > > > > 1. Use port mirroring to get the TCP data sent to a dedicated OpenBSD > > box with two NICs. One for the TCP, the other for the multicast UDP. > > You'll incur an extra penalty offloading to the kernel. Winsock is > already doing that, though. > > > 2. Put the TCP port in a promiscuous mode. > > Why? You can just set up the right bits to listen to on the network, > and pull raw frames to be processed. Or, just let the network stack > behave as it should. > > > 3. Write my TCP->UDP logic directly into ether_input.c > > Any reason to not use pf for this translation? > > > > > > > Now for the questions: > > 1. Am I on the right track? or in other words how crazy is my idea? > > Pretty crazy. You may want to see if there's hardware accelerated or > on NIC TCP off-load options instead. > > > 2. What would be the latency? Can I achieve 50 microseconds between > > getting the interrupt and until sending the new packet through the > > NIC? > > See above. You'll end up having to do some tuning. > > > 3. Which NIC/CPU/Memory should I use? Money is not a problem. > > Custom order a few NICs, hire a developer to write a driver to offload > TCP/UDP on the NIC, and enable as little kernel interference as > possible. > > Money's not a problem, right?
Re: Low latency High Frequency Trading
On Thu, Nov 8, 2012 at 4:12 AM, Dan Shechter wrote: > Hi All, > > > A windows 2008 server is receiving TCP traffic from a stock exchange > and sends it, almost as is, using UDP multicast to automated high > frequancy traders. > > StockExchange --TCP---> windows2008 ---MCAST-UDP> > > On average, the time it take to do the TCP to UDP translation, using > winsock, is 240 micro seconds. It can even be as high as 60,000 micro > seconds. > > > > 1. Use port mirroring to get the TCP data sent to a dedicated OpenBSD > box with two NICs. One for the TCP, the other for the multicast UDP. You'll incur an extra penalty offloading to the kernel. Winsock is already doing that, though. > 2. Put the TCP port in a promiscuous mode. Why? You can just set up the right bits to listen to on the network, and pull raw frames to be processed. Or, just let the network stack behave as it should. > 3. Write my TCP->UDP logic directly into ether_input.c Any reason to not use pf for this translation? > > > Now for the questions: > 1. Am I on the right track? or in other words how crazy is my idea? Pretty crazy. You may want to see if there's hardware accelerated or on NIC TCP off-load options instead. > 2. What would be the latency? Can I achieve 50 microseconds between > getting the interrupt and until sending the new packet through the > NIC? See above. You'll end up having to do some tuning. > 3. Which NIC/CPU/Memory should I use? Money is not a problem. Custom order a few NICs, hire a developer to write a driver to offload TCP/UDP on the NIC, and enable as little kernel interference as possible. Money's not a problem, right?
Re: OpenBGPd iBGP and IPv6
On Fri, Nov 09, 2012 at 12:23:45AM +0800, Patrick Coleman wrote: > Is there any reason you need to restrict capabilities like this on > iBGP? Have you tried removing the the announce IPv6 unicast lines (so > the announce all inherits from the parent clause) to see what happens? Hi, announce (IPv4|IPv6) (none|unicast|vpn) For the given address family, control which subsequent address families (at the moment, only none, which disables the announcement of that address family, unicast, and vpn, which allows the distribution of BGP MPLS VPNs, are supported) are announced during the capabilities negotiation. Only routes for that address family and subsequent address family will be announced and processed. Since I only want/need/have unicast address family, there is apparently no need to change it. After trying with this statement removed, it doesn't change anything :( > The announce IPv6 unicast statement will affect the capabilities you > send to the peer, reading the manpage, so that seems suspicious. The > other thing I would check would be the internal routing - I assume you > have appropriate routes between the peers? Sure I do. Internal routing between loopback interfaces is done through OSPF. > Here's some of my IPv6 iBGP config. Note when filtering I also permit > all from the iBGP group (last line). These could probably be tightened > up a bit, but it might be a good place to start. I tried with disabled filters, it didn't help either. I'll no matter what still continue to find where this issue comes from. Cheers, Laurent
Re: OpenBGPd iBGP and IPv6
On Tue, Nov 6, 2012 at 7:52 PM, Laurent CARON wrote:
>
> group "iBGP_VTY_TMM" {
> remote-as 49463
> announceall
> set nexthop self
>
> neighbor 2a02:27d0:100:114::4 {
> descr iv6_gw-001_to_004
> local-address 2a02:27d0:0:112::1
> announce IPv6 unicast
Is there any reason you need to restrict capabilities like this on
iBGP? Have you tried removing the the announce IPv6 unicast lines (so
the announce all inherits from the parent clause) to see what happens?
The announce IPv6 unicast statement will affect the capabilities you
send to the peer, reading the manpage, so that seems suspicious. The
other thing I would check would be the internal routing - I assume you
have appropriate routes between the peers?
Here's some of my IPv6 iBGP config. Note when filtering I also permit
all from the iBGP group (last line). These could probably be tightened
up a bit, but it might be a good place to start.
-Patrick
...
group "iBGP - AS55881" {
remote-as 55881
announce all
set nexthop self
neighbor 202.1.16.86 {
descr "iBGP: atri-edge-001-a IPv4"
}
neighbor 2405:6a00:6:fd::2 {
descr "iBGP: atri-edge-001-a IPv6"
}
...
deny from any
allow from any inet prefixlen 8 - 24
allow from any inet6 prefixlen 8 - 48
# filter bogus networks: IPv4
deny from any prefix 10.0.0.0/8 prefixlen >= 8
deny from any prefix 172.16.0.0/12 prefixlen >= 12
deny from any prefix 192.168.0.0/16 prefixlen >= 16
deny from any prefix 169.254.0.0/16 prefixlen >= 16
deny from any prefix 192.0.2.0/24 prefixlen >= 24
deny from any prefix 224.0.0.0/4 prefixlen >= 4
deny from any prefix 240.0.0.0/4 prefixlen >= 4
# filter bogus networks: IPv6
deny from any prefix 3ffe::/16 prefixlen >= 16# 6bone
deny from any prefix 2001:db8::/32 prefixlen >=32# documentation
deny from any prefix ::/8 prefixlen >= 8# loopback
deny from any prefix fe00::/8 prefixlen >= 8# multicast (RFC3513)
deny from any prefix ff00::/8 prefixlen >= 8# multicast (RFC3513)
# Permit anything from our iBGP peers
allow from group "iBGP - AS55881"
Re: Building OpenConnect with libintl
On Thu, 2012-11-08 at 14:36 +0100, Marc Espie wrote: > > Pass LIBTOOL=/usr/bin/libtool on make's command line. > > Trying to get through the spaghetti of gnu autocrap only leads to > insanity. > > That falls under the "don't fight that shit, it's hopeless". Hm, OpenBSD libtool doesn't seem to honour the argument '-version-number 2:1', so my resulting library is 'libopenconnect.so.0.0' instead of 'libopenconnect.so.2.1'. Should I be specifying that differently? -- Sent with MeeGo's ActiveSync support. David WoodhouseOpen Source Technology Centre david.woodho...@intel.com Intel Corporation [demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]
Re: crypto volume damaged after crash
On Thu, Nov 08, 2012 at 10:51:05PM +1100, Joel Sing wrote: > If this is a hardware failure then it is not overly interesting, however if > the underlying device is healthy then I would be interested in getting > further details. Thanks. I ran a diagnostics and it turned out to be the disk. Wonder when I will learn to always check the most obvious thing first. Sorry for the noise.
Re: Building OpenConnect with libintl
Woodhouse, David wrote: > It seems that libintl *is* present, but it's installed in /usr/local and > the compiler doesn't find it by default. [...] > surely I shouldn't have to advise users to build things that way when > using the platform's stock libintl? I would like to clarify that libintl is NOT part of a stock OpenBSD installation. It's third-party software and needs to be explicitly added as a package. (Well, most likely you add something else and it will pull in libintl as a dependency.) -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: Building OpenConnect with libintl
On Thu, 2012-11-08 at 14:36 +0100, Marc Espie wrote:
> Pass LIBTOOL=/usr/bin/libtool on make's command line.
Thanks, that works. This commit should make it work for everyone
automatically, without them having to override it manually:
http://git.infradead.org/users/dwmw2/openconnect.git/commitdiff/8e2a463043
Now it should build out-of-the-box even when there's already a version
installed. It doesn't get NLS support unless you manually add /usr/local
to the compiler's search paths, but I'm prepared not to care about that
if that's what's expected on OpenBSD. I don't think it's sane for me to
*automatically* try adding /usr/local/{include,lib} on OpenBSD, is it?
I don't quite understand why that isn't the native toolchain default,
but it's not for me to fix it up in my own autohell. I have enough hell
of my own in there, without dealing with yours :)
Thanks for the help.
--
Sent with MeeGo's ActiveSync support.
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com Intel Corporation
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which
had a name of smime.p7s]
Re: Building OpenConnect with libintl
On Thu, Nov 08, 2012 at 01:27:31PM +, Woodhouse, David wrote: > On Thu, 2012-11-08 at 14:06 +0100, Marc Espie wrote: > > *our* libtool looks first under .libs. If it doesn't, that's a bug. > > I surmise the bug-reporter is actually using gnu-libtool, or the > > libtool generated by THAT software. > > Hm, yes. I am *indeed* using GNU libtool. That's confusing; I didn't > even know it was installed. If I run 'libtool --version', I get the > non-GNU one. But "./libtool" in my build directory ??? built from the git > tree with libtoolize on the OpenBSD system, not from a tarball which > obviously would have its own pre-autotoolised stuff ??? is the GNU one. > > So perhaps the next question is: what's wrong with my ./autogen.sh > script? It currently looks like this: > > #!/bin/sh > > aclocal && \ > libtoolize --automake --copy --force && \ > automake --foreign --add-missing && \ > autoconf > > Should it have some kind of special case for OpenBSD? It looks like > 'libtoolize' on my default path is the GNU one, while 'libtool' isn't. I > don't think I did anything to screw with that; this should be a simple > OpenBSD 5.2 install. > > And even if I fix the autogen.sh script for people building from the git > tree, what about tarball releases that I make? Do I just let people know > that those are *broken* on OpenBSD because GNU libtool doesn't work > there? > > Confused... and hating autohell a little more than I did yesterday. > Which I didn't know was possible. Pass LIBTOOL=/usr/bin/libtool on make's command line. Trying to get through the spaghetti of gnu autocrap only leads to insanity. That falls under the "don't fight that shit, it's hopeless".
openbsd 5.2 hangs on mpii
Hi We are trying to install openbsd 5.2 on a dell poweredge R910 server, but without success. It hangs on the following message: mpii0: mpii_scsi_cmd_tmo We have a PERC H200 INTEGRATED RAID CONTROLLER card flashed to the latest version. When trying to install version 4.9 there are no problems, but it happens to versions 5.0 5.1 and 5.2 Regards Martijn
Re: Building OpenConnect with libintl
On Thu, Nov 08, 2012 at 12:57:42PM +, Stuart Henderson wrote: > > Anyway, it doesn't *work* either ??? the build failed. It seems that when > > building the openconnect executable, it finds the old libopenconnect.so > > in /usr/local/lib *before* the new one it's just built in the build > > directory. And thus the link fails. That sounds like it might be a > > libtool/autotools bug ??? surely it should link against the library it > > just built, and put -L./.libs on the search path *before* anything else? > > I was using the latest available tools where given the choice; autoconf > > 2.69, automake 1.12 and libtool (not GNU libtool) 1.5.26. Should I try > > with GNU libtool instead? > > > > I assume I'm doing something wrong here. Advice on how to make it build > > properly on OpenBSD would be much appreciated... > > libtool people: is there something we should be doing something to > reorder the library directory list to ensure the .libs directory > comes first in the search list? Or is there something else going on > here? We have some places in the ports tree where we explicitly override > LDFLAGS to include .libs directories (e.g. imagemagick) which I presume > is for this same reason - there aren't very many instances of this though > it's possible people have only worked-around this problem in cases > where they found it really painful to uninstall an existing package > and its dependencies when working on an update. *our* libtool looks first under .libs. If it doesn't, that's a bug. I surmise the bug-reporter is actually using gnu-libtool, or the libtool generated by THAT software. There's totally nothing we can do about gnu libtool, it is broken by design on anything that's not standard linux elf linking (and we're not, we treat libraries as specific objects, and don't really support linking stuff with libiconv.so)... fixing THAT upstream is really tiresome, because most of the FSF upstream guys will only cringe, tell us we should "conform", and not change anything in their way of thinking/doing things (we probably don't follow some writ of Saint Stallman, god preserve us).
Re: Building OpenConnect with libintl
On 2012/11/08 11:23, Woodhouse, David wrote: > I saw the OpenBSD 5.2 release and figured I should make sure the > OpenConnect VPN client builds OK on it still. It does, but I noticed > that it didn't build with localisation support, and tried to fix that. > > It seems that libintl *is* present, but it's installed in /usr/local and > the compiler doesn't find it by default. I'm not entirely sure if this > is a bug in the libintl/gettext installation, in the compiler default > search paths, or a deliberate design decision that an installed library > should fail to work by default... but I attempted to work around it by > adding 'CFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib' to my > configure invocation. That's tolerable for a first test build, but > surely I shouldn't have to advise users to build things that way when > using the platform's stock libintl? It's deliberate that the system preprocesser and linker don't search /usr/local/include and /usr/local/lib, I'm not sure of the original reasoning behind it. The usual method is exactly what you've done with the explicit LDFLAGS/CPPFLAGS and this is afaik standard behaviour on BSDs. When pkg-config is used there's generally no problem but this doesn't help with the very common libintl. > (I also needed to explicitly link against -liconv in addition to -lintl, > which I've now added to the configure script in git.) > > Anyway, it doesn't *work* either — the build failed. It seems that when > building the openconnect executable, it finds the old libopenconnect.so > in /usr/local/lib *before* the new one it's just built in the build > directory. And thus the link fails. That sounds like it might be a > libtool/autotools bug — surely it should link against the library it > just built, and put -L./.libs on the search path *before* anything else? > I was using the latest available tools where given the choice; autoconf > 2.69, automake 1.12 and libtool (not GNU libtool) 1.5.26. Should I try > with GNU libtool instead? > > I assume I'm doing something wrong here. Advice on how to make it build > properly on OpenBSD would be much appreciated... libtool people: is there something we should be doing something to reorder the library directory list to ensure the .libs directory comes first in the search list? Or is there something else going on here? We have some places in the ports tree where we explicitly override LDFLAGS to include .libs directories (e.g. imagemagick) which I presume is for this same reason - there aren't very many instances of this though it's possible people have only worked-around this problem in cases where they found it really painful to uninstall an existing package and its dependencies when working on an update.
Low latency High Frequency Trading
Hi All, A windows 2008 server is receiving TCP traffic from a stock exchange and sends it, almost as is, using UDP multicast to automated high frequancy traders. StockExchange --TCP---> windows2008 ---MCAST-UDP> On average, the time it take to do the TCP to UDP translation, using winsock, is 240 micro seconds. It can even be as high as 60,000 micro seconds. 1. Use port mirroring to get the TCP data sent to a dedicated OpenBSD box with two NICs. One for the TCP, the other for the multicast UDP. 2. Put the TCP port in a promiscuous mode. 3. Write my TCP->UDP logic directly into ether_input.c Now for the questions: 1. Am I on the right track? or in other words how crazy is my idea? 2. What would be the latency? Can I achieve 50 microseconds between getting the interrupt and until sending the new packet through the NIC? 3. Which NIC/CPU/Memory should I use? Money is not a problem. Thanks, Dan
Building OpenConnect with libintl
I saw the OpenBSD 5.2 release and figured I should make sure the OpenConnect VPN client builds OK on it still. It does, but I noticed that it didn't build with localisation support, and tried to fix that. It seems that libintl *is* present, but it's installed in /usr/local and the compiler doesn't find it by default. I'm not entirely sure if this is a bug in the libintl/gettext installation, in the compiler default search paths, or a deliberate design decision that an installed library should fail to work by default... but I attempted to work around it by adding 'CFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib' to my configure invocation. That's tolerable for a first test build, but surely I shouldn't have to advise users to build things that way when using the platform's stock libintl? (I also needed to explicitly link against -liconv in addition to -lintl, which I've now added to the configure script in git.) Anyway, it doesn't *work* either â the build failed. It seems that when building the openconnect executable, it finds the old libopenconnect.so in /usr/local/lib *before* the new one it's just built in the build directory. And thus the link fails. That sounds like it might be a libtool/autotools bug â surely it should link against the library it just built, and put -L./.libs on the search path *before* anything else? I was using the latest available tools where given the choice; autoconf 2.69, automake 1.12 and libtool (not GNU libtool) 1.5.26. Should I try with GNU libtool instead? I assume I'm doing something wrong here. Advice on how to make it build properly on OpenBSD would be much appreciated... -- Sent with MeeGo's ActiveSync support. David WoodhouseOpen Source Technology Centre david.woodho...@intel.com Intel Corporation [demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]
Re: crypto volume damaged after crash
On Thu, 8 Nov 2012, Erling Westenvik wrote: > I'm running current on a ThinkPad T500 with a fully encrypted disk (sd0) > and using a usb keydisk (sd1) to assemble the crypto volume on sd2. Last > snapshot upgrade was around 11th of October. > > Yesterday the machine suddenly stopped responding to keystrokes (even > though xscreensaver was running "fine"). Pinging it from one of my other > OpenBSD-machines worked, but when I tried to ssh into it, the connection > just timed out. Finally, when I tried to switch console by hitting > Ctrl-Alt-F2, it froze completely. > > No big deal, I thought. It had crashed numerous times before from empty > battery. So I booted, plugged in the keydisk, but after entering the > usual location for boot and swap partitions: > > root device (default sd0a): sd2a > swap device (default sd2b): sd0b > > I got this: (I had to write this down by hand. FYI, in case of typos.) > > ---8<--- > root on sd2a swap on sd0b dump on sd0b > Automatic boot in process: starting file system check. > /dev/sd2a (290d4f6dcbc2d7a7.a): file system is clean; not checking > softraid0: i/o error on block 257269168 This is the biggest hint at the real issue - for some reason the I/O to the underlying device has failed, which has then been propagated to the softraid volume. Unfortunately, at this stage this is sufficient to force the softraid crypto volume offline, hence from here on in there will be nothing but I/O errors when reading or writing to the volume (hence fsck_ffs complaining). If this is a hardware failure then it is not overly interesting, however if the underlying device is healthy then I would be interested in getting further details. > CANNOT READ: BLK 183692704 > /dev/sd2k (290d4f6dcbc2d7a7.k): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. CANNOT READ: BLK 128 > /dev/sd2d (290d4f6dcbc2d7a7.d): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. CANNOT READ: BLK 128 > /dev/sd2f (290d4f6dcbc2d7a7.f): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. CANNOT READ: BLK 128 > /dev/sd2g (290d4f6dcbc2d7a7.g): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. CANNOT READ: BLK 128 > /dev/sd2h (290d4f6dcbc2d7a7.h): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. CANNOT READ: BLK 128 > /dev/sd2j (290d4f6dcbc2d7a7.j): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. CANNOT READ: BLK 128 > /dev/sd2i (290d4f6dcbc2d7a7.i): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. CANNOT READ: BLK 128 > /dev/sd2e (290d4f6dcbc2d7a7.e): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. THE FOLLOWING FILE SYSTEMS HAD AN UNEXPECTED INCONSISTENSY: > ffs: 290d4f6dcbc2d7a7.k (/home), ffs: 290d4f6dcbc2d7a7.d (/tmp), > ffs: 29 0d4f6dcbc2d7a7.f (/usr), ffs: 290d4f6dcbc2d7a7.g (/usr/X11R6), ffs: > 290d4f6dcbc2 d7a7.h (/usr/local), ffs: 290d4f6dcbc2d7a7.j (/usr/obj), ffs: > 290d4f6dcbc2d7a7.i (/usr/src), ffs: 290d4f6dcbc2d7a7.e (/var) > Automatic file system check failed; help! > Nov 7 23:09:59 init: /etc/pwd.db: Input/output error > Enter pathname of shell or RETURN for sh: > # fsck_ffs 290d4f6dcbc2d7a7.k > ** /dev/sd2k (290d4f6dcbc2d7a7.k ) > > CANNOT READ: BLK 128 > CONTINUE? [Fyn?] > > THE FOLLOWING DISK SECTORS COULD NOT BE READ: 128, 129, 130, 131, 132, 133, > 134, 135, 136, 137, 138, 139, 140, 141, 142, 143 > > LOOK FOR ALTERNATE SUPERBLOCKS? [Fyn?] _ > --->8--- > > Pressing "y" just causes similar messages to pop up "ad infitum". > > Any clues? I got everything backed up but would like to understand what > is going on rather than just do a fresh install. > > Erling -- "Reason is not automatic. Those who deny it cannot be conquered by it. Do not count on them. Leave them alone." -- Ayn Rand
Re: strange cron behavior
Thanks a lot! It is working perfectly. On Thu, Nov 8, 2012 at 8:21 AM, Antoine Jacoutot wrote: > On Thu, Nov 08, 2012 at 08:16:13AM -0200, Friedrich Locke wrote: > > Dear member list, > > > > i am running a backup script. When i exec it from a login shell it works > > nicely altough it shows a message about a file being modified during copy > > to archive. > > > > The backup is performed ok, but when i schedule the script to be executed > > by cron, it stops executing at the first file being modified during copy > to > > archive. > > > > May someone point me my mistaken ? > > Make sure all commands you run are in you PATH. > Default PATH for crontab is: PATH=/bin:/sbin:/usr/bin:/usr/sbin > and for e.g. curl is not in there. > > > > > > Thanks in advance. > > > > Here you have the output executing it from a terminal: > > > > # > > /etc/backup/bk-paginas.sh > > > > tar: Removing leading / from absolute path names in the archive > > % Total% Received % Xferd Average Speed TimeTime Time > > Current > > Dload Upload Total SpentLeft > > Speed > > 0 00 00 0 0 0 --:--:-- --:--:-- > > --:--:-- 0 > > tar: File /home/_syslogd/httpd.acs was modified during copy to archive > > 100 281M0 00 281M 0 11.5M --:--:-- 0:00:24 --:--:-- > > 10.6M > > # > > > > Here you have the script itself: > > > > #!/bin/sh > > > > #Calculando Datas > > DIA_MES=`date +%d` > > MES=`date +%b` > > ANO=`date +%Y` > > > > #Gerando Caminhos e Nomes dos Arquivos > > NOME_SERVIDOR='LOKI' > > PAGINAS_ORIG1='/home/_syslogd' > > PAGINAS_ORIG2='/var/log' > > PAGINAS_DEST='/var/tmp' > > PAGINAS_DEST_DIARIO='/var/tmp' > > > > # Compactando as pastas > > rm -f $PAGINAS_DEST_DIARIO/*PAGINAS*.gz > > > > #tar -czvf > > $PAGINAS_DEST_DIARIO/$NOME_SERVIDOR-PAGINAS_$ANO$MES$DIA_MES.tar.gz > > $PAGINAS_ORIG1 $PAGINAS_ORIG2 > > > $PAGINAS_DEST/$NOME_SERVIDOR-PAGINAS-LOG_$ANO$MES$DIA_MES.log > > tar -czvf $PAGINAS_DEST_DIARIO/p $PAGINAS_ORIG1 $PAGINAS_ORIG2 > > > $PAGINAS_DEST/$NOME_SERVIDOR-PAGINAS-LOG_$ANO$MES$DIA_MES.log & > > > > #cp -f $PAGINAS_DEST_DIARIO/*PAGINAS*.gz $PAGINAS_DEST/. > > > > # Envio para o LAMPIAO via FTP > > cat $PAGINAS_DEST_DIARIO/p | curl -T - > > > ftp://bk_sysop:xxxyyy...@lampiao.cpd.ufv.br/linux/$NOME_SERVIDOR/$NOME_SERVIDOR-PAGINAS_$ANO$MES$DIA_MES.tar.gz&&; > > echo 'Envio OK' | mail -s 'OK - LOKI - PAGINAS' > > backup-...@ufv.br || echo 'Envio ERRO' | mail -s 'ERRO - LOKI - PAGINAS' > > backup-...@ufv.br > > > > -- > Antoine
Re: Benchmark for nginx + php + mysql
Hi, On 2012-11-08 at 08:06 CET "Raindy Long" wrote: >Sorry , my php script just like It's still nowhere near full configuration description... Show you nginx config. >And , I think even the static html file test is unreasonable . That depends on many different factors. Like how many other daemons you have running on this system (and how much ram is left for cache) and what kind of disks you have there. >From: Rafal Bisingier >Date: 2012-11-08 00:42 >To: sopato >CC: misc >Subject: Re: Benchmark for nginx + php + mysql >Hi, > >On Wed, 07 nov 2012 at 23:43 CET >"Raindy Long" wrote: > >> Hi @misc, >> >> Just create a webserver in openbsd5.2 by nginx+php+mysql , hardware is : >> 512M + 2.4G CPU + 40G disk . >> And I do some benchmark by ab/webbench tools, open 100 clients & 10 process >> to do the test . the result is(close nginx access log in all tests) : >> (1)static html file>498 requests/sec >> (2)php file> 284 requests/sec , and five php-fpm process use 100% cpu !! >> :( >> I think the test result is so bad . >> >> next is my /etc/sysctl.conf context: >> --- >> kern.maxvnodes=131072 >> kern.maxproc=65536 >> kern.maxfiles=65536 >> kern.somaxconn=65536 >> kern.sominconn=256 >> kern.maxclusters=32768 >> net.inet.tcp.recvspace=65536 >> net.inet.tcp.sendspace=65536 >> net.inet.udp.recvspace=65536 >> net.inet.udp.sendspace=65536 >> --- >> >> What can I do to improve the performance ? >> Thanks a lot . > >Try this: >echo "" > test.php > >PS. You didn't even show what are you testing (your configuration and >the php script code) and want some improvement advices? You must be >kidding... ;-) -- Greetings Rafal Bisingier
Re: crypto volume damaged after crash
On Thu, Nov 8, 2012 at 3:27 AM, Erling Westenvik wrote: > I'm running current on a ThinkPad T500 with a fully encrypted disk (sd0) > and using a usb keydisk (sd1) to assemble the crypto volume on sd2. Last > snapshot upgrade was around 11th of October. > > Yesterday the machine suddenly stopped responding to keystrokes (even > though xscreensaver was running "fine"). Pinging it from one of my other > OpenBSD-machines worked, but when I tried to ssh into it, the connection > just timed out. Finally, when I tried to switch console by hitting > Ctrl-Alt-F2, it froze completely. > > No big deal, I thought. It had crashed numerous times before from empty > battery. So I booted, plugged in the keydisk, but after entering the > usual location for boot and swap partitions: > > root device (default sd0a): sd2a > swap device (default sd2b): sd0b > > I got this: (I had to write this down by hand. FYI, in case of typos.) > > ---8<--- > root on sd2a swap on sd0b dump on sd0b > Automatic boot in process: starting file system check. > /dev/sd2a (290d4f6dcbc2d7a7.a): file system is clean; not checking > softraid0: i/o error on block 257269168 > CANNOT READ: BLK 183692704 > /dev/sd2k (290d4f6dcbc2d7a7.k): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. > CANNOT READ: BLK 128 > /dev/sd2d (290d4f6dcbc2d7a7.d): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. > CANNOT READ: BLK 128 > /dev/sd2f (290d4f6dcbc2d7a7.f): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. > CANNOT READ: BLK 128 > /dev/sd2g (290d4f6dcbc2d7a7.g): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. > CANNOT READ: BLK 128 > /dev/sd2h (290d4f6dcbc2d7a7.h): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. > CANNOT READ: BLK 128 > /dev/sd2j (290d4f6dcbc2d7a7.j): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. > CANNOT READ: BLK 128 > /dev/sd2i (290d4f6dcbc2d7a7.i): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. > CANNOT READ: BLK 128 > /dev/sd2e (290d4f6dcbc2d7a7.e): UNEXPECTED INCONSISTENCY: RUN fsck_ffs > MANUALLY. > THE FOLLOWING FILE SYSTEMS HAD AN UNEXPECTED INCONSISTENSY: > ffs: 290d4f6dcbc2d7a7.k (/home), ffs: 290d4f6dcbc2d7a7.d (/tmp), ffs: > 29 > 0d4f6dcbc2d7a7.f (/usr), ffs: 290d4f6dcbc2d7a7.g (/usr/X11R6), ffs: > 290d4f6dcbc2 > d7a7.h (/usr/local), ffs: 290d4f6dcbc2d7a7.j (/usr/obj), ffs: > 290d4f6dcbc2d7a7.i > (/usr/src), ffs: 290d4f6dcbc2d7a7.e (/var) > Automatic file system check failed; help! > Nov 7 23:09:59 init: /etc/pwd.db: Input/output error > Enter pathname of shell or RETURN for sh: > # fsck_ffs 290d4f6dcbc2d7a7.k > ** /dev/sd2k (290d4f6dcbc2d7a7.k ) > > CANNOT READ: BLK 128 > CONTINUE? [Fyn?] > > THE FOLLOWING DISK SECTORS COULD NOT BE READ: 128, 129, 130, 131, 132, 133, > 134, > 135, 136, 137, 138, 139, 140, 141, 142, 143 > > LOOK FOR ALTERNATE SUPERBLOCKS? [Fyn?] _ > --->8--- > > Pressing "y" just causes similar messages to pop up "ad infitum". > > Any clues? I got everything backed up but would like to understand what > is going on rather than just do a fresh install. Since you have backups, I suggest you to just do a fresh install and take advantage of the new boot(8) code jsing@ did in -current. Real FDE is simply fantastic! Ciao, David
Re: strange cron behavior
On Thu, Nov 08, 2012 at 08:16:13AM -0200, Friedrich Locke wrote: > Dear member list, > > i am running a backup script. When i exec it from a login shell it works > nicely altough it shows a message about a file being modified during copy > to archive. > > The backup is performed ok, but when i schedule the script to be executed > by cron, it stops executing at the first file being modified during copy to > archive. > > May someone point me my mistaken ? Make sure all commands you run are in you PATH. Default PATH for crontab is: PATH=/bin:/sbin:/usr/bin:/usr/sbin and for e.g. curl is not in there. > > Thanks in advance. > > Here you have the output executing it from a terminal: > > # > /etc/backup/bk-paginas.sh > > tar: Removing leading / from absolute path names in the archive > % Total% Received % Xferd Average Speed TimeTime Time > Current > Dload Upload Total SpentLeft > Speed > 0 00 00 0 0 0 --:--:-- --:--:-- > --:--:-- 0 > tar: File /home/_syslogd/httpd.acs was modified during copy to archive > 100 281M0 00 281M 0 11.5M --:--:-- 0:00:24 --:--:-- > 10.6M > # > > Here you have the script itself: > > #!/bin/sh > > #Calculando Datas > DIA_MES=`date +%d` > MES=`date +%b` > ANO=`date +%Y` > > #Gerando Caminhos e Nomes dos Arquivos > NOME_SERVIDOR='LOKI' > PAGINAS_ORIG1='/home/_syslogd' > PAGINAS_ORIG2='/var/log' > PAGINAS_DEST='/var/tmp' > PAGINAS_DEST_DIARIO='/var/tmp' > > # Compactando as pastas > rm -f $PAGINAS_DEST_DIARIO/*PAGINAS*.gz > > #tar -czvf > $PAGINAS_DEST_DIARIO/$NOME_SERVIDOR-PAGINAS_$ANO$MES$DIA_MES.tar.gz > $PAGINAS_ORIG1 $PAGINAS_ORIG2 > > $PAGINAS_DEST/$NOME_SERVIDOR-PAGINAS-LOG_$ANO$MES$DIA_MES.log > tar -czvf $PAGINAS_DEST_DIARIO/p $PAGINAS_ORIG1 $PAGINAS_ORIG2 > > $PAGINAS_DEST/$NOME_SERVIDOR-PAGINAS-LOG_$ANO$MES$DIA_MES.log & > > #cp -f $PAGINAS_DEST_DIARIO/*PAGINAS*.gz $PAGINAS_DEST/. > > # Envio para o LAMPIAO via FTP > cat $PAGINAS_DEST_DIARIO/p | curl -T - > ftp://bk_sysop:xxxyyy...@lampiao.cpd.ufv.br/linux/$NOME_SERVIDOR/$NOME_SERVIDOR-PAGINAS_$ANO$MES$DIA_MES.tar.gz&&; > echo 'Envio OK' | mail -s 'OK - LOKI - PAGINAS' > backup-...@ufv.br || echo 'Envio ERRO' | mail -s 'ERRO - LOKI - PAGINAS' > backup-...@ufv.br > -- Antoine
strange cron behavior
Dear member list, i am running a backup script. When i exec it from a login shell it works nicely altough it shows a message about a file being modified during copy to archive. The backup is performed ok, but when i schedule the script to be executed by cron, it stops executing at the first file being modified during copy to archive. May someone point me my mistaken ? Thanks in advance. Here you have the output executing it from a terminal: # /etc/backup/bk-paginas.sh tar: Removing leading / from absolute path names in the archive % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 tar: File /home/_syslogd/httpd.acs was modified during copy to archive 100 281M0 00 281M 0 11.5M --:--:-- 0:00:24 --:--:-- 10.6M # Here you have the script itself: #!/bin/sh #Calculando Datas DIA_MES=`date +%d` MES=`date +%b` ANO=`date +%Y` #Gerando Caminhos e Nomes dos Arquivos NOME_SERVIDOR='LOKI' PAGINAS_ORIG1='/home/_syslogd' PAGINAS_ORIG2='/var/log' PAGINAS_DEST='/var/tmp' PAGINAS_DEST_DIARIO='/var/tmp' # Compactando as pastas rm -f $PAGINAS_DEST_DIARIO/*PAGINAS*.gz #tar -czvf $PAGINAS_DEST_DIARIO/$NOME_SERVIDOR-PAGINAS_$ANO$MES$DIA_MES.tar.gz $PAGINAS_ORIG1 $PAGINAS_ORIG2 > $PAGINAS_DEST/$NOME_SERVIDOR-PAGINAS-LOG_$ANO$MES$DIA_MES.log tar -czvf $PAGINAS_DEST_DIARIO/p $PAGINAS_ORIG1 $PAGINAS_ORIG2 > $PAGINAS_DEST/$NOME_SERVIDOR-PAGINAS-LOG_$ANO$MES$DIA_MES.log & #cp -f $PAGINAS_DEST_DIARIO/*PAGINAS*.gz $PAGINAS_DEST/. # Envio para o LAMPIAO via FTP cat $PAGINAS_DEST_DIARIO/p | curl -T - ftp://bk_sysop:xxxyyy...@lampiao.cpd.ufv.br/linux/$NOME_SERVIDOR/$NOME_SERVIDOR-PAGINAS_$ANO$MES$DIA_MES.tar.gz&&; echo 'Envio OK' | mail -s 'OK - LOKI - PAGINAS' backup-...@ufv.br || echo 'Envio ERRO' | mail -s 'ERRO - LOKI - PAGINAS' backup-...@ufv.br
