Re: [OpenIndiana-discuss] OI Hipster becomes unreachable over network after a certain length of uptime

[Doug Hughes]

On 4/11/2022 9:14 AM, Bob Friesenhahn wrote:

On Sun, 10 Apr 2022, Judah Richardson wrote:

OK. The Solaris documentation I linked to says that Solaris (and 
presumably
distros downstream of that codebase) expects the DHCP server to be 
another
Solaris machine, and so DHCP servers that don't behave like the 
latter can

result in unexpected behavior.


The above seems to be a meaningless statement.

DHCP (and its expectations) are not very complicated.

I use the ISC DHCP server here, but any reasonable DHCP server should do.

The DHCP client is expected to contact the server every so often and 
if the server fails to respond for a very long time, the DHCP lease 
will expire and the DHCP client should remove the IP address from the 
interface.


However, it appears likely that the DHCP server and client are not the 
guilty parties here.  It appears that some other software, or 
something in the kernel, is removing the IP address.  The DHCP client 
expects the kernel to retain the IP address until the interface is 
turned down.  It is not going to continually check to verify that 
nothing has disturbed it.


Bob


Just a thought:

Maybe that doc (which I didn't chase down since it fell off of thread) 
is referencing the fact that there are special tags used for jumpstart 
server, image server, image name, etc. That's all I could think off: if 
you are trying to jumpstart a machine. And yeah, those tags do have to 
exist, but you can put them on an isc or any other dhcp server too. It's 
all just a matter of what the client is asking for and what the server 
replies with, the specifics of the OS don't matter, frankly. From a pure 
IP request/renew/accept point of view, the protocol is the same.


___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
https://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS on Openindiana vs ZFS on oracle linux

[Doug Hughes]

On 4/21/2017 2:06 PM, C. R. Oldham wrote:
> For Ubuntu, it is an effort to get ZFS on the root partition.  See
> these wiki entries:
>
> https://github.com/zfsonlinux/zfs/wiki/Ubuntu
>
> I have several machines installed with these instructions on 16.04,
> 16.10, and 17.04.  They do work, but it's a pain to get it installed.
>
> Note that ZoL does have a couple of limitations that can make it
> challenging to use it as the backing store for KVM.  We have an
> OpenNebula private cloud and some of qemu's caching options will cause
> the VMs not to start, claiming the FS underneath doesn't expose the
> right primitives.  I left the OS on ZFS on those machines and
> converted the partitions where the VMs keep their disks to XFS.
Not to Hijack, but:

I struggled with KVM on XFS for months before giving up. XFS is somewhat
fundamentally incompatible with qcow2. If you are using raw, I'm sure
it's great. Sine we use snapshots for backups and replication, the qcow2
functionality is a must-have. Even on a 96GB machine on a large (~40TB)
qcow2 image, the fragmentation in XFS got to be enough that it was
struggling to get 4MB allocations for XFS, and XFS would just fill the
logs. Even echo 3> /proc/sys/vm/drop_caches and tuning linux to have at
least 8MB available and use hugetlb and tuning scanning were not enough.
XFS would get irrevlocably stuck and require a hard reset of the entire
machine to free.

You *can* tune xfs allocation sizes, but only on an empty file. You
can't do it after the fact, which makes discovering that very annoying.
You can also completely preallocate a qcow2, but I'm not sure it's
helpful in the long run.

Running KVM on ZoL on the other hand, has been wonderful so far with
qcow2. (CentOS7.2)

Anyway, just saying, XFS is great until it isn't, and it's failure cases
will bite you months down the road and you may not have options. I'm
looking forward, soon, to having another zfs machine to replicate too,
then I'll use KVM+QCOW2 to take the snapshot, use zfs to replicate, then
blockcommit and done. Right now it's rsync and it takes way too long.



___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
https://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] NSA Solaris hacks

[Doug Hughes]

It has been a topic of discussion on the illumos developers list today.


On 4/11/2017 3:25 PM, jason matthews wrote:
>
> https://www.theregister.co.uk/2017/04/11/solaris_shadow_brokers_nsa_exploits/
>
>
> has anyone reviewed this for relevancy?
>
>
> j.
>
>
> ___
> openindiana-discuss mailing list
> openindiana-discuss@openindiana.org
> https://openindiana.org/mailman/listinfo/openindiana-discuss


___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
https://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] mbuffer connection refused on varius ports ... what to try

[Doug Hughes]

On 3/28/2017 5:24 AM, Harry Putnam wrote:
> Timothy Coalson  writes:
>
>> On Mon, Mar 27, 2017 at 9:44 PM, Harry Putnam  wrote:
>>
>>> Geoff Nordli  writes:
>>>
>>> [...]
>>>
 Just a thought here, you may want to try a different ssh cipher. Give
 arcfour a try and see if that is fast enough for you.

>>> Can that be done on ssh command line or must be done in ssh_config?
>>>
>>> You are transferring via mbuffer through a separate TCP connection, so
>> changing the ssh cipher won't change the speed.
> I think what Timothy C has in mind is dropping mbuffer/tamp and using
> ssh for the whole thing
>
>  zfs send p/fs@now | ssh RECV_HOST zfs recv p/fs
>

In my experience, while using mbuffer has some extra pitfalls in terms
of sockets and keeping up with things when there are issues (sometimes
leaving mbuffer's lying around, which was subsequently fixed in the
mbuffer code with SIGCHLD), it's a lot more efficient on the network and
faster to do the backups by 30%. If that isn't so important, ssh works
pretty well.

___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
https://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] send/receive .. I expected .zfs directories

[Doug Hughes]

silly question: is the filesystem mounted on the receive side? if you just sent 
it, you'll want to mount it.

Sent from my android device.

-Original Message-
From: Harry Putnam 
To: openindiana-discuss@openindiana.org
Sent: Sat, 25 Mar 2017 19:35
Subject: [OpenIndiana-discuss] send/receive .. I expected .zfs directories

I just completed my very first send/receive.  I was suprised to find
no .zfs directories once the recv completed.
Did I do something wrong or is that normal?
Am I supposed to create them?

Here is what I did in general.

On send-host I had two fs p0/vb p0/vb/vm
Sent/recv'ed p0/vb and then ditto for p0/vb/vm

on recv host I tried first to just create p0.  But that didn't seem to
work when attempting to send p0/vb. Eventually I tried creating p0/vb
on recv host... then an error message told me to use -F on send host.
So:
  `zfs send -v p0/vb | ssh recv-host zfs recv -vF p0/vb'

That worked.
Then without creating anything more on recv host:

 `zfs send -v p0/vb/vm | ssh recv-host zfs recv -v p0/vb/vm'

And that also worked.

I really don't now what or how experienced people do these things.
I do know that in general the docs on zfs promote using fairly fine
grained fs.  I assume that means as in the case above.

Instead of one fs p0/vb/vm I have two as shown above... Ok when it
comes to sending and receiving those fs, it would seem then that one
must send first p0/vb and then p0/vb/vm If one had fs several layers
deeper or with more endpoints, it seems this could get quite tedious.

I suspect I may be missing the boat here somewhere but lets say it was
even more fine grained.  Like an fs for each vm and one for the ISOs
required.

  p0/vb/
  p0/vb/vm
  p0/vb/vm/vm1
  p0/vb/vm/vm2
  p0/vb/vm/vm3
  p0/vb/vm/vm4
  p0/vb/isos

Then it seem this would require a whole lot of sending receiving, to
move this data.

Seven separate send receives by my count.

So I'm convinced that either fine grainedness is not so wonderful or
I'm really not understanding how this is done.

I wonder how folks with some experience in this stuff, do this sort of
thing?



___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
https://openindiana.org/mailman/listinfo/openindiana-discuss
___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
https://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] arp response tuning for IP Source Guard

[Doug Hughes]

It seems to me that you might be hitting up against "arp_defend_rate" 
which by default says that the maximum arps it should be expecting in 
one hour is 100. It's he's sending 3 per minute, that's already 180. I 
could be wrong. I'd probably try setting that to 300 and confirm what's 
going on by using "snoop arp" and then focussing in on the mac address 
of the switch and seeing how many are coming in an hour.




On 1/6/2017 2:30 AM, the outsider wrote:

Are you or the technicians SURE that this technique is still valid for
current world with a lot of virtual servers attached to one switch port?
This technique was invented in a time that every switchport was connected to
at most one MAC address, so when the switch detects more than one MAC
address at a port there was something illegal happening. (or the switchport
was connected to a switch)

-Oorspronkelijk bericht-
Van: Tim Mooney [mailto:tim.moo...@ndsu.edu]
Verzonden: vrijdag 6 januari 2017 0:50
Aan: openindiana-discuss@openindiana.org
Onderwerp: Re: [OpenIndiana-discuss] arp response tuning for IP Source Guard

In regard to: Re: [OpenIndiana-discuss] arp response tuning for IP
Source...:


On 01/05/17 15:37, Tim Mooney wrote:

When that was enabled for the subnet I'm on, my hipster workstation
and the hipster VirtualBox VM I have both started experiencing packet

loss.

Talking with the network engineers, the Cisco switch is sending
batches of 3 ARP probes periodically, and both my workstation and the
VM appear to be periodically not responding to the ARP probes.  That
causes the switch to temporarily ban/block packets from either
system, which is what's causing the intermittent packet loss.

Anyone have any suggestions for what tuning I should be looking at
that would tell the Illumos network stack that it's OK to respond to
semi-frequent batches of ARP probes?

It would be great to see the syslog messages and (if possible) a
packet trace showing what's going on.  In general, if the system
itself is directly responsible for these outages, it will at least log
something about the event.

At the log level I've been running at, there hasn't been anything useful
logged related to this.  If necessary, I can definitely dial up the logging.


Are these ARP requests or responses?  There are subtle differences
between the two.

According to our principal network engineer, the Cisco switch was defaulting
to sending 3 ARP probes (in quick succession) every 60 seconds.  He has
since dialed that back to just 1 per 60 seconds for this particular switch,
to see if that had any impact on the issue, but it did not.

He's done a bunch more research since I sent my initial question to this
list, and right now he thinks the issue may be that the ARP probe from the
Cisco switch is unicast, but Solaris apparently may be issuing ARP responses
as *broadcast*, which the switch may not be expecting.

The reference he found related to broadcast ARP responses is here:

http://seclists.org/nmap-dev/2009/q1/176
  
http://unix.derkeiler.com/Mailing-Lists/SunManagers/2009-01/msg00015.html


He's also suggested that I might be able to set 'arp_defend_interval'
to something like 20 seconds, so that my workstation just periodically sends
unsolicited ARPs for itself, to essentially preempt the switch's probes.
Based on the docs he found:

http://docs.oracle.com/cd/E36784_01/html/E36845/gnogz.html

Since the docs say "Never" in answer to the "When to change" for any of
these settings, I haven't actually tried setting arp_defend_interval.
The way I read the docs, it seems like arp_publish_interval might be better,
but I know better than to argue with our principal network engineer about
anything network related.  :-)


Based on what I remember from working on this code many years ago, one
of the really confusing bits to deal with is Ethernet bridge
("switch") behavior itself.  Many bridges (I think at least Extreme,
and probably
others) have special mechanisms built-in to protect against ARP
storms, and they rate-limit based on the number of broadcasts.  This
is (I
believe!) independent of any sort of "Source Guard" feature.  I ran
into this issue numerous times when testing Solaris IP Duplicate
Address Detection.

Thanks much for the response!

Tim



___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
https://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] using touch ... whats wrong here

[Doug Hughes]

That is the classical answer for ctime. James is correct. However some may find 
it interesting to note that GPFS does indeed keep the original creation time as 
an additional attribute that can then be used for policy applications. It is 
not exposed to Unix, but it's there. Alas, GPFS is only available on AIX and 
Linux at this time.

(misc morning info share)

Sent from my android device.

-Original Message-
From: James Carlson 
To: openindiana-discuss@openindiana.org
Sent: Thu, 03 Dec 2015 8:24
Subject: Re: [OpenIndiana-discuss] using touch ... whats wrong here

On 12/03/15 04:56, Peter Tribble wrote:
> You're mixing the various timestamps.

Indeed; that's the problem.

> access, shown by ls -lu
> modify (of data), shown by ls -l
> creation (or modify metadata), shown by ls -lc

Perhaps a nit, but one I think is important.  UNIX (including
OpenIndiana) does not keep track of file or directory "creation" time.
The 'ctime' element tracks the last attribute change time.  There's no
way (in UNIX) of knowing when a file was created.

And, yes, the OP should be using the file's mtime (the "-newer" option).
 The attribute change time is rarely what you want.  :-/

-- 
James Carlson 42.703N 71.076W 

___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] What is the recommended way to back up root pool?

[Doug Hughes]

I'd put that in my config mgmt system (or installer)


Sent from my android device.

-Original Message-
From: Bob Friesenhahn <bfrie...@simple.dallas.tx.us>
To: Discussion list for OpenIndiana <openindiana-discuss@openindiana.org>
Sent: Wed, 28 Oct 2015 21:00
Subject: Re: [OpenIndiana-discuss] What is the recommended way to back up root 
pool?

On Wed, 28 Oct 2015, Doug Hughes wrote:

> for home or for office?
> for office, I don't back up root pool. it's considered disposible and
> reproducible via reinstall. (that plus config management)
> for home, you can zfs send it somewhere to a file if you want, or you can
> tar it up since that's probably easier to restore individual files after an
> oops. I do the latter. Then you could reinstall from golden image and
> restore the files you need.

Assume that this is for a network server with advanced network 
configuration settings, ssh config, zones, etc.

If was the same as a standard OS install without subsequent 
configuration, then backing up would not be so important.

Bob
-- 
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/

___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] What is the recommended way to back up root pool?

[Doug Hughes]

for home or for office?
for office, I don't back up root pool. it's considered disposible and
reproducible via reinstall. (that plus config management)
for home, you can zfs send it somewhere to a file if you want, or you can
tar it up since that's probably easier to restore individual files after an
oops. I do the latter. Then you could reinstall from golden image and
restore the files you need.


On Wed, Oct 28, 2015 at 7:24 PM, Bob Friesenhahn <
bfrie...@simple.dallas.tx.us> wrote:

> What is the recommended approach to back up a zfs root pool?
>
> For other pools I use zfs send/receive and/or rsync-based methods.
>
> The zfs root pool is different since it contains multiple filesystems,
> with the filesystem for one one BE being mounted at a time:
>
> % zfs list -r -t filesystem rpool
> NAMEUSED  AVAIL  REFER  MOUNTPOINT
> rpool  79.7G   377G50K  /rpool
> rpool/ROOT 29.5G   377G31K  legacy
> rpool/ROOT/openindiana 15.8M   377G  3.15G  /
> rpool/ROOT/openindiana-1   38.9M   377G  5.97G  /
> rpool/ROOT/openindiana-2   40.6M   377G  11.3G  /
> rpool/ROOT/openindiana-2-backup-1   124K   377G  10.5G  /
> rpool/ROOT/openindiana-3   48.4M   377G  13.2G  /
> rpool/ROOT/openindiana-3-backup-176K   377G  11.4G  /
> rpool/ROOT/openindiana-3-backup-245K   377G  11.5G  /
> rpool/ROOT/openindiana-3-backup-3   123K   377G  11.9G  /
> rpool/ROOT/openindiana-3-backup-444K   377G  12.1G  /
> rpool/ROOT/openindiana-4   20.1M   377G  18.8G  /
> rpool/ROOT/openindiana-4-backup-195K   377G  13.3G  /
> rpool/ROOT/openindiana-4-backup-2   156K   377G  18.8G  /
> rpool/ROOT/openindiana-5   17.5M   377G  18.9G  /
> rpool/ROOT/openindiana-6   29.4G   377G  17.6G  /
> rpool/export121M   377G32K  /export
> rpool/export/home   121M   377G32K  /export/home
>
> This means that there are multiple filesystems which would need to be
> backed up in order to save a replica of the pool.
>
> At the moment I am using rsync-based backup of only selected filesystems.
>
> It is not clear to me where configuration due to utilities like 'ipadm'
> and 'dladm' is stored, but I am pretty sure it is to files under the /etc
> directory.
>
> What is recommended/common practice for backing up the root pool?
>
> Bob
> --
> Bob Friesenhahn
> bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
>
> ___
> openindiana-discuss mailing list
> openindiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
>
___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Antwort: Transferring everything in one zpool to be a, subtree in another larger zpool (Hans J. Albertsson)

[Doug Hughes]

you can also use ttcp or iperf just fine. I do that all the time and they are 
pretty much available anywhere. (ttcp is so simple it's just a single .c source 
file). The buffering isn't really the important part, using as close to raw tcp 
transport gets you the biggest benefit (vs encryption/decryption overhead and 
slowdown).


Sent from my android device.

-Original Message-
From: Hans J. Albertsson hans.j.alberts...@gmail.com
To: Discussion list for OpenIndiana openindiana-discuss@openindiana.org
Sent: Tue, 05 May 2015 9:31
Subject: Re: [OpenIndiana-discuss] Antwort: Transferring everything in one 
zpool to be a, subtree in another larger zpool (Hans J. Albertsson)

mbuffer isn't included in FreeNAS.

It IS available for FreeBSD, but I don't know if that version will 
behave responsibly in FreeNAS.

FreeNAS devs say mbuffer won't be included in FreeNAS because we 
already do proper buffering via dd.

Would dd work for buffering in Solaris/OI??

On 2015-05-05 14:26, Sebastian Gabler wrote:
 Am 05.05.2015 um 14:00 schrieb 
 openindiana-discuss-requ...@openindiana.org:
 As a side note; just piping to ssh is excruciatingly slow, using netcat,
 nc, speeds things up at least 4fold.
 The method that worked fine for me was using mbuffer. (I couldn't make 
 netcat work, shame on me, but I also remember that there were several 
 pros of using mbuffer. I think one of them was the ability to use 
 digest to secure the transfer.)

 See: 
 http://everycity.co.uk/alasdair/2010/07/using-mbuffer-to-speed-up-slow-zfs-send-zfs-receive/

 That being said, it's exactly two years when I last did a migration 
 with zfs send | zfs receive. I hope that still works.

 BR,

 Sebastian

 ___
 openindiana-discuss mailing list
 openindiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss


___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] data on rpool - Bad idea?

[Doug Hughes]

The reason that data volumes on rpool is generally not a good idea is for 
recoverability. You can take all of the disks of a given pool and move them to 
another system, except rpool. rpool defines the system itself, so data volumes 
there are tied to that system. Data volumes in any other pool can move between 
systems very easily, which is great when you have certain classes of hardware 
failures, or you want to upgrade your system, or something similar. The data is 
thus independent of the current disposition of the OS.

At some point later on, you'll likely regret having done it.

Sent from my android device.

-Original Message-
From: Harry Putnam rea...@newsguy.com
To: openindiana-discuss@openindiana.org
Sent: Sat, 07 Feb 2015 7:24
Subject: [OpenIndiana-discuss] data on rpool - Bad idea?

I've read that it is a bad idea to have datasets on the same disc
that rpool is on.  I'm not sure why that is so.

In my current situation I need more space but to rig up my
`HP xw8600 workstation' with more disc space will require getting a cage
and some new discs, not to mention the huffing and puffing necessary
to get it all installed and usable.

When I installed oi on this hardware I thought it was a smart idea to
use 2 older 250 GB discs for a 2 disc mirrored setup of rpool.  And a
pair of 1tb discs for 2disc mirror of datasets.

At that time I hadn't really realized what it would take to add
discs. 

But of course I'm now running out of space... and still not flush
enough to pop for the $400 or so to add a cage and 4 2-3 tb discs.

However I do have 2 1tb discs laying unused.

So, (finally cutting to chase) how bad of an idea would it be to use
those two 1tb discs for a mirrored rpool and whatever data.

I would gain about 800-850 GB of mirrored data space but it would
involve putting data on same discs and rpool.

(I'll leave for another thread to discuss how to get rpool onto the new
discs in the most painless manner)

So, what are your best thoughts on employing the two 1tb discs as
described above?


___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] A ZFS related question: How successful is ZFS, really???

[Doug Hughes]

Couple of points and counter points from my own experience.
*) tape really isn't dead. No, really. at about $.01/GB/copy, and 1x10^20
bit error rate, you can't beat it. Use it for the right thing though. This
excels as an offline archival media with media lifetimes expected at around
30 years. Contrast that to cheap disk. Everybody forgets that you still
have to account for the chassis, memory, space, transportability, media
bandwidth, brackets, cpus, and power used by an online storage system.
Also, if you go with the really inexpensive disks to keep costs down, you
sacrifice 5 orders of magnitude of media reliability when compared to tape.
Seriously. There's a place for both. Places serious about data integrity
and offsite archive at good cost still use tape. it's not dead. It's not
even dying contrary to what some popular reading might say in some places.
You can backup zfs to tape using zfs diff to get your list and then
whatever mechanism you want to get those items to tape (rsync, tsm,
whatever). The point is, that full-filesystem-scan for backups is
definitely dead. Don't do that. There are better mechanisms. Full scan
doesn't scale.

*) ZFS is really cool and all. We use it a lot! To say it is unquestionably
the best might be a bit of an overstatement. GPFS is also really cool, and
arguably better in many ways. The erasure coding in the new de-clustered
RAID is better than ZFS in terms of rebuild time and certain rare causes of
data loss. GPFS allows easy separation of metadata onto fast storage for
index and search. The GPFS policy engine is way cool. You can arbitrarily
relayout your GPFS across storage. You can have pools of different
characteristics and do policy migration or data placement policies
arbitrarily to use them. (We STILL can't do any relayout of any kind on
ZFS. Argh).  The big downside is price, of course. There are plenty of free
ZFS solutions available but none for GPFS. BTRFS has a technological edge
over zfs in relayout as well. Why isn't it possible to shrink/relayout ZFS
still? IMHO this should have been delivered about 2 years ago. As far as
stability and marketshare go, ZFS gets the big win, naturally.


On Mon, Jan 12, 2015 at 9:56 AM, Hans J Albertsson 
hans.j.alberts...@gmail.com wrote:

 Does anyone have anything beyond own impressions and war stories?

 Is anyone collecting statistics on storage solutions sold?

 Hans J. Albertsson
 From my Nexus 5
 Den 12 jan 2015 15:24 skrev Schweiss, Chip c...@innovates.com:

  On Mon, Jan 12, 2015 at 8:17 AM, Andrew Gabriel 
  illu...@cucumber.demon.co.uk wrote:
 
  
   Since you mention Sun/Oracle, I don't see them pushing ZFS very much
   anymore, although I am aware their engineers still work on it.
  
 
  Oracle pushes ZFS hard and aggressively.   I dare you to fill out their
  contact form or download their virtual appliance demo.  Their sales
 people
  will be calling within the hour.
 
  We just recently went through a bidding war on an HA + DR system with 1/2
  PB useable storage with many vendors including Nexenta and Oracle.
 Oracle
  was price competitive with Nexenta and is in my opinion a much more
  polished product.
 
  We still chose to build our own on OmniOS because we could still do that
  for about 1/2 the price of Oracle / Nexenta.  That's less than 1/4 the
  price of 3par/HP, IBM, Dell/Compellent   BTW, our OmniOS build is on the
  exact same hardware, Nexenta would have been.
 
  -Chip
  ___
  openindiana-discuss mailing list
  openindiana-discuss@openindiana.org
  http://openindiana.org/mailman/listinfo/openindiana-discuss
 
 ___
 openindiana-discuss mailing list
 openindiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss

___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] How to revert a permissive chmod

[Doug Hughes]

Well, it's not the exactly same, but you could do
chmod -R go-w .

On 1/11/2015 3:33 PM, Harry Putnam wrote:

Is there some similarly all incompassing command to revert:

  /bin/chmod -R A=everyone@:full_set:fd:allow /some-dir

back to a more default chmod 755 on directories and 644 on type -f files?
(and no acls set)

I mean short of using `find' in comination with chmod.


___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss



___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] How to revert a permissive chmod

[Doug Hughes]

You could use A= with -R as long as you know the number of the item that 
you are trying to modify (visible with -v or -V depending on your 
viewing pleasure)

It's probably the 'everyone' one...

On 1/11/2015 4:21 PM, Harry Putnam wrote:

Doug Hughes d...@will.to writes:


Well, it's not the exactly same, but you could do
chmod -R go-w .

Has no effect on all the thousands of type -f files


___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss



___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] destroy all but specific snapshot

[Doug Hughes]

On 9/30/2014 8:31 PM, Tim Aslat wrote:

Harry Putnam wrote on 01/10/2014 09:52:

This is not so easy to find in google searches

How does one go about destroying all but a specific snapshot?  The one
I want is somewhere in the middle timewise So not wanting to use
`destroy -r'.

This is in a .zfs/snapshot/* where there are many auto snaps. So
looking for a way besides destroying -r or 1 by 1.


Hi Harry,


Simplest option would be something like

zfs list -t snapshot -o name -H -r filesystem/path  /tmp/snaplist.txt

edit /tmp/snaplist.txt to remove the snapshot you want to keep

for SNAP in `cat /tmp/snaplist.txt`
do
zfs destroy $SNAP
done




Here's an equivalent one liner for you.

zfs list -t snapshot -H pool | grep -v the one you want to exclude | 
awk '{print $1}' | xargs -n 1 zfs destroy


(you should run up to the last pipe and confirm that's what you want 
before adding on the xargs. ;)


* warning: if you have replication to a backup server, removing all of 
the snapshots for other filesystems will make those replicated snapshots 
non-operative.



___
openindiana-discuss mailing list
openindiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] fuiture small expansion of raidz1 possible?

[Doug Hughes]

On 2/11/2014 5:57 PM, Brogyányi József wrote:

And on a similar note: Suppose I have a signgle disk with data, and I
decide I can just afford a raidz1 of 4 disks, so I buy 3 more can I
somehow migrate the data on the original single disk onto a 3 disk
raidz1 and then add the original disk to that raidz?

You do realize that this will be effectively a RAID-0 reliability with 
RAID-5 performance? It's an interesting concept, but I'd be very worried 
about a single block error causing data loss and real problems.


Maybe your window is short enough that you don't mind the risk.



Hi
Here is the technik:
http://www.dev-eth0.de/creating-raidz-with-missing-device/
It must work.
Brogyi

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Powerloss protected SSDs for ...Re: Low low end server

[Doug Hughes]

Why not Intel 320 series? Also 710 series work fine for this, for a bit
more $$ and a bit more speed. The 320 are not as fast as the S3700 or S3500
but they are a LOT less expensive.



On Mon, Feb 10, 2014 at 4:32 PM, Schweiss, Chip c...@innovates.com wrote:

 On Mon, Feb 10, 2014 at 5:22 AM, Hans J. Albertsson 
 hans.j.alberts...@branneriet.se wrote:

  Samsung 843




 The 843 while called and enterprise SSD, does not have capacitors for power
 loss protection.

 http://www.thessdreview.com/our-reviews/samsung-843/2/
 ___
 OpenIndiana-discuss mailing list
 OpenIndiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Powerloss protected SSDs for ...Re: Low low end server

[Doug Hughes]

true, Volker..

Just to note though, the 320s have no battery, but they do have enough
capacitor to flush anything from the small ram into flash on power outage.



On Mon, Feb 10, 2014 at 4:58 PM, Volker A. Brandt v...@bb-c.de wrote:

  Why not Intel 320 series? Also 710 series work fine for this, for a
  bit more $$ and a bit more speed. The 320 are not as fast as the
  S3700 or S3500 but they are a LOT less expensive.

 This thread started out as a discussion of the merits of the HP N54L
 microserver for home use.  I am not really sure if a home server needs
 mirrored battery-protected SSDs.  :-)


 Regards -- Volker
 --
 
 Volker A. Brandt   Consulting and Support for Oracle Solaris
 Brandt  Brandt Computer GmbH   WWW: http://www.bb-c.de/
 Am Wiesenpfad 6, 53340 Meckenheim, GERMANYEmail: v...@bb-c.de
 Handelsregister: Amtsgericht Bonn, HRB 10513  Schuhgröße: 46
 Geschäftsführer: Rainer J.H. Brandt und Volker A. Brandt

 When logic and proportion have fallen sloppy dead

 ___
 OpenIndiana-discuss mailing list
 OpenIndiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Low low end server

[Doug Hughes]

smartctl -x reports the wear on decent ones (read: you shouldn't consider
any that doesn't have this feature). When it gets close to 0, or you see a
lot of errors, it's time to replace it.


On Mon, Feb 10, 2014 at 5:00 PM, Brogyányi József bro...@gmail.com wrote:

 2014.02.09. 22:19 keltezéssel, Jim Klimov írta:

  2*120Gb Samsung Pro SSDs (with powerloss
 protection, formatted to use 100Gb for mirrored rpool/mirrored
 zil/striped l2arc since their 100Gb sub-model has much higher
 reliability and speed - but is not on sale here).

 Jim

 Only one question about the SSD usage. How do I know the SSD status?
 You know the SSD capacity is decreasing. When have to replace it?
 Regards
 Brogyi




 ___
 OpenIndiana-discuss mailing list
 OpenIndiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Powerloss protected SSDs for ...Re: Low low end server

[Doug Hughes]

capacitors are better. Batteries wear out and are difficult to have the
correct monitoring for replacement.


On Mon, Feb 10, 2014 at 5:51 PM, Ben Taylor bentaylor.sol...@gmail.comwrote:

 Surprised someone hasn't developed a SATA power cable with small battery
 Passthrough for this exact application.


 On Mon, Feb 10, 2014 at 5:06 PM, Doug Hughes d...@will.to wrote:

  true, Volker..
 
  Just to note though, the 320s have no battery, but they do have enough
  capacitor to flush anything from the small ram into flash on power
 outage.
 
 
 
  On Mon, Feb 10, 2014 at 4:58 PM, Volker A. Brandt v...@bb-c.de wrote:
 
Why not Intel 320 series? Also 710 series work fine for this, for a
bit more $$ and a bit more speed. The 320 are not as fast as the
S3700 or S3500 but they are a LOT less expensive.
  
   This thread started out as a discussion of the merits of the HP N54L
   microserver for home use.  I am not really sure if a home server needs
   mirrored battery-protected SSDs.  :-)
  
  
   Regards -- Volker
   --
  
 
   Volker A. Brandt   Consulting and Support for Oracle
 Solaris
   Brandt  Brandt Computer GmbH   WWW:
 http://www.bb-c.de/
   Am Wiesenpfad 6, 53340 Meckenheim, GERMANYEmail:
 v...@bb-c.de
   Handelsregister: Amtsgericht Bonn, HRB 10513  Schuhgröße:
 46
   Geschäftsführer: Rainer J.H. Brandt und Volker A. Brandt
  
   When logic and proportion have fallen sloppy dead
  
   ___
   OpenIndiana-discuss mailing list
   OpenIndiana-discuss@openindiana.org
   http://openindiana.org/mailman/listinfo/openindiana-discuss
  
  ___
  OpenIndiana-discuss mailing list
  OpenIndiana-discuss@openindiana.org
  http://openindiana.org/mailman/listinfo/openindiana-discuss
 
 ___
 OpenIndiana-discuss mailing list
 OpenIndiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] real-time syncing two directories across servers (unison ?)

[Doug Hughes]

On 12/11/2013 8:13 PM, Edward Ned Harvey (openindiana) wrote:

From: Gregory Youngblood [mailto:greg...@youngblood.me]

Check out owncloud. The open source components might be useful.


I personally, and two other IT guys that I've spoken with from different 
companies, have been burned by placing any trust in owncloud.  In fact, I'm 
still subscribed to their mailing list, and at least once a day, new people 
write in, asking for help, because it either fails to sync files it should 
sync, or some data seems to be lost.  My advice:  Steer clear.

In Microsoft, what you're looking for is called DFS.  I don't know any good 
linux/unix based bidirectional tools such as requested, but if you start 
searching for alternatives to DFS (or if you simply pay MS and deploy DFS) then 
you have an answer, and hopefully can find some FOSS solution out there.

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss



Seafile seems to work reasonably well. We've actually been looking for 
something with more capability though, one that can handle various 
levels of permissions and multiple-projects. But, if all you want is 
something to plugin to your own website with Dropbox like capabilities 
and multi-device synchronization capability, Seafile seems to be ready 
to fill that bit.



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] VMware

[Doug Hughes]

We have lacp working between force10, hp, and cisco switches in all possible 
combinations with no difficulties. We do monitor and alert on excessive errors 
and drops for interfaces, but lacp isnt a culprit. If anything, it's an 
underlying interface when we find them. Also, it beats the heck out of spanning 
tree and is 2 orders of magnitude simpler than ospf, and 1 order simpler and 
more portable than ecmp. I am quite surprised by your observations.

Sent from my android device.

-Original Message-
From: Edward Ned Harvey (openindiana) openindi...@nedharvey.com
To: Discussion list for OpenIndiana openindiana-discuss@openindiana.org
Sent: Tue, 13 Aug 2013 7:22 AM
Subject: Re: [OpenIndiana-discuss] VMware

 From: James Relph [mailto:ja...@themacplace.co.uk]
 Sent: Monday, August 12, 2013 4:47 PM
 
 No, we're not getting any ping loss, that's the thing.  The network looks
 entirely faultless.  We've run pings for 24 hours with no ping loss.

Yeah, I swore you said you had ping loss before - but if not - I don't think 
ping alone is sufficient.  You have to find the error counters on the LACP 
interfaces.  Everybody everywhere seems to blindly assume LACP works reliably, 
but to me, simply saying the term LACP is a red flag.  It's extremely 
temperamental, and the resultant behavior is exactly as you've described.

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Sun Fire

[Doug Hughes]

Most of the early marvell problems have been addressed with software fixes in 
the oracle release of solaris. I couldnt say about OI. Yes, the 4540 is better 
all around.

Sent from my android device.

-Original Message-
From: Ben Taylor bentaylor.sol...@gmail.com
To: Discussion list for OpenIndiana openindiana-discuss@openindiana.org
Sent: Fri, 21 Jun 2013 1:34 PM
Subject: Re: [OpenIndiana-discuss] Sun Fire

x4540s use the LSI controller.  We got an upgrade on our x4500 because of
systemic problems with our Raid-z setup and the Marvel controllers and the
drivers (This has been several years, maybe 4)


On Fri, Jun 21, 2013 at 4:48 PM, Gary Gendel g...@genashor.com wrote:

 On 06/21/2013 11:56 AM, Peter Tribble wrote:

 On Fri, Jun 21, 2013 at 1:58 PM, Gary Gendel g...@genashor.com wrote:

  I have a possibility of picking up a decommissioned X4340 (thumper)


 Hm. An X4340 is some sort of power transfer unit.

 Thumper = X4500

 Thor = X4540


 Whoops, it's the X4540



  w/o disks


 That's the killer. These boxes are very fussy about the drives that work;
 the
 real value of them now is for the disks they contain, you're going to have
 trouble getting drives (and it's going to cost).

 Interesting.  I thought they used the same Marvell MV88SX6081 8-port
 controllers I picked up for the v20z.  I've been using that for modern WD
 and Seagate SATA drives without issue.



  to replace my aging V20z ( and multiple external drive stacks cheap.  It
 looks like it should be supported by looking at the HCL.

 Is there any gotchas I should be aware of before I commit to purchase?
  Is
 there something more modern I should be looking at instead?

  Probably look at something more modern, and something sized to suit.
 It really needs to be full of 48 drives, only a couple of models are
 supported,
 and they're all getting a bit old.

 Just looking for something cheap and big enough for at least 8 drives.  I
 currently have two 5-drive external cases.  Since I haven't gotten the sata
 multiplexers to work, I have all 8 sata cables running out of the marvell
 controller directly to each drive, my own personal octopus.  Can't get much
 more of a kludge than that.

 Gary



 __**_
 OpenIndiana-discuss mailing list
 OpenIndiana-discuss@**openindiana.orgOpenIndiana-discuss@openindiana.org
 http://openindiana.org/**mailman/listinfo/openindiana-**discusshttp://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Recommendations for fast storage (OpenIndiana-discuss Digest, Vol 33, Issue 20)

[Doug Hughes]

some of these points are a bit dated. Allow me to make some updates. I'm sure 
that you are aware that most 10gig switches these days are cut through and not 
store and forward. That's Arista, HP, Dell Force10, Mellanox, and IBM/Blade. 
Cisco has a mix of things, but they aren't really in the low latency space. The 
10g and 40g port to port forwarding is in nanoseconds. buffering is mostly 
reserved to carrier operations anymore, and even there it is becoming less 
common because of the toll it causes to things like IPVideo and VOIP. Buffers 
are good for web farms, still, and to a certain extent storage servers or WAN 
links where there is a high degree of contention from disparate traffic.
  
At a physical level, the signalling of IB compared to Ethernet (10g+) is very 
similar, which is why Mellanox can make a single chip that does 10gbit 40gbit, 
and QDR and FDR infiniband on any port.
 there are also a fair number of vendors that support RDMA in ethernet NIC now, 
like SolarFlare with Onboot technology.

The main reason for lowest achievable latency is higher speed. Latency is 
roughly equivalent to the inversion of bandwidth.  But, the higher levels of 
protocols that you stack on top contribute much more than the hardware 
theoretical minimums or maximums. TCP/IP is a killer in terms of adding 
overhead. That's why there are protocols like ISER, SRP, and friends. RDMA is 
much faster than the kernel overhead induced by TCP session setups and other 
host side user/kernel boundaries and buffering. PCI latency is also higher than 
the port to port latency on a good 10g switch, nevermind 40 or FDR infiniband.

There is even a special layer that you can write custom protocols to on 
Infiniband called Verbs for lowering latency further.

Infiniband is inherently a layer1 and 2 protocol, and the subnet manager 
(software) is resposible for setting up all virtual circuits (routes between 
hosts on the fabric) and rerouting when a path goes bad. Also, the link 
aggregation, as you mention, is rock solid and amazingly good. Auto rerouting 
is fabulous and super fast. But, you don't get layer3. TCP over IB works out of 
the box, but adds large overhead. Still, it does make it possible that you can 
have IB native and IP over IB with gateways to a TCP network with a single 
cable. That's pretty cool.


Sent from my android device.

-Original Message-
From: Edward Ned Harvey (openindiana) openindi...@nedharvey.com
To: Discussion list for OpenIndiana openindiana-discuss@openindiana.org
Sent: Tue, 16 Apr 2013 10:49 AM
Subject: Re: [OpenIndiana-discuss] Recommendations for fast storage 
(OpenIndiana-discuss Digest, Vol 33, Issue 20)

 From: Bob Friesenhahn [mailto:bfrie...@simple.dallas.tx.us]
 
 It would be difficult to believe that 10Gbit Ethernet offers better
 bandwidth than 56Gbit Infiniband (the current offering).  The swiching
 model is quite similar.  The main reason why IB offers better latency
 is a better HBA hardware interface and a specialized stack.  5X is 5X.

Put another way, the reason infiniband is so much higher throughput and lower 
latency than ethernet is because the switching (at the physical layer) is 
completely different from ethernet, and messages are passed directly from 
user-level to user-level on remote system ram via RDMA, bypassing the OSI layer 
model and other kernel overhead.  I read a paper from vmware, where they 
implemented RDMA over ethernet and doubled the speed of vmotion (but still not 
as fast as infiniband, by like 4x.)

Beside the bypassing of OSI layers and kernel latency, IB latency is lower 
because Ethernet switches use store-and-forward buffering managed by the 
backplane in the switch, in which a sender sends a packet to a buffer on the 
switch, which then pushes it through the backplane, and finally to another 
buffer on the destination.  IB uses cross-bar, or cut-through switching, in 
which the sending host channel adapter signals the destination address to the 
switch, then waits for the channel to be opened.  Once the channel is opened, 
it stays open, and the switch in between is nothing but signal amplification 
(as well as additional virtual lanes for congestion management, and other 
functions).  The sender writes directly to RAM on the destination via RDMA, no 
buffering in between.  Bypassing the OSI layer model.  Hence much lower latency.

IB also has native link aggregation into data-striped lanes, hence the 1x, 2x, 
4x, 16x designations, and the 40Gbit specifications.  Something which is 
quasi-possible in ethernet via LACP, but not as good and not the same.  IB 
guarantees packets delivered in the right order, with native congestion control 
as compared to ethernet which may drop packets and TCP must detect and 
retransmit...  

Ethernet includes a lot of support for IP addressing, and variable link speeds 
(some 10Gbit, 10/100, 1G etc) and all of this asynchronous.  For these reasons, 
IB is not a suitable replacement for IP 

Re: [OpenIndiana-discuss] Tape backup

[Doug Hughes]

On 3/17/2013 6:23 PM, Reginald Beardsley wrote:

Tape as an archival medium has significant issues.  Reading poorly stored tapes is a 
one try proposition w/ no assurance of success.  The first high volume 
commercial application for digital tape was seismic data acquisition for the oil 
industry.  The oil companies had very detailed  cleaning and retensioning schedules w/ a 
large staff to perform them on the tape archives. Absent that level of care, reading old 
tapes is very difficult and requires great skill.  Old tape is NOT fun to work with.

High capacity tape drives and tapes are not cheap either.  Blank LTO tape is 
almost as expensive as SATA disk. A ZFS based remote replicating server using 
triple parity RAIDZ is probably cheaper than tape.  For extremely large volumes 
and long archival periods, optical tape is probably the best choice.  But then 
you're probably working for the government.

I would strongly urge comparing the cost of a ZFS backup server w/ daily 
snapshots to the cost of conventional tape backup. I think you'll be quite 
surprised at the implications.



reading old disks is a just as significant if not more of an issue, by 
my estimation. Try to find a machine that you can do low voltage 
differential disks with these days. That was only 15 years ago. what 
about SMD? The controllers keep changing over time. Also, after a disk 
has been in use for a significant period of time (say years), the 
lubrication on the platters tends to evaporate a little bit so that when 
you leave it off for a long period of time (days/weeks) it will stick to 
the heads and the platters. Tape doesn't have that issue.


An LTO5 tape is about $30 each, better in quantity. 1.5TB, more 
depending upon compression. That's an enterprise quality tape with much 
longer shelf life than a cheap deskstar disk if cared for properly. Even 
the cheap 1TB disks are $70. That's almost a 3X advantage. There's still 
a place for tape for archival, and yes, you do have to care for it 
properly, just like you have to care for everything else. But, cared for 
properly, tape should still outlive disk.


But, used it in its proper place. zfs snapshots make a lot of sense for 
online backups!




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] multiple IP addresses, same NIC

[Doug Hughes]

On 3/5/2013 10:18 PM, Edward Ned Harvey (openindiana) wrote:

From: Doug Hughes [mailto:d...@will.to]

2) explicitly set the route for 192.168.10.x :
route add 192.168.10.0/mask 192.168.2.1


That's what I'm saying I have already done.  I set the default route to 
192.168.1.1, and I set a static route, 192.168.10.x/24 via 192.168.2.1.  The 
route is in effect, as evidenced:

For simplicity, let's say 192.168.1.1 has MAC 11:11:11:11:11:11 and let's say 
192.168.2.1 has mac 22:22:22:22:22:22.

When I ping something on the internet, I see a packet go out my NIC, source IP 
192.168.1.100, destination MAC 11:11:11:11:11:11 and destination IP 8.8.8.8.  
It all works, I get a ping response.

When I ping 192.168.2.1 directly, I see a packet go out my NIC, source IP 
192.168.2.100, destination MAC 22:22:22:22:22:22 and destination IP 
192.168.2.1.  It all works, I get a ping response.

When I ping something on the other end of the VPN, I see a packet go out of my 
NIC, source IP 192.168.1.100, destination MAC 22:22:22:22:22:22 and destination 
IP 192.168.10.10 (or whatever.)  The firewall drops the packet, because duh, 
the source IP isn't in the same subnet as the firewall.

I am also exploring the NAT option, assuming I'm not going to be able to 
resolve the above problem.


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss



is it the same if you use something other than ping?
Does it work correctly if you use ping -i to specify the source interface?

Another option may be tagged vlans. Supporting a separate tagged 
interface would be very clear for the host. It would appear as a totally 
separate layer2 so there'd be no chance for mac reuse. That's only if 
your router can support that too, however...



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] OI Crash

[Doug Hughes]

On 1/18/2013 7:53 PM, dormitionsk...@hotmail.com wrote:

On Jan 17, 2013, at 8:47 PM, Reginald Beardsley wrote:


As far as I'm concerned, problems like this are a bottomless abyss.  Which is 
why I'm still putting up w/ my OI box hanging.  It's annoying, but not 
critical.  It's also why critical stuff still runs on Solaris 10.

Intermittent failures are the worst time sink there is. There is no assurance 
that devoting all your time to the problem will fix it even at very high skill 
levels w/ a full complement of the very best tools.

If you're getting crash dumps there is hope of finding the cause, so that's a 
big improvement.

Good luck,
Reg

BTW Back in the 80's there was a VAX operator in Texas who went out to his 
truck, got a .357 and shot the computer.  His employer was not happy.  But I 
can certainly understand how the operator felt.



 From 1992 to I used to 1998, I used to work at the Denver Museum of Natural 
History -- now the Denver Museum of Nature and Science.  We had two or three 
DEC Vax's and an AIX machine there.  It was their policy that once a week we 
had to power each of the servers all the way down to clear out any memory 
problems -- or whatever -- as preventive maintenance.

Since then, I've always had the habit of setting up a cron job to reboot my 
servers once a week.  It's not as good as a full power down, but it's better 
than nothing.  And in all these years, I've never had to deal with intermittent 
problems like this, except for a few brief times when I used Red Hat Linux ten 
plus years ago.  (I've tried most of Red Hat's versions since 6.2, and RHEL 6 
is the first version I've found that runs decent enough on our hardware, and 
that I'm happy enough with, for us to use.)

So, if you can do it, you might want try setting up a cron job to reboot your 
server once a week -- or every night.  I reboot our LTSP thin client server 
every night just because it gets hit with running lots of desktop applications 
that I think give it a greater potential for these kinds of memory problems.

On the other hand, we have all of our websites hosted on one of our 
parishioner's servers -- and he doesn't reboot his machines periodically like I 
do -- and about every two months, I have to call him up and tell him something 
is wrong.  And he goes and powers down his system -- sometimes he has to even 
unplug it -- and then turn it back on, and everything works again.

I know there are system admins that just love to brag about how great their 
up-times are on their machines -- but this might just save you a lot of time 
and grief.

Of course, if you're running a real high-volume server, this might not be 
workable for you; but it only takes 2-5 minutes or so to reboot... Perhaps in 
the middle of the night you might be able to spare it being down that short 
time?

Just a friendly suggestion.

Shared experience.

I know others may tell you that that's no longer necessary anymore in these 
more modern times; but my experience has been otherwise.

I hope it helps.

+Peter, hieromonk



Haven't we passed the days of mystical sysadmin without understanding 
and characterization? Keeping up tradition for tradition's sake without 
understanding the underlying reasons really doesn't do anybody a favor. 
If there are memory leaks, we posses the technology to find them. My 
organization has thousands of machines that run jobs sometimes for 
months at a time. If I had to reboot servers once a week, my users would 
be at the doors with pitchforks. The only time we take downtime is when 
there are reasons to do so, including OS updates, hardware failures, and 
user software run amok. They can run a very long time like this.


Not that memory leaks never happen. Of course they do, but they 
eventually get found and fixed, or the program causing them passes into 
obsolescence. Always.


I encourage discovery rather than superstition, and diagnosis rather 
than repetition.


Be a knight, not a victim!


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Finding pools after reinstall

[Doug Hughes]

That isn't necessarily a driver. What model hba do you have? If it is among
many versions of megaraid or similar, you won't see any disks until you
create logical units from the raid card bios interface.
On Dec 19, 2012 9:53 AM, Thommy M. Malmström thommy.m.malmst...@gmail.com
wrote:

 Sorry I didn't read this thread till today - The first thing is, you *must*
  be able to see the disks via
  sudo format -e  /dev/null
  If they're not there, reboot, make sure they're enabled in BIOS, and
  sudo devfsadm -Cv
  After that, they should be visible, or else you're missing a driver.
 

 I'm definitely missing a driver. Can't see the disks with format.

 Will test Solaris 11.1 now to see if it finds all my disks.

 Thanks for response.

 /Thommy M.
 ___
 OpenIndiana-discuss mailing list
 OpenIndiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Finding pools after reinstall

[Doug Hughes]

Have you tried to import the pools by name? Often that will work where 
discovery isn't so great.


On 12/9/2012 1:35 PM, Thommy M. Malmström wrote:

Gentlemen, you have to be really careful with me here. It's some years
since I've touched Solaris. :)

I have a file server with several disks and OpenSolaris b125 (iirc). It has
hummed away nicely since 2009 or so and I've done nothing except for
changing a faulty disk drive. The other day it went down and refused to
boot. I got a new disk and disconnected all drives and replaced the boot
disk. Then I installed OpenIndiana 147 on the new disk and reconnected the
other disks. BUT, when I try to find the zpools on the old disk with zpool
import, it says nada. And format only lists one disk besides the boot disk.
So, ideas?



/Thommy M.




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS High-Availability and Sync Replication

[Doug Hughes]

On Tue, Nov 20, 2012 at 7:01 AM, Edward Ned Harvey (openindiana) 
openindi...@nedharvey.com wrote:

  From: Doug Hughes [mailto:d...@will.to]
 
  Well, to me, the most obvious is use another box with ZFS to mirror the
  ISCSI devices on ZFS. I'm in the process of doing this myself with some
  stuff. 2 head nodes and 2 storage nodes. The storage nodes are targets
  and the head nodes are initators. You get performance and reliability,
  and failover between head nodes is relatively easy with zpool import.

 This is precisely the thing I did with poor results, except that I didn't
 have 4 nodes - I had only 2.  So each node was both an initiator and a
 target.  I made the initiator service dependent on the target service (so
 the initiator would go down first and come up second).  I made zpool import
 of the iscsi pool dependent on the initiator being up, and I made vbox
 dependent on the iscsi zpool mounter.

 (see simplesmf, http://code.google.com/p/simplesmf/ for the zpool
 dependent on iscsi initiator)

 My word of advice, until such time (which may never come) that I might
 know more...  Never, ever let the iscsi targets go down unless the
 initiators are both down first.


strange. I tested failing the targets while the initiators are up and
failover worked fine. One half of the mirror went away and everything
worked as expected. I wonder if you might be running into some race or
other strange locking condition in the ISCSI kernel code being both target
and initiator for the same mirrored zpool? I tested it pretty thoroughly.
Aside from needing to run tcpkill to make Linux NFS clients really give up
on the current NFS/TCP connection and reestablish (or wait 5 minutes),
things worked as expected.
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS High-Availability and Sync Replication

[Doug Hughes]

On 11/19/2012 9:39 PM, Edward Ned Harvey (openindiana) wrote:

From: Sašo Kiselkov [mailto:skiselkov...@gmail.com]

I've been lately looking around the net for high-availability and sync
replication solutions for ZFS and came up pretty dry - seems like all
the jazz is going around on Linux with corosync/pacemaker and DRBD. I
found a couple of tools, such as AVS and OHAC, but these seem rather
unmaintained, so it got me wondering what others use for ZFS clustering,
HA and sync replication. Can somebody please point me in the right
direction?

Despite my recent failure, I still think there must be a good solution 
mirroring an iscsi device with another iscsi device.  I've put this project on 
the back burner, but I'm still optimistic that there's a simple explainable and 
repeatable problem causing the failure I experienced.  (And avoidable.)

Well, to me, the most obvious is use another box with ZFS to mirror the 
ISCSI devices on ZFS. I'm in the process of doing this myself with some 
stuff. 2 head nodes and 2 storage nodes. The storage nodes are targets 
and the head nodes are initators. You get performance and reliability, 
and failover between head nodes is relatively easy with zpool import.



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS remote receive

[Doug Hughes]

On 10/23/2012 7:52 AM, Sebastian Gabler wrote:

Hi,

I am facing a problem with zfs receive through ssh. As usually, root 
can't log on ssh; the log on users can't receive a zfs stream (rights 
problem), and pfexec is disabled on the target host (as I understand 
it is nowadays default for OI151_a...)


What are the suggestions to solve this? I tried several approaches 
with sudo, and su to no avail. I had tried to enable pfexec on the 
target system, too and couldn't do it.


you can run it over any tcp transport. Do it yourself options include 
using ttcp or netcat as a transport, but almost anything will do. It 
requires a little bit more synchronization to do it this way since the 
receiver must be running (piped into zfs recv) before the transmitter is 
started or the transmitter will abort.
In the end, though, you need to run the zfs recv as root somehow. If 
that's the crux of the problem and not the inability to ssh as root, 
you'll have to figure out a fix to get root at least for the zfs recv 
process.


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS; what the manuals don't say ...

[Doug Hughes]

On 10/23/2012 8:29 AM, Robin Axelsson wrote:

Hi,
I've been using zfs for a while but still there are some questions 
that have remained unanswered even after reading the documentation so 
I thought I would ask them here.


I have learned that zfs datasets can be expanded by adding vdevs. Say 
that you have created say a raidz3 pool named mypool with the command

# zpool create mypool raidz3 disk1 disk2 disk3 ... disk8

you can expand the capacity by adding vdevs to it through the command

# zpool add mypool raidz3 disk9 disk10 ... disk16

The vdev that is added doesn't need to have the same raid/mirror 
configuration or disk geometry, if I understand correctly. It will 
merely be dynamically concatenated with the old storage pool. The 
documentations says that it will be striped but it is not so clear 
what that means if data is already stored in the old vdevs of the pool.


Unanswered questions:

* What determines _where_ the data will be stored on a such a pool? 
Will it fill up the old vdev(s) before moving on to the new one or 
will the data be distributed evenly?
* If the old pool is almost full, an even distribution will be 
impossible, unless zpool rearranges/relocates data upon adding the 
vdev. Is that what will happen upon adding a vdev?
* Can the individual vdevs be read independently/separately? If say 
the newly added vdev faults, will the entire pool be unreadable or 
will I still be able to access the old data? What if I took a snapshot 
before adding the new vdev?


* Can several datasets be mounted to the same mount point, i.e. can 
multiple file system-datasets be mounted so that they (the root of 
them) are all accessed from exactly the same (POSIX) path and 
subdirectories with coinciding names will be merged? The purpose of 
this would be to seamlessly expand storage capacity this way just like 
when adding vdevs to a pool.
* If that's the case how will the data be distributed/allocated over 
the datasets if I copy a data file to that path?


Kind regards
Robin.


*) yes, you can dynamically add more disks and zfs will just start using 
them.

*) zfs stripes across all vdevs evenly, as it can.
*) as your old vdev gets full, zfs will only allocate blocks to the 
newer, less full vdev
*) since it's a stripe across vdevs (and they should all be raidz2 or 
better!) if one vdev fails, your filesystem will be unavailable. They 
are not independent unless you put them in a separate pool.
*) you cannot have overlapping /mixed filesystems at exactly the same 
place, however it is perfectly possible to have e.g. /export be on 
rootpool, /export/mystuff on zpool1 and /export/mystuff/morestuff be on 
zpool2.


The unasked question is If I wanted the vdevs to be equally balanced, 
could I?. The answers is a qualified yes. What you would need to do is 
reopen every single file, buffer it to memory, then write every block 
out again. We did this operation once. It means that all vdevs will 
roughly have the same block allocation when you are done.




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS; what the manuals don't say ...

[Doug Hughes]

On 10/23/2012 11:08 AM, Robin Axelsson wrote:

On 2012-10-23 15:41, Doug Hughes wrote:

On 10/23/2012 8:29 AM, Robin Axelsson wrote:

Hi,
I've been using zfs for a while but still there are some questions 
that have remained unanswered even after reading the documentation 
so I thought I would ask them here.


I have learned that zfs datasets can be expanded by adding vdevs. 
Say that you have created say a raidz3 pool named mypool with the 
command

# zpool create mypool raidz3 disk1 disk2 disk3 ... disk8

you can expand the capacity by adding vdevs to it through the command

# zpool add mypool raidz3 disk9 disk10 ... disk16

The vdev that is added doesn't need to have the same raid/mirror 
configuration or disk geometry, if I understand correctly. It will 
merely be dynamically concatenated with the old storage pool. The 
documentations says that it will be striped but it is not so clear 
what that means if data is already stored in the old vdevs of the pool.


Unanswered questions:

* What determines _where_ the data will be stored on a such a pool? 
Will it fill up the old vdev(s) before moving on to the new one or 
will the data be distributed evenly?
* If the old pool is almost full, an even distribution will be 
impossible, unless zpool rearranges/relocates data upon adding the 
vdev. Is that what will happen upon adding a vdev?
* Can the individual vdevs be read independently/separately? If say 
the newly added vdev faults, will the entire pool be unreadable or 
will I still be able to access the old data? What if I took a 
snapshot before adding the new vdev?


* Can several datasets be mounted to the same mount point, i.e. can 
multiple file system-datasets be mounted so that they (the root of 
them) are all accessed from exactly the same (POSIX) path and 
subdirectories with coinciding names will be merged? The purpose of 
this would be to seamlessly expand storage capacity this way just 
like when adding vdevs to a pool.
* If that's the case how will the data be distributed/allocated over 
the datasets if I copy a data file to that path?


Kind regards
Robin.


*) yes, you can dynamically add more disks and zfs will just start 
using them.

*) zfs stripes across all vdevs evenly, as it can.
*) as your old vdev gets full, zfs will only allocate blocks to the 
newer, less full vdev
*) since it's a stripe across vdevs (and they should all be raidz2 or 
better!) if one vdev fails, your filesystem will be unavailable. They 
are not independent unless you put them in a separate pool.
*) you cannot have overlapping /mixed filesystems at exactly the same 
place, however it is perfectly possible to have e.g. /export be on 
rootpool, /export/mystuff on zpool1 and /export/mystuff/morestuff be 
on zpool2.


The unasked question is If I wanted the vdevs to be equally 
balanced, could I?. The answers is a qualified yes. What you would 
need to do is reopen every single file, buffer it to memory, then 
write every block out again. We did this operation once. It means 
that all vdevs will roughly have the same block allocation when you 
are done.



Do you happen to know how that's done in OI? Otherwise I would have to 
move each file one by one to a disk location outside the dataset and 
then move it back or zfs send the dataset to another pool of at least 
equal size and then zfs receive it back to the expanded pool.


you don't have to move it, you just have to open, read it into memory, 
seek back to the beginning, and write it out again. Rewriting those 
blocks will take care of it since ZFS is copy-on-write. You will need to 
be wary of your snapshots during this process since all files will be 
rewritten and you'll double your space consumption.


(basically a perl, python, or other similar script could do this)


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS remote receive

[Doug Hughes]

On 10/23/2012 4:13 PM, Timothy Coalson wrote:


Works pretty well, though I get ~70MB/s on gigabit ethernet instead of the
theoretically possible 120MB/s, and I'm not sure why (NFS gets pretty close
to 120MB/s on the same network).



There's a fair bit of overhead to ssh and to zfs send/recive, so you're 
doing very well.



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Zfs stability

[Doug Hughes]

yes, you shoud do a scrub  and no, there isn't very much risk to this. This
will scan your disks for bits that have gone stale or the like. You should
do it. We do a scrub once per week.



On Fri, Oct 12, 2012 at 3:55 PM, Roel_D openindi...@out-side.nl wrote:

 Being on the list and reading all ZFS problem and question posts makes me
 a little scared.

 I have 4 Sun X4140 servers running in the field for 4 years now and they
 all have ZFS mirrors (2x HD). They are running Solaris 10 and 1 is running
 solaris 11. I also have some other servers running OI, also with ZFS.

 The Solaris servers N E V E R had any ZFS scrub. I didn't even knew such
 existed ;-)

 Since it all worked flawless for years now i am a huge Solaris/OI fan.

 But how stable are things nowaday? Does one need to do a scrub? Or a
 resilver?

 How come i see so much ZFS trouble?



 Kind regards,

 The out-side
 ___
 OpenIndiana-discuss mailing list
 OpenIndiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Zfs stability Scrubs

[Doug Hughes]

So?}?\, a lot of people have already answered this in various ways. 
I'm going to provide a little bit of direct answer and focus to some of 
those other answers (and emphasis)


On 10/12/2012 5:07 PM, Michael Stapleton wrote:

It is easy to understand that zfs srubs can be useful, But, How often do
we scrub or the equivalent of any other file system? UFS? VXFS?
NTFS? ...
ZFS has scrubs as a feature, but is it a need? I do not think so. Other
file systems accept the risk, mostly because they can not really do
anything if there were errors.
That's right. They cannot do anything. Why is that a good thing? If you 
have a corruption on your filesystem because a block or even a single 
bit went wrong, wouldn't you want to know? Wouldn't you want to fix it? 
What if a number in an important financial document changed? Seems 
unlikely, but we've discovered at least 5 instances of spontaneous disk 
data corruption over the course of a couple of years. zfs corrected them 
transparently. No data lost, automatic, clean,  and transparent. The 
more data that we make, the more that possibility of spontaneous data 
corruption becomes reality.

It does no harm to do periodic scrubs, but I would not recommend doing
them often or even at all if scrubs get in the way of production.
What is the real risk of not doing scrubs?
data changing without you knowing it. Maybe this doesn't matter on an 
image file (though a jpeg could end up looking nasty or destroyed, and 
mpeg4 could be permanently damaged, but in a TIFF or other uncompressed 
format, you'd probably never know)




Risk can not be eliminated, and we have to accept some risk.

For example, data deduplication uses digests on data to detect
duplication. Most dedup systems assume that if the digest is the same
for two pieces of data, then the data must be the same.
This assumption is not actually true. Two differing pieces of data can
have the same digest, but the chance of this happening is so low that
the risk is accepted.
but, the risk of data being flipped once you have TBs of data is way 
above 0%. You can also do your own erasure coding if you like. That 
would be one way to achieve the same affect outside of ZFS.



I'm only writing this because I get the feeling some people think scrubs
are a need. Maybe people associate doing scrubs with something like
doing NTFS defrags?


NTFS defrag would only help with performance. scrub helps with 
integrity. Totally different things.



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] rebuilding man page catalog

[Doug Hughes]

catman -w rebuilds the index (-M to supply directory)

On Tue, Oct 9, 2012 at 11:18 AM, Boris Epstein borepst...@gmail.com wrote:

 Hello listmates,

 If may man command does not display some of the pages including those
 clearly present under /usr/share/man - how do I fix that? I remember there
 was a command that rebuilt the catalog, I just can't recall what it was.

 Thanks.

 Boris.
 ___
 OpenIndiana-discuss mailing list
 OpenIndiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] How to test ethernet throughput?

[Doug Hughes]

A third recommendation for iperf. It's the tool you want. Don't mess 
around with anything else.


On 5/18/2012 10:02 AM, Rich wrote:

iperf is also your friend, as is dd+netcat+pv if you want something a
bit less rigorous.

- Rich




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] nfs mounting and cascading dataset access?

[Doug Hughes]

On 2/27/2012 11:32 AM, James Carlson wrote:

Scott LeFevre wrote:

I have a pools setup named tank1 with multiple datasets/filesystems
defined.  At the top, I have tank1/media followed by tank1/media/Video
and tank1/media/Music.  I've setup tank1/media as a nfs share
(e.g. /export/media) and can mount it from other systems.  When I try to
access media/Video from the nfs client, it shows an empty directory.
(From the oi 151a2 host, everything shows up locally.)   I can mount
tank1/media/Video directly from the nfs client and everything shows up
as it should.

Is there a way to make this work where I expose just tank1/media as an
nfs export and can access the underlying datasets/filesystems (e.g.
Video and Music)?

This isn't really a special issue with ZFS.

NFS has always done exports by file system.  If you have /foo and
/foo/bar, and /foo/bar is a mount point for some other file system, then
exporting /foo does not give clients access to the contents of /foo/bar,
because there's no way to cross the file system boundary.  Instead,
clients looking at /foo/bar will see there whatever is actually in the
/foo file system -- probably just an underlying empty directory used as
the mount point.

What you can do is set sharenfs=on (or some list of NFS options
instead of just on) using the zfs command on tank/media, and all of
the filesystems under that point will inherit this setting.  That means
that clients will see separate NFS exports for each of the underlying
datasets -- one for Video and one for Music in this case.

Automount, of course, is your friend.  ;-}

You can do this (hierarchical implicit shared subfs) with NFSv4, but not 
with NFSv3. There's a lot more setup for V4 though and it would likely 
not be as robust.


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ZFS send recv soln

[Doug Hughes]

Yes, that's a very fine way to do it and we do it here. I'm afraid our
python scripts wouldn't be much use to you though. They are very site
specific. My main point for posting is a word of encouragement.

On Mon, Jan 9, 2012 at 5:47 PM, Marcus Dillury mar...@dillury.net wrote:

 Hi,

 I have a customer who would like to use a zfs backup solution for his
 production zpool. I was thinking of using incremental snapshot
 send/recv to do this.

 I was wondering if anyone has any useful scripts that can facilitate
 the recursive send/recv of approx 2000 zfs datasets. My client has
 limited technical experience, so I was ideally after something that
 can automatically remove old snapshots when space becomes constrained.

 Thanks in advance.

 -Marcus

 ___
 OpenIndiana-discuss mailing list
 OpenIndiana-discuss@openindiana.org
 http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Fileserver performance with log and cache devices?

[Doug Hughes]

On 12/1/2011 3:15 PM, David Brodbeck wrote:

On Thu, Dec 1, 2011 at 8:09 AM, Geoff Flaritygeoff.flar...@gmail.comwrote:


My advice would be to max out your RAM (for ARC) before you bother
with a ZIL, or L2ARC.  Where a fast SSD for a ZIL really shines is
random synchronized writes. IE a database transaction. You'll notice
this when you run filebench and look at the results. I'm not sure how
NFS handles FSYNC and DSYNC to be honest.


NFS does a *lot* of synchronous writes and will probably benefit from a
fast ZIL device.

It's particularly valuable data for heavy meta-data intensive things 
with NFS, which includes deleting directories with rm -r and creating or 
updating lots of small files. NFS commits beat on the ZIL.



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Supermicro SC216 chassis with multiple HBAs?

[Doug Hughes]

On 10/26/2011 7:40 PM, Jason J. W. Williams wrote:

Has anyone here tried using two LSI HBAs (9211-8i) in a single SC216
chassis? I'm being told there's only one iPass cable off the backplane
and so only one HBA can be used...but with the number of folks adding
in SATA SSDs it seems like there must be a solution so you don't have
to mix SATA and SAS on the same HBA.

Any insight is greatly appreciated.



My integrator is configuring with 3 of the 9211-8i..Either they know 
something, or they don't know something.. But I  have 3 integrators 
essentially offering the samething, so I assume they all must know 
something.



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] ssh sessions timing out?

[Doug Hughes]

On 10/12/2011 10:56 PM, Dan Swartzendruber wrote:

This one is driving me nuts.  I can't seem to keep an ssh session open for
more than 10 minutes or so.  When I check after that, putty has an error up
about network error caused software abort (or words to that effect.)  I
have TCP/IP keepalives turned on, as well as ssh keepalives, and that hasn't
helped.  This is the only server that has this problem :(  I've double and
tripled checked the putty settings and don't see anything helpful.  Any
ideas?  Thanks!


you could try running sshd -d, or put a 'truss' onto the sshd after 
running to see what it's doing.


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Help with website

[Doug Hughes]

On 10/11/2011 6:10 AM, Dmitry Kozhinov wrote:

  doesn't pull let them manage their own time better?

Yes. This is why my vote is for forum, not mailing list.

Dmitry.



I've stayed out of this thread thus far, and probably will going 
forward, but I vastly prefer mail lists and I think the above is a 
non-argument. Forums do not allow better time management by any 
objective means. In a mail list, I am equally capable of sending all of 
the mail list to a folder, or not, threaded in my mail reader, or not, 
as I choose, and I can read them whenever I want. If I see a topic pop 
up that interests me, I can read it then, or not. I can make agile 
changes in my interests easily.


Forums have one major failing for me: Out of sight is out of mind.

If I have to go to a website, login (maybe it has cookies, maybe it 
doesn't; maybe it does and I don't want the website to remember my 
password!) then find the thread or topic where I left off, which will 
take a few clicks or more depending upon how long it has been. Mail 
messages are all right there, and I only need to go to the most recent 
undeleted or unread one with less effort.


But, there is also a tradeoff in volume. If a mail list starts receiving 
100s of messages a day it becomes less useful and it might as well be a 
forum. Linux kernel development lists come to mind. Those would be more 
useful as a forum. There are very few topics in there that actually 
might interest me and the flood of mail makes it a chore. OI is a 
smallish mail list (aside from this thread!) so managing the (normally) 
20 messages that come in in a day is easier in my inbox.


thresholds matter.

$.02


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] OI vs U9 performance surprises

[Doug Hughes]

Box = Sun x4240 with 8xIntel 320 160GB flash


R6x6 SSD oi R6x6 SSD u9 R5x5 SSD u9 R5x6u9  R6 ssd oi/12
100k4m 37s  3m 31s  3m 24s  3m 27s  4m 18s
rm 100k 1m 39s  1m 3s   1m  1m  1m 25s
star-x  3m 5s   3m 37s  2m 37s  2m 43s  2m 53s
cp b.tar2m 14s  2m 18s  2m 8s   2m 9s   2m 1s
dual touch  3m 41s  3m 35s  3m 43s  4m 24s
dual rm 1m 4s   1m 2s   1m 4s   1m 25s
dual star -x2.5/5m  3.3m 4.5m   3m/4m   3.5m 4.5m
d cp 12.7G  3m 45s  3m 45s  4m  3m 45s


SSD = Intel 320 160GB drives
R6 = 6 drives in raid6 config
u9  = Solaris 10 update 9
R5x5 = Intel 320 160GB in raid5 with 5 disks
oi= OpenIndiana 148 release
oi/12 = OpenIndiana 148 release with zpool-12 patch for ashift=12

100k = 100k files: 1000 directories, 100 files each, using touch (0 byte 
files - metadata/ops intensive)

rm 100k = removing all files and directories
star -x = extracting (over NFS, 1gig) a 12.7GB file with a mix of small 
and large file

cp b.tar = copying 12.7GB tar file over NFS to server (bounded  by wire)
dual touch = touch 100k files over NFS on dedicated 1gig links to server
dual rm = removing those files
dual cp = copying the 12.7GB tar file from the node (bounded by 1gbit 
wire again)


The odd thing is that, aside from the things that are bounded by 1gbit 
wire, that u9 is quite a bit faster than OpenIndiana. I have 128 NFS 
servers defined, and they never seem to be bottlenecked.


I would post the numbers for 73GB spinning media, but the run times for 
OpenIndiana were unboundedly slow, and I hadn't been able to figure that 
out (hours later, I had to ctrl-c vs. minutes on U9).


when I ran dstrace ops script against the 73GB disks it was taking 
hundreds of mseconds for ops to complete.  But, since I haven't posted 
the performance numbers of the 73GB disks here (will re-do benchmarks in 
a bit and repost if asked), I guess we could focus on why OI appears to 
be significantly slower. ashift=12 improves things by a small amount, 
but still not to the point of u9 speed. Advice/tips welcome. This is a 
relatively new/virgin OI-148 install.


(one note, the double numbers often include 2 numbers. It appears that 
the using dladm on the second interface to create an aggregate has a 
signficant impact on transfer speed/latency, but we can ignore that for now)





___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] OI vs U9 performance surprises

[Doug Hughes]

On 5/9/2011 12:01 PM, Doug Hughes wrote:


Box = Sun x4240 with 8xIntel 320 160GB flash


R6x6 SSD oi R6x6 SSD u9 R5x5 SSD u9 R5x6u9  R6 ssd oi/12
100k4m 37s  3m 31s  3m 24s  3m 27s  4m 18s
rm 100k 1m 39s  1m 3s   1m  1m  1m 25s
star-x  3m 5s   3m 37s  2m 37s  2m 43s  2m 53s
cp b.tar2m 14s  2m 18s  2m 8s   2m 9s   2m 1s
dual touch  3m 41s  3m 35s  3m 43s  4m 24s
dual rm 1m 4s   1m 2s   1m 4s   1m 25s
dual star -x2.5/5m  3.3m 4.5m   3m/4m   3.5m 4.5m
d cp 12.7G  3m 45s  3m 45s  4m  3m 45s


SSD = Intel 320 160GB drives
R6 = 6 drives in raid6 config
u9  = Solaris 10 update 9
R5x5 = Intel 320 160GB in raid5 with 5 disks
oi= OpenIndiana 148 release
oi/12 = OpenIndiana 148 release with zpool-12 patch for ashift=12

100k = 100k files: 1000 directories, 100 files each, using touch (0 
byte files - metadata/ops intensive)

rm 100k = removing all files and directories
star -x = extracting (over NFS, 1gig) a 12.7GB file with a mix of 
small and large file

cp b.tar = copying 12.7GB tar file over NFS to server (bounded  by wire)
dual touch = touch 100k files over NFS on dedicated 1gig links to server
dual rm = removing those files
dual cp = copying the 12.7GB tar file from the node (bounded by 1gbit 
wire again)


The odd thing is that, aside from the things that are bounded by 1gbit 
wire, that u9 is quite a bit faster than OpenIndiana. I have 128 NFS 
servers defined, and they never seem to be bottlenecked.


I would post the numbers for 73GB spinning media, but the run times 
for OpenIndiana were unboundedly slow, and I hadn't been able to 
figure that out (hours later, I had to ctrl-c vs. minutes on U9).


when I ran dstrace ops script against the 73GB disks it was taking 
hundreds of mseconds for ops to complete.  But, since I haven't posted 
the performance numbers of the 73GB disks here (will re-do benchmarks 
in a bit and repost if asked), I guess we could focus on why OI 
appears to be significantly slower. ashift=12 improves things by a 
small amount, but still not to the point of u9 speed. Advice/tips 
welcome. This is a relatively new/virgin OI-148 install.


(one note, the double numbers often include 2 numbers. It appears that 
the using dladm on the second interface to create an aggregate has a 
signficant impact on transfer speed/latency, but we can ignore that 
for now)






Here are some dtrace ops numbers for the server side (linux client, same 
client for both) with open indiana:

hughesd@solaris:/var/tmp/mravi# bin/nfs-ops-response.d
dtrace: script 'bin/nfs-ops-response.d' matched 3 probes
^C
CPU FUNCTION
  1 | :END

NFS v3 Ops count
==
RFS3_FSSTAT   2
RFS3_MKDIR 1000
RFS3_GETATTR   1001
RFS3_ACCESS1006
RFS3_CREATE  101000
RFS3_SETATTR 101000
RFS3_LOOKUP  101003


NFS v3 Ops average RESPONSE time (usec)

RFS3_GETATTR 39
RFS3_LOOKUP  46
RFS3_ACCESS  52
RFS3_FSSTAT 304
RFS3_SETATTR449
RFS3_CREATE 508
RFS3_MKDIR  667


NFS v3 Ops average SYSTEM time (usec)
==
RFS3_GETATTR 32
RFS3_ACCESS  40
RFS3_LOOKUP  40
RFS3_FSSTAT  80
RFS3_SETATTR143
RFS3_CREATE 200
RFS3_MKDIR  340


---
here are numbers with Sol 10 U9 (same server hardware, same client)
6 disk raidz1:

dtrace: script '/var/tmp/mravi/bin/nfs-ops-response.d' matched 3 probes
^C
CPU FUNCTION
  5 | :END

NFS v3 Ops count
==
RFS3_GETATTR   1000
RFS3_MKDIR 1000
RFS3_ACCESS1004
RFS3_CREATE  101000
RFS3_LOOKUP  101000
RFS3_SETATTR 101000


NFS v3 Ops average RESPONSE time (usec)

RFS3_LOOKUP  17
RFS3_GETATTR 18
RFS3_ACCESS  21
RFS3_SETATTR270
RFS3_CREATE 294
RFS3_MKDIR  340


NFS v3 Ops average SYSTEM time (usec)
==
RFS3_GETATTR 16
RFS3_LOOKUP  16
RFS3_ACCESS  18
RFS3_SETATTR 54
RFS3_CREATE  75
RFS3_MKDIR  124

---
Here's the dtrace script
#!/usr/sbin/dtrace -FCs
/*
# Author: Ravi Mallarapu
# Date: 20101117
# Description: display NFS v3 ops count and average user and system 
response time

#
*/


fbt:nfssrv:common_dispatch:entry
/args[0]-rq_vers == 3  ! self-trace

[OpenIndiana-discuss] dladm create-aggr fails with link busy

[Doug Hughes]

fresh openinidna 148 install. It came up with ipv6 for nge1/nge2 after 
auto-discovering and I did the unplumb on those, so the interfaces are 
clean and unused, but still I can't create the aggregate.


hughesd@x4240-3-1-17:~# dladm show-aggr
hughesd@x4240-3-1-17:~# dladm create-aggr -d nge2 1
dladm: create operation failed: link busy
hughesd@x4240-3-1-17:~# dladm create-aggr -d nge1 1
dladm: create operation failed: link busy
hughesd@x4240-3-1-17:~# dladm show-aggr
hughesd@x4240-3-1-17:~# ifconfig -a
lo0: flags=2001000849UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL mtu 
8232 index

inet 127.0.0.1 netmask ff00
nge0: flags=1004843UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4 mtu 1500 
index 8

inet 10.232.4.60 netmask fc00 broadcast 10.232.7.255
ether 0:14:4f:ed:b8:ec
hughesd@x4240-3-1-17:~# uname -a
SunOS x4240-3-1-17 5.11 oi_148 i86pc i386 i86pc
hughesd@x4240-3-1-17:~# dladm create-aggr -d nge1 -d nge2 1
dladm: create operation failed: link busy
hughesd@x4240-3-1-17:~#

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] dladm create-aggr fails with link busy

[Doug Hughes]

that did it! thanks! (never would have found that)

On 5/6/2011 12:25 PM, Lucas Van Tol wrote:

I saw something like this when I hadn't disabled NWAM before trying to create 
an aggregate...





Date: Fri, 6 May 2011 11:43:39 -0400
From: d...@will.to
To: openindiana-discuss@openindiana.org
Subject: [OpenIndiana-discuss] dladm create-aggr fails with link busy


fresh openinidna 148 install. It came up with ipv6 for nge1/nge2 after
auto-discovering and I did the unplumb on those, so the interfaces are
clean and unused, but still I can't create the aggregate.

hughesd@x4240-3-1-17:~# dladm show-aggr
hughesd@x4240-3-1-17:~# dladm create-aggr -d nge2 1
dladm: create operation failed: link busy
hughesd@x4240-3-1-17:~# dladm create-aggr -d nge1 1
dladm: create operation failed: link busy
hughesd@x4240-3-1-17:~# dladm show-aggr
hughesd@x4240-3-1-17:~# ifconfig -a
lo0: flags=2001000849UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL  mtu
8232 index
  inet 127.0.0.1 netmask ff00
nge0: flags=1004843UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4  mtu 1500
index 8
  inet 10.232.4.60 netmask fc00 broadcast 10.232.7.255
  ether 0:14:4f:ed:b8:ec
hughesd@x4240-3-1-17:~# uname -a
SunOS x4240-3-1-17 5.11 oi_148 i86pc i386 i86pc
hughesd@x4240-3-1-17:~# dladm create-aggr -d nge1 -d nge2 1
dladm: create operation failed: link busy
hughesd@x4240-3-1-17:~#

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss