Re: [OpenIndiana-discuss] Current ZFS Backup projects (OpenIndiana-discuss Digest, Vol 26, Issue 34) (OpenIndiana-discuss Digest, Vol 26, Issue 39)

[Ong Yu-Phing]

Yes, there are two slightly different goals in terms of backup, (1) to 
backup ZFS based CIFS servers (2) to backup non-ZFS based servers.  For 
(1) I've been trying to use tools like zetaback, warmer, etc.  For (2) 
I've been using good-ol' backuppc.


I'm quite glad this thread started, because lo and behold we now have 
zrep pointed out by Frank (written by Philip Brown, according to the 
website), which looks exactly like what I'm interested in for (1), and 
mdbackup from Julius, which looks like I could use for (2), although I 
might still stay with backuppc as my staff are quite familiar with this.


So thanks all for your valuable inputs!

Phing

On 13/09/2012 17:23, Jim Klimov wrote:

2012-09-12 6:05, Ong Yu-Phing ?:

Jim, I assume you are referring to this:
http://wiki.openindiana.org/oi/rsync+daemon+service+on+OpenIndiana, 
thanks!


Yes, I think that's it ;)


My concern is that typically rsync will take quite a while to traverse a
large set of files before sending only changed files; a classic example
is backing up say 1TB of maildir emails, it may take 4+ hours, and you
now have to deal with a situation where your midnight backup is really a
"somewhere between midnight and 4am" backup.  And of course, if you want
to take snapshots/backups of comstar volumes, rsync isn't quite the
right fit.

On the other hand, a zfs snapshot gives an almost-at-the-time backup
(give or take a few seconds), versus the aforementioned rsync. The zfs
snapshot can then be sent off-site, independent of the backup activity.


I got an impression that you needed to backup some "client" machines
with varied OSes, such as Windows or Linux desktops, onto a ZFS server.
In that case rsync should help, although you're right that it would
take long to scan the directory trees for changes.

With client FSes that support snapshots (ZFS, NTFS shadow copy) you
might have some luck making scripts that take the client's snapshot,
hold it (in case of ZFS, to avoid it being destroyed while you're
working), rsync the changes from the snapshot (so the 0am backup is
really the state at 0am) and release/destroy the snapshot on client.
In case of ZFS at least, you might have some optimization by using
"zfs diff" to determine changed files between two snapshots on the
client - but then you should not destroy rsynced snapshots right
away, but keep a backlog of one or two at least. And you should have
some locking to prevent several instances of the backup job crawling
the same client space and bringing IOPS to a halt.

Now, before you ask "why not zfs-send client snapshots directly?" -
there may be reasons, such as incompatible ZFS versions on client
and server, differing dataset layouts, flaky network preventing
transfer of large zfs-send streams (though that should have been
addressed with resumable zfs-send feature, if that was integrated).

HTH,
//Jim Klimov







___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Problems with netcat zfs receive

[Flo]

Hi Carsten,

thank you for your suggestion.

I had not time yet to test my script with netcat, but I will give 
mbuffer a try!


I heard from that tool some time ago, but I did not think on it.

CU
Florian

On 09/14/2012 08:29 AM, real-men-dont-cl...@gmx.net wrote:

-Original message-
To: openindiana-discuss@openindiana.org;
From:   Flo 
Sent:   Thu 13-09-2012 10:56
Subject:Re: [OpenIndiana-discuss] Problems with netcat zfs receive

Hello,

thank you very much! With the "d" flag, the nc commands works very well!

Regards
Florian




Hi Florian,

I played with zfs send/reveive some time ago. One important thing I learned was 
that netcat is not the first choice to use.
There is a tool called mbuffer out there. mbuffer works similar to netcat but 
allows a specific buffer size and block size.

From various resources I found out that the best buffer and block sizes for zfs 
send/receive seem to be 1GB for the buffer with a block size of 131073.

Replacing netcat by mubuffer dramatically increases the throughput.


The resulting commands are like:

ssh -f $REMOTESRV "/opt/csw/bin/mbuffer -q -I $PORT -m 1G -s 131072 | zfs receive 
-vFd $REMOTEPOOL"

zfs send $CURRENTLOCAL | /opt/csw/bin/mbuffer -q -O $REMOTESRV:$PORT -m 1G -s 
131072 > /dev/null


cu


Carsten

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] Problems with netcat zfs receive

[real-men-dont-click]

-Original message-
To: openindiana-discuss@openindiana.org; 
From:   Flo 
Sent:   Thu 13-09-2012 10:56
Subject:Re: [OpenIndiana-discuss] Problems with netcat zfs receive
> Hello,
> 
> thank you very much! With the "d" flag, the nc commands works very well!
> 
> Regards
> Florian
> 


Hi Florian,

I played with zfs send/reveive some time ago. One important thing I learned was 
that netcat is not the first choice to use.
There is a tool called mbuffer out there. mbuffer works similar to netcat but 
allows a specific buffer size and block size.
>From various resources I found out that the best buffer and block sizes for 
>zfs send/receive seem to be 1GB for the buffer with a block size of 131073.
Replacing netcat by mubuffer dramatically increases the throughput.


The resulting commands are like:

ssh -f $REMOTESRV "/opt/csw/bin/mbuffer -q -I $PORT -m 1G -s 131072 | zfs 
receive -vFd $REMOTEPOOL"

zfs send $CURRENTLOCAL | /opt/csw/bin/mbuffer -q -O $REMOTESRV:$PORT -m 1G -s 
131072 > /dev/null


cu


Carsten

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] OT: Sun Open Web Server...

[Dave Koelmeyer]

Not related to OpenIndiana, but I thought it interesting stumbling across
this today: the open source guts of the "Sun Java System" web server
product which Oracle apparently swept under the carpet (the open source
bits that is, not the proprietary, rebranded offering).

http://173.255.252.27/jyri/articles/index.php/web-server/

-- 
Dave Koelmeyerhttp://blog.davekoelmeyer.co.nz
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Problem with pkg image-update

[Yuri Pankov]

On Thu, 13 Sep 2012 23:21:30 -0400, Mark Creamer wrote:

My server is currently at oi151a5, and it's been about a month, so I ran
pkg image-update to check on available updates. As expected, the packages
from oi151a6 were available, so I let it run. When the installation was
complete, I noticed it said that pkg could not activate the new BE. I ran
beadm list, and saw all of the BEs listed, with the one I'm in now marked
N, and the new one marked R. The new one shows a mount point under
/tmp/.

Because I saw that it said it could not activate the new BE, I decided not
to reboot because I wasn't sure what would happen to my old BE. I then
attempted beadm activate opensolaris-6 which is my current running BE, the
one marked N in beadm list. It said it could not promote the BE.

I'm leaving it be until I can get some advice on what to do. For example,
should I destroy the latest BE that was just created but could not be
activated?


Not really related to your question, but try setting BE_PRINT_ERR 
environment variable to "true" to check why beadm is unable to activate 
new BE.


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] Problem with pkg image-update

[Mark Creamer]

My server is currently at oi151a5, and it's been about a month, so I ran
pkg image-update to check on available updates. As expected, the packages
from oi151a6 were available, so I let it run. When the installation was
complete, I noticed it said that pkg could not activate the new BE. I ran
beadm list, and saw all of the BEs listed, with the one I'm in now marked
N, and the new one marked R. The new one shows a mount point under
/tmp/.

Because I saw that it said it could not activate the new BE, I decided not
to reboot because I wasn't sure what would happen to my old BE. I then
attempted beadm activate opensolaris-6 which is my current running BE, the
one marked N in beadm list. It said it could not promote the BE.

I'm leaving it be until I can get some advice on what to do. For example,
should I destroy the latest BE that was just created but could not be
activated?

Thanks

-- 
Mark
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Announcing mdbackup, perl scripts for zfs backup servers (was: Current ZFS Backup projects)

[Dave Koelmeyer]

On Fri, Sep 14, 2012 at 12:53 AM, Julius Roberts wrote:

> Hi all, I've created https://sourceforge.net/projects/mdbackup/,
> please have a look if you're interested.  I'm very keen on getting
> feedback/suggestions/help so feel free to email me off list etc.  I've
> never done anything like this before, so please be gentle :)
>

Looks great - will be sure to give this a spin :)

Cheers,

-- 
Dave Koelmeyerhttp://blog.davekoelmeyer.co.nz
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] CIFS sharing in a Windows Domain

[Robbie Crash]

I had a similar problem last year that had the same solution, I wrote a
post which goes into a bit more detail about it:
https://robbiecrash.me/?p=89


On Thu, Sep 13, 2012 at 8:45 PM, Gordon Ross wrote:

> On Wed, Sep 12, 2012 at 5:29 AM, Gabriele Bulfon 
> wrote:
> > Hi,
> > when sharing a zfs filesystem inside a domain, I usually chmod 777 the
> base filesystem,
> > then the Windows Administsrator can start creating folders and giving
> permissions.
> > Maybe there is somtehing wrong with the way I do it, but the strange
> thing is that the base share
> > results available to Everyone, they can create folder, files, and these
> get only the permissions of this
> > user and just System. So, not even the Administrator can delete this
> files.
> > If I remember well, there is no way to change the permission of the base
> share (the zfs filesystem)
> > by the Administrator, but only subfolders inside it.
> > When sharing a zfs filesystem via CIFS, I'd like this base share to have
> just permission for the
> > Administrator, so that he can only create folders / files there, and
> then give permissions to others.
> > What's wrong?
> > Gabriele.
>
> Probably what's wrong is that "chmod 777" does not enable inheritance.
> Instead, I'd recommend:
>
>  chmod A=everyone@:rwxpdDaARWcCos:fd:allow /pool/fs-name
>
> which enables the file+directory inheritane bits.
> After you do that, connect with Windows and set the
> security the way you really want it to be.
>
> Also remember to set zfs aclmode and aclinherit to passthrough
> if you want ACL changes from unix to make sense to cifs clients.
>
> --
> Gordon Ross 
> Nexenta Systems, Inc.  www.nexenta.com
> Enterprise class storage for everyone
>
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
>



-- 
Seconds to the drop, but it seems like hours.

http://www.openmedia.ca
https://robbiecrash.me
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Current ZFS Backup projects

[Christopher X. Candreva]

On Wed, 12 Sep 2012, Julius Roberts wrote:

> mdbackup and I manage it all using subversion. As for releasing it into the
> wild I'm happy do do that, but there are no appropriate licensing
> statements in any if the files nor have I considered what licence to even
> use. I was thinking of pushing the code to a slave svn instance from which
> the public can sync if they like.  Pretty busy right now though :) oh it's

Github is great for that sort of thing.


==
Chris Candreva  -- ch...@westnet.com -- (914) 948-3162
WestNet Internet Services of Westchester
http://www.westnet.com/

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] CIFS sharing in a Windows Domain

[Gordon Ross]

On Wed, Sep 12, 2012 at 5:29 AM, Gabriele Bulfon  wrote:
> Hi,
> when sharing a zfs filesystem inside a domain, I usually chmod 777 the base 
> filesystem,
> then the Windows Administsrator can start creating folders and giving 
> permissions.
> Maybe there is somtehing wrong with the way I do it, but the strange thing is 
> that the base share
> results available to Everyone, they can create folder, files, and these get 
> only the permissions of this
> user and just System. So, not even the Administrator can delete this files.
> If I remember well, there is no way to change the permission of the base 
> share (the zfs filesystem)
> by the Administrator, but only subfolders inside it.
> When sharing a zfs filesystem via CIFS, I'd like this base share to have just 
> permission for the
> Administrator, so that he can only create folders / files there, and then 
> give permissions to others.
> What's wrong?
> Gabriele.

Probably what's wrong is that "chmod 777" does not enable inheritance.
Instead, I'd recommend:

 chmod A=everyone@:rwxpdDaARWcCos:fd:allow /pool/fs-name

which enables the file+directory inheritane bits.
After you do that, connect with Windows and set the
security the way you really want it to be.

Also remember to set zfs aclmode and aclinherit to passthrough
if you want ACL changes from unix to make sense to cifs clients.

-- 
Gordon Ross 
Nexenta Systems, Inc.  www.nexenta.com
Enterprise class storage for everyone

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] OI stopped booting after 151a6 update.

[Milan Jurik]

Hi,

interesting you see it on T61p which is not far from my T60. What I need
it is described here:

https://www.illumos.org/issues/3150#note-5

https://www.illumos.org/issues/3150#note-9

I need to collect data to see if all cases are same.

Best regards,

Milan

Gary Gendel píše v čt 13. 09. 2012 v 10:49 -0400:
> Milan,
> 
> Please contact me directly if you need me to run some experiments or 
> collect some data.  I tried Richard Palo's workaround but it only worked 
> in the current session; a reboot brought it back and wouldn't go away 
> with a re-applied fix.
> 
> I blew a6 away, but I can put it back if you tell me what you want me to 
> try.  BTW, this is on a Lenovo Thinkpad T61p.
> 
> Gary
> 
> On 9/13/12 10:40 AM, Milan Jurik wrote:
> > Hi,
> >
> > it would be nice if people affected by this are active on that issue. 
> > Currently I have only one reporting person who wrote that the problem 
> > seems to be visible only in case that hald is daemonized. Ongoing 
> > investigation. I cannot reproduce it on my system (yet).
> >
> > Best regards,
> >
> > Milan
> >
> > On 13.09.2012 13:55, Dmitry Kozhinov wrote:
> >> Yes, it's the same. Production server affected... I am stick to 151a5
> >> for a while.
> >>
> >> On 07.09.2012 18:23, Richard PALO wrote:
> >>> It's possibly the same as a couple of us have suffered: 
> >>> https://www.illumos.org/issues/3150
> >>>
> >>> Le 07/09/12 14:20, Dmitry Kozhinov a écrit :
>  Thank you, Andrey, for pointing me into right direction.
>  Now I have figured out that it does not hang, but stops at console 
>  login
>  prompt. GUI not loads for some reason. Graphics card is ATI Radeon 
>  9600.
>  How do I inspect a boot log (if any)?
> 
>  Maybe the reason of the glitch is that online updates not always go
>  smoothly, especially when there was many incremental updates 
>  already for
>  given system. I remember that I already had similar problem, and fresh
>  install did resolve it (cannot remember OSol or OI). Maybe I will
>  reinstall the whole system at next week. Though it is a production
>  server, there is not much to configure. Apache, vsftpd, and content
>  files. Downtime is not critical.
> 
>  Though reinstall may help, it would be useful to find out what went 
>  wrong.
> 
>  On 07.09.2012 1:03, Andrey Sokolov wrote:
> > Hi,
> >
> > Where does OI hang? Use -v option for kernel and boot without 
> > progress
> > bar
> >
> > 2012/9/6 Dmitry Kozhinov 
> >
> >> Hi all,
> >>
> >> I have a quite old machine running OI: ASUS A7N8X-X motherboard,
> >> Athlon XP
> >> 1150Mhz CPU, 1GB RAM. Everything worked fine until the 151a6 
> >> update. Now
> >> the boot process hangs (progress bar is moving, no HDD activity,
> >> nothing).
> >> Previous 151a5 BE boots fine.
> >>
> >> Any advices appreciated.
> >>
> >
> >
> > ___
> > OpenIndiana-discuss mailing list
> > OpenIndiana-discuss@openindiana.org
> > http://openindiana.org/mailman/listinfo/openindiana-discuss
> 
> 
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] OI stopped booting after 151a6 update.

[Gary Gendel]

Milan,

Please contact me directly if you need me to run some experiments or 
collect some data.  I tried Richard Palo's workaround but it only worked 
in the current session; a reboot brought it back and wouldn't go away 
with a re-applied fix.


I blew a6 away, but I can put it back if you tell me what you want me to 
try.  BTW, this is on a Lenovo Thinkpad T61p.


Gary

On 9/13/12 10:40 AM, Milan Jurik wrote:

Hi,

it would be nice if people affected by this are active on that issue. 
Currently I have only one reporting person who wrote that the problem 
seems to be visible only in case that hald is daemonized. Ongoing 
investigation. I cannot reproduce it on my system (yet).


Best regards,

Milan

On 13.09.2012 13:55, Dmitry Kozhinov wrote:

Yes, it's the same. Production server affected... I am stick to 151a5
for a while.

On 07.09.2012 18:23, Richard PALO wrote:
It's possibly the same as a couple of us have suffered: 
https://www.illumos.org/issues/3150


Le 07/09/12 14:20, Dmitry Kozhinov a écrit :

Thank you, Andrey, for pointing me into right direction.
Now I have figured out that it does not hang, but stops at console 
login
prompt. GUI not loads for some reason. Graphics card is ATI Radeon 
9600.

How do I inspect a boot log (if any)?

Maybe the reason of the glitch is that online updates not always go
smoothly, especially when there was many incremental updates 
already for

given system. I remember that I already had similar problem, and fresh
install did resolve it (cannot remember OSol or OI). Maybe I will
reinstall the whole system at next week. Though it is a production
server, there is not much to configure. Apache, vsftpd, and content
files. Downtime is not critical.

Though reinstall may help, it would be useful to find out what went 
wrong.


On 07.09.2012 1:03, Andrey Sokolov wrote:

Hi,

Where does OI hang? Use -v option for kernel and boot without 
progress

bar

2012/9/6 Dmitry Kozhinov 


Hi all,

I have a quite old machine running OI: ASUS A7N8X-X motherboard,
Athlon XP
1150Mhz CPU, 1GB RAM. Everything worked fine until the 151a6 
update. Now

the boot process hangs (progress bar is moving, no HDD activity,
nothing).
Previous 151a5 BE boots fine.

Any advices appreciated.




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Won't start

[Jan Owoc]

Hi Bob,

On Thu, Sep 13, 2012 at 6:49 AM, Bob Palank  wrote:
> Running XP Pro SP3 w Virtual Box 4.1.22.
> OI goes through the startup process, shows the blue screen with the logo and 
> I see the black mouse pointer which I can move.  After ten minutes the cd 
> load indicator is still flashing and I see no icons - just the logo.
> Suggestions?

Follow the suggestions in this conversation thread and post back:
http://openindiana.org/pipermail/openindiana-discuss/2012-February/007103.html

A) what version of OI are you trying to run?
B) what do you have set as the OS type?
C) how much memory have you allocated to the machine?
D) have you tried it more than once?

Jan

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] OI stopped booting after 151a6 update.

[Milan Jurik]

Hi,

it would be nice if people affected by this are active on that issue. 
Currently I have only one reporting person who wrote that the problem 
seems to be visible only in case that hald is daemonized. Ongoing 
investigation. I cannot reproduce it on my system (yet).


Best regards,

Milan

On 13.09.2012 13:55, Dmitry Kozhinov wrote:

Yes, it's the same. Production server affected... I am stick to 151a5
for a while.

On 07.09.2012 18:23, Richard PALO wrote:
It's possibly the same as a couple of us have suffered: 
https://www.illumos.org/issues/3150


Le 07/09/12 14:20, Dmitry Kozhinov a écrit :

Thank you, Andrey, for pointing me into right direction.
Now I have figured out that it does not hang, but stops at console 
login
prompt. GUI not loads for some reason. Graphics card is ATI Radeon 
9600.

How do I inspect a boot log (if any)?

Maybe the reason of the glitch is that online updates not always go
smoothly, especially when there was many incremental updates 
already for
given system. I remember that I already had similar problem, and 
fresh

install did resolve it (cannot remember OSol or OI). Maybe I will
reinstall the whole system at next week. Though it is a production
server, there is not much to configure. Apache, vsftpd, and content
files. Downtime is not critical.

Though reinstall may help, it would be useful to find out what went 
wrong.


On 07.09.2012 1:03, Andrey Sokolov wrote:

Hi,

Where does OI hang? Use -v option for kernel and boot without 
progress

bar

2012/9/6 Dmitry Kozhinov 


Hi all,

I have a quite old machine running OI: ASUS A7N8X-X motherboard,
Athlon XP
1150Mhz CPU, 1GB RAM. Everything worked fine until the 151a6 
update. Now

the boot process hangs (progress bar is moving, no HDD activity,
nothing).
Previous 151a5 BE boots fine.

Any advices appreciated.




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Announcing mdbackup, perl scripts for zfs backup servers (was: Current ZFS Backup projects)

[Frank Lahm]

There's zrep for (remote) replication:

-f

2012/9/13 Julius Roberts :
> Hi all, I've created https://sourceforge.net/projects/mdbackup/,
> please have a look if you're interested.  I'm very keen on getting
> feedback/suggestions/help so feel free to email me off list etc.  I've
> never done anything like this before, so please be gentle :)
>
> I'm currently writing install/admin doco and also end user doco
> targetted at healthchecking and restores etc.  They are about 80% done
> already, but I do apologize for the tardy doco until they're
> available.
>
> Kind regards, Julius
>
> On 12 September 2012 09:52, Julius Roberts  wrote:
>> I have done this myself but haven't yet got around to releasing it. It's
>> called mdbackup, is written in perl and basically goes off and connects to a
>> bunch of remote servers (defined in a file called mdtab) via ssh or a smb
>> mount, rsyncs the data to a location on your local zpool and then snapshots
>> the deepest parent zfs filesystem. Another script renames snaps to preserve
>> them as monthlies or annuals and another script purges the remainder of
>> snaps based on a retention policy defined in that severs configuration file,
>> normally 21 nighties, 12 monthly and 7 annuals. It all runs from cronies. It
>> does lots of nice logging and error checking and has nice health check
>> scripts which can be used to integrate this with nagios.  it's reasonably
>> mature and stable, I rewrote if from bash to perl a year ago, but in concept
>> it's been running g for 5 years ago, I started it using solaris express
>> developers edition, zfs v10.  I have 4 servers running mdbackup and I manage
>> it all using subversion. As for releasing it into the wild I'm happy do do
>> that, but there are no appropriate licensing statements in any if the files
>> nor have I considered what licence to even use. I was thinking of pushing
>> the code to a slave svn instance from which the public can sync if they
>> like.  Pretty busy right now though :) oh it's all documented too. If it
>> sounds good and you can help some way that would be awesome. Oh and it does
>> encrypted offsite exports too but I'm not real happy with the performance of
>> that at this stage, seems to die over 200gb on the few servers I've tested
>> it on. Anyhow, mdbackup.
>>
>> Kind regards Hoolio
>>
>>
>> On Wednesday, 12 September 2012, Mark Creamer  wrote:
>>> A recent thread caused me to look for open source projects that leverage
>>> ZFS to backup systems. I found a couple, such as OmniTI's
>>> Zetaback,
>>> but that one appears to be dead - at least the links don't work and the
>>> Git
>>> page shows no recent activity. Nexenta's commercial product for Windows,
>>> "Delorean", also appears to have been killed (unfortunately without first
>>> being released to the community as far as I can tell). Wondering if anyone
>>> knows of any other projects that use scripts or some other method to
>>> manage
>>> system backups with zfs. I'm hoping to build on the ideas of someone more
>>> knowledgeable to automate my snapshot and recovery efforts.
>>>
>>> --
>>> Mark Creamer
>>> ___
>>> OpenIndiana-discuss mailing list
>>> OpenIndiana-discuss@openindiana.org
>>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>>>
>>
>> --
>> Kind regards, Jules
>>
>>  whats so wrong with plumb?
>>  nothing, in itself.  it's just for me, knowing what it means infers
>> i cannot any longer pretend to not be a complete square when it comes to
>> computers
>>  I don't know that knowing anything about plumb turns you into a
>> nerd, but this conversation already has
>>  are you calling me nerdy?
>>  hoolio: you know what initramfs means, AND does. You're lost to
>> the non-geek world already
>>  yes
>>  hrm
>>  goodbye cruel world.
>
>
>
> --
> Kind regards, Jules
>
>  whats so wrong with plumb?
>  nothing, in itself.  it's just for me, knowing what it means
> infers i cannot any longer pretend to not be a complete square when it
> comes to computers
>  I don't know that knowing anything about plumb turns you
> into a nerd, but this conversation already has
>  are you calling me nerdy?
>  hoolio: you know what initramfs means, AND does. You're
> lost to the non-geek world already
>  yes
>  hrm
>  goodbye cruel world.
>
> ___
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss@openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] Announcing mdbackup, perl scripts for zfs backup servers (was: Current ZFS Backup projects)

[Julius Roberts]

Hi all, I've created https://sourceforge.net/projects/mdbackup/,
please have a look if you're interested.  I'm very keen on getting
feedback/suggestions/help so feel free to email me off list etc.  I've
never done anything like this before, so please be gentle :)

I'm currently writing install/admin doco and also end user doco
targetted at healthchecking and restores etc.  They are about 80% done
already, but I do apologize for the tardy doco until they're
available.

Kind regards, Julius

On 12 September 2012 09:52, Julius Roberts  wrote:
> I have done this myself but haven't yet got around to releasing it. It's
> called mdbackup, is written in perl and basically goes off and connects to a
> bunch of remote servers (defined in a file called mdtab) via ssh or a smb
> mount, rsyncs the data to a location on your local zpool and then snapshots
> the deepest parent zfs filesystem. Another script renames snaps to preserve
> them as monthlies or annuals and another script purges the remainder of
> snaps based on a retention policy defined in that severs configuration file,
> normally 21 nighties, 12 monthly and 7 annuals. It all runs from cronies. It
> does lots of nice logging and error checking and has nice health check
> scripts which can be used to integrate this with nagios.  it's reasonably
> mature and stable, I rewrote if from bash to perl a year ago, but in concept
> it's been running g for 5 years ago, I started it using solaris express
> developers edition, zfs v10.  I have 4 servers running mdbackup and I manage
> it all using subversion. As for releasing it into the wild I'm happy do do
> that, but there are no appropriate licensing statements in any if the files
> nor have I considered what licence to even use. I was thinking of pushing
> the code to a slave svn instance from which the public can sync if they
> like.  Pretty busy right now though :) oh it's all documented too. If it
> sounds good and you can help some way that would be awesome. Oh and it does
> encrypted offsite exports too but I'm not real happy with the performance of
> that at this stage, seems to die over 200gb on the few servers I've tested
> it on. Anyhow, mdbackup.
>
> Kind regards Hoolio
>
>
> On Wednesday, 12 September 2012, Mark Creamer  wrote:
>> A recent thread caused me to look for open source projects that leverage
>> ZFS to backup systems. I found a couple, such as OmniTI's
>> Zetaback,
>> but that one appears to be dead - at least the links don't work and the
>> Git
>> page shows no recent activity. Nexenta's commercial product for Windows,
>> "Delorean", also appears to have been killed (unfortunately without first
>> being released to the community as far as I can tell). Wondering if anyone
>> knows of any other projects that use scripts or some other method to
>> manage
>> system backups with zfs. I'm hoping to build on the ideas of someone more
>> knowledgeable to automate my snapshot and recovery efforts.
>>
>> --
>> Mark Creamer
>> ___
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss@openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>>
>
> --
> Kind regards, Jules
>
>  whats so wrong with plumb?
>  nothing, in itself.  it's just for me, knowing what it means infers
> i cannot any longer pretend to not be a complete square when it comes to
> computers
>  I don't know that knowing anything about plumb turns you into a
> nerd, but this conversation already has
>  are you calling me nerdy?
>  hoolio: you know what initramfs means, AND does. You're lost to
> the non-geek world already
>  yes
>  hrm
>  goodbye cruel world.



-- 
Kind regards, Jules

 whats so wrong with plumb?
 nothing, in itself.  it's just for me, knowing what it means
infers i cannot any longer pretend to not be a complete square when it
comes to computers
 I don't know that knowing anything about plumb turns you
into a nerd, but this conversation already has
 are you calling me nerdy?
 hoolio: you know what initramfs means, AND does. You're
lost to the non-geek world already
 yes
 hrm
 goodbye cruel world.

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

[OpenIndiana-discuss] Won't start

[Bob Palank]

Running XP Pro SP3 w Virtual Box 4.1.22.
OI goes through the startup process, shows the blue screen with the logo and I 
see the black mouse pointer which I can move.  After ten minutes the cd load 
indicator is still flashing and I see no icons - just the logo.
Suggestions?
Bob
___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] OI stopped booting after 151a6 update.

[Dmitry Kozhinov]

Yes, it's the same. Production server affected... I am stick to 151a5 
for a while.


On 07.09.2012 18:23, Richard PALO wrote:
It's possibly the same as a couple of us have suffered: 
https://www.illumos.org/issues/3150


Le 07/09/12 14:20, Dmitry Kozhinov a écrit :

Thank you, Andrey, for pointing me into right direction.
Now I have figured out that it does not hang, but stops at console login
prompt. GUI not loads for some reason. Graphics card is ATI Radeon 9600.
How do I inspect a boot log (if any)?

Maybe the reason of the glitch is that online updates not always go
smoothly, especially when there was many incremental updates already for
given system. I remember that I already had similar problem, and fresh
install did resolve it (cannot remember OSol or OI). Maybe I will
reinstall the whole system at next week. Though it is a production
server, there is not much to configure. Apache, vsftpd, and content
files. Downtime is not critical.

Though reinstall may help, it would be useful to find out what went 
wrong.


On 07.09.2012 1:03, Andrey Sokolov wrote:

Hi,

Where does OI hang? Use -v option for kernel and boot without progress
bar

2012/9/6 Dmitry Kozhinov 


Hi all,

I have a quite old machine running OI: ASUS A7N8X-X motherboard,
Athlon XP
1150Mhz CPU, 1GB RAM. Everything worked fine until the 151a6 
update. Now

the boot process hangs (progress bar is moving, no HDD activity,
nothing).
Previous 151a5 BE boots fine.

Any advices appreciated.

__**_
OpenIndiana-discuss mailing list
OpenIndiana-discuss@**openindiana.org 



http://openindiana.org/**mailman/listinfo/openindiana-**discuss 





___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Current ZFS Backup projects (OpenIndiana-discuss Digest, Vol 26, Issue 34)

[Jim Klimov]

2012-09-12 6:05, Ong Yu-Phing пишет:

Jim, I assume you are referring to this:
http://wiki.openindiana.org/oi/rsync+daemon+service+on+OpenIndiana, thanks!


Yes, I think that's it ;)


My concern is that typically rsync will take quite a while to traverse a
large set of files before sending only changed files; a classic example
is backing up say 1TB of maildir emails, it may take 4+ hours, and you
now have to deal with a situation where your midnight backup is really a
"somewhere between midnight and 4am" backup.  And of course, if you want
to take snapshots/backups of comstar volumes, rsync isn't quite the
right fit.

On the other hand, a zfs snapshot gives an almost-at-the-time backup
(give or take a few seconds), versus the aforementioned rsync.  The zfs
snapshot can then be sent off-site, independent of the backup activity.


I got an impression that you needed to backup some "client" machines
with varied OSes, such as Windows or Linux desktops, onto a ZFS server.
In that case rsync should help, although you're right that it would
take long to scan the directory trees for changes.

With client FSes that support snapshots (ZFS, NTFS shadow copy) you
might have some luck making scripts that take the client's snapshot,
hold it (in case of ZFS, to avoid it being destroyed while you're
working), rsync the changes from the snapshot (so the 0am backup is
really the state at 0am) and release/destroy the snapshot on client.
In case of ZFS at least, you might have some optimization by using
"zfs diff" to determine changed files between two snapshots on the
client - but then you should not destroy rsynced snapshots right
away, but keep a backlog of one or two at least. And you should have
some locking to prevent several instances of the backup job crawling
the same client space and bringing IOPS to a halt.

Now, before you ask "why not zfs-send client snapshots directly?" -
there may be reasons, such as incompatible ZFS versions on client
and server, differing dataset layouts, flaky network preventing
transfer of large zfs-send streams (though that should have been
addressed with resumable zfs-send feature, if that was integrated).

HTH,
//Jim Klimov


___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Current ZFS Backup projects

[Jim Klimov]

2012-09-12 3:52, Julius Roberts wrote:

As for releasing it into the
wild I'm happy do do that, but there are no appropriate licensing
statements in any if the files nor have I considered what licence to even
use. I was thinking of pushing the code to a slave svn instance from which
the public can sync if they like.  Pretty busy right now though :) oh it's
all documented too.


Judging from my small experience with a few projects, you
can get free SVN/tarball/wiki hosting on sourceforge.net.
If you want to just release the code and don't care who or
how uses it and don't require that you get feedbacks and
improvements, you can consider some variants of the BSD
license.

HTH,
//Jim Klimov

___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] Problems with netcat zfs receive

[Flo]

Hello,

thank you very much! With the "d" flag, the nc commands works very well!

Regards
Florian

On 09/11/2012 02:33 PM, James Carlson wrote:

Florian wrote:

/root@oi151a6-1:~# ssh -n -f 192.168.10.201 "/usr/bin/nc -l -p 1337 |
/usr/sbin/zfs receive -F tank/raid1-2"/


Try "nc -dl -p 1337" instead of just "-l".  The problem is that the ssh
"-n" flag provides /dev/null on stdin, rather than a tty, and that
confuses netcat.  The netcat "-d" option tells it to ignore stdin.




___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Re: [OpenIndiana-discuss] How to help if you are not developer: Re: "OpenIndiana lead Alasdair Lumsden resigns"

[Paolo Marcheschi]

Hi

I think that first we need to speed up the process of publication of 
contributed software,
for example I have contributed 2 software (dcmtk, xmedcon)  to illumos in 2011, 
and after the approval and corrections they are not published yet.
In the mean time due to my work  I ported several other software , we use for 
daily job, but I'm doubtful whether to publish or not due to this very 
difficult process.

Paolo


On 09/12/12 09:32 PM, Bob Friesenhahn wrote:

On Wed, 12 Sep 2012, Milan Jurik wrote:


It would be useful to build up a comparison matrix of top packages in
the Debian Popularity Contest (http://popcon.debian.org/) and compare
these packages to the ones that OpenIndiana currently offers to make
sure that OpenIndiana is not ignoring popular packages which might
easily compile and be useful on OpenIndiana.


I would prefer to avoid such way. Sun internal people know how such
thing ends... Packages should be added by people who need them and
volunteer to maintain them. Otherwise system is full of popular packages
which are not maintained.


I agree that adding new packages is not trivial and each package needs a maintainer.  However, it would be good to know about any commonly-installed portable packages in stable Linux distributions that OI is missing.  Knowledge of something missing does not mean that OI is prepared to accept it.  
Many OI users are traditional Solaris users and might not also use Linux so they may not be aware of something useful which is missing.


My impression is that OI + SFE is doing pretty good with the selection of packages, but some packages in OI are in need of being updated and can be updated without any risk.  For example, I found an annoying bug in OI's emacs which made it somewhat unusable.  The issue is not specific to OI 
because emacs in Ubuntu 10.4 has the same bug and the bug is well known.  The SFE version of emacs is new enough to solve that bug.


Bob



___
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss