Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Thu, Jun 29, 2017 at 09:37:25PM -0400, Greg Troxel wrote:
> (Are you really using ACLS?  It would be interesting if you posted a
> note about what you are doing, how, why, and how well it works.)

Thanks for above clarifications. I have not used bup till now. We were
mainly comparing bup+cgd with encfs/cryfs. I just cited NetBSD specific
bullet points from upstream web page for clarity.

Mayuresh.

Re: Can NetBSD cgd be used for encrypted backup?

[Greg Troxel]

Mayuresh  writes:

> On Tue, Jun 27, 2017 at 07:53:35PM -0400, Greg Troxel wrote:
>> One could use sysutils/bup for this, or one of the bup followon programs
>
> bup seems a very good tool to me. I have used duplicity before and find
> bup to be a lot simpler than it.
>
> But a few notes on the bup web page[1] intrigue me, namely:
>
>"bup is overly optimistic about mmap. Right now bup just assumes that
>it can mmap as large a block as it likes, and that mmap will never
>fail.  Yeah, right... If nothing else, this has failed on 32-bit
>architectures (and 31-bit is even worse -- looking at you, s390)."
>
> Does it imply it will have issues on NetBSD i386?

bup save mmaps indexes, but not the data itself.  If you try to back up
about 400G on a machine with only 2G of RAM, you will run into problems
with the default build.  I have a patch that causing the hashsplit
pieces to be 8x bigger which results in ~8x less RAM use, and then 400G
backups are fine.

> Also:
>
> "bup fuse" presents every directory/file as inode 0. The directory
> traversal code ("fts") in NetBSD's libc will interpret this as a cycle and
> error out, so "ls -R" and "find" will not work.

That's true.  But you don't need bup fuse to do backups or restores.   I
have been using bup for backups and actually had to restore after a disk
failure, and it worked.

> There is no support for ACLs. If/when some entrprising person fixes this,
> adjust t/compare-trees.

My impression is that bup does ACLs fine, and that the issue is that
some file systems don't.

(Are you really using ACLS?  It would be interesting if you posted a
note about what you are doing, how, why, and how well it works.)


signature.asc
Description: PGP signature

Re: Can NetBSD cgd be used for encrypted backup?

[Brad Spencer]

Mayuresh  writes:

> On Tue, Jun 27, 2017 at 07:53:35PM -0400, Greg Troxel wrote:
>> Comparing encfs to cgd, the big issues are
>> 
>>  - stability of codebase to be able to get your bits back much later (?,
>>but cgd seems like it has been quite stable.  But it's NetBSD only
>>AFAIK.)
>> 
>>  - exposing the structure of your filesystem such as the histogram of
>>file sizes, directory organization, and when various parts were
>>updated (cgd wins)
>> 
>>   - having ciphertext size scale with size of plaintext easily (encfs
>> wins)
>
> One more point: I have to identify a virtual disk or partition to use cgd.
> When doing so I have to decide its size up front and reserve that much
> space, even if I may not need that today. And when that space fills, I am
> not sure whether there ways to expand the FS easily. Even if there are,
> encfs/cryfs are lot more convenient here as they piggyback on native FS in
> user space and do not really require setting aside space for them.
>
> Mayuresh.

I put a cgd on top of a lvm and had no particular trouble resizing the
cgd.  I just used resize_ffs like I might have done on a Not-cgd fs and
it worked just fine.


-- 
Brad Spencer - b...@anduin.eldar.org - KC8VKS
http://anduin.eldar.org  - & -  http://anduin.ipv6.eldar.org [IPv6 only]

Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Tue, Jun 27, 2017 at 07:53:35PM -0400, Greg Troxel wrote:
> One could use sysutils/bup for this, or one of the bup followon programs

bup seems a very good tool to me. I have used duplicity before and find
bup to be a lot simpler than it.

But a few notes on the bup web page[1] intrigue me, namely:

   "bup is overly optimistic about mmap. Right now bup just assumes that
   it can mmap as large a block as it likes, and that mmap will never
   fail.  Yeah, right... If nothing else, this has failed on 32-bit
   architectures (and 31-bit is even worse -- looking at you, s390)."

Does it imply it will have issues on NetBSD i386?

Also:

"bup fuse" presents every directory/file as inode 0. The directory
traversal code ("fts") in NetBSD's libc will interpret this as a cycle and
error out, so "ls -R" and "find" will not work.

There is no support for ACLs. If/when some entrprising person fixes this,
adjust t/compare-trees.


[1] https://github.com/bup/bup

Mayuresh

Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Tue, Jun 27, 2017 at 07:53:35PM -0400, Greg Troxel wrote:
> Comparing encfs to cgd, the big issues are
> 
>  - stability of codebase to be able to get your bits back much later (?,
>but cgd seems like it has been quite stable.  But it's NetBSD only
>AFAIK.)
> 
>  - exposing the structure of your filesystem such as the histogram of
>file sizes, directory organization, and when various parts were
>updated (cgd wins)
> 
>   - having ciphertext size scale with size of plaintext easily (encfs
> wins)

One more point: I have to identify a virtual disk or partition to use cgd.
When doing so I have to decide its size up front and reserve that much
space, even if I may not need that today. And when that space fills, I am
not sure whether there ways to expand the FS easily. Even if there are,
encfs/cryfs are lot more convenient here as they piggyback on native FS in
user space and do not really require setting aside space for them.

Mayuresh.

Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Tue, Jun 27, 2017 at 07:53:35PM -0400, Greg Troxel wrote:
>  - exposing the structure of your filesystem such as the histogram of
>file sizes, directory organization, and when various parts were
>updated (cgd wins)

There is a sibling of encfs - cryfs, which while being similar in many
other aspects, addresses this point. It is not yet available on NetBSD,
but surprisingly someone made an erroneous entry on wikipedia that shows
it to be available on NetBSD[1].

Thread on this:
https://mail-index.netbsd.org/netbsd-users/2017/06/10/msg019645.html

Mayuresh

Re: Can NetBSD cgd be used for encrypted backup?

[Greg Troxel]

Mayuresh  writes:

> On Sun, Jun 11, 2017 at 04:32:02PM +0200, Kamil Rytarowski wrote:
>> > - Can the native cgd of NetBSD be used for the purpose of encrypted
>> >   backup? Basically can I mount such filesystem in a way that it shows
>> >   encrypted files?
>> 
>> I use cgd(4) devices for encrypted backup.
>
> Alternatively one could backup the virtual file, but that's not so useful
> for cloud backup.

One could use sysutils/bup for this, or one of the bup followon programs
(attic?).  They use an rsync-like rolling-checksum algorithm to
deduplicate fragments of large files.  So in theory if you rsync your
main filesystem to the filesystem within the cgd, running bup should not
create vast incremental output.

cgd works very well for, e.g., encrypting an entire 1T desk.  This is
great for taking a disk offsite and bringing it back occasionally to
write new backups (times N, rotating).  But I realize that's not what
you are talking about.

Comparing encfs to cgd, the big issues are

 - stability of codebase to be able to get your bits back much later (?,
   but cgd seems like it has been quite stable.  But it's NetBSD only
   AFAIK.)

 - exposing the structure of your filesystem such as the histogram of
   file sizes, directory organization, and when various parts were
   updated (cgd wins)

  - having ciphertext size scale with size of plaintext easily (encfs
wins)

I would be tempted to try an encfs filesystem, but to use bup and put
the BUPDIR in the encfs.  So what would be stored would be git-format
packfiles, which contain the structure of your fs but don't betray it in
their size.  Or you could use dump, but the nice thing about things like
bup is that the subsequent backups are both fast and small.  Yet they
can be restored straightforwardly.

I have the impression some bup-like programs have builtin
application-layer encryption.  I at first thought this was unreasonable
and something like encfs should be used, but having seeen the history of
encfs being broken in pkgsrc for so long, having the encryption for
packfiles could be a good choice, given that the encryption part is
fastly simpler than the FUSE/VFS part.


signature.asc
Description: PGP signature

Re: Can NetBSD cgd be used for encrypted backup?

[Michael van Elst]

swiftgri...@gmail.com (Swift Griggs) writes:

>Anyhow, based on my bad experience, I wouldn't recommend iSCSI for anyone 
>unless they simply had no other choice. I have seen it be workable, 
>especially with dedicated hardware (Equallogic gear seems to work okay, 
>and it's got NetBSD bits in there too!), but overall, I'd run screaming 
>away.

I guess, if you want performance, you just use fibre channel,
for cheap and ubiquitous you use iSCSI and the future is
NVMe over fabrics.

-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."

Re: Can NetBSD cgd be used for encrypted backup?

[Swift Griggs]

On Mon, 19 Jun 2017, Mayuresh wrote:
Just curious. How does iscsi compare with NFS? Guess even NFS has a 
notion of block size, that would help optimize io.


Sorry for butting in, but I'd point out that NFS is file-based and layers 
on top of an existing filesystem. So, the block size of the underlying 
file system is going to determine the block size. There is also the 
consideration of the network parameters such as send and receive buffer 
sizes and several others that matter quite a bit (depending on the 
layer-4 protocol in use and the version of NFS).


iSCSI only provides block devices, it can't do file-based I/O natively 
without a filesystem on top of it. My experience with iSCSI has overall 
been quite poor. I once did a long whitepaper on iSCSI vs AoE. Being a big 
fan of SCSI (and not a huge fan of ATA) I was hoping & expecting iSCSI was 
going to be better than it turned out. However, the experience turned out 
completely opposite. Not only did AoE stomp it in every performance test I 
tried, it also scaled better, recovered from failures better, and so 
forth. iSCSI also has a million dials and settings for mostly useless crap 
few are going to fiddle with. It feels like some kind of top-heavy 
machination designed by some committee somewhere that never has to use 
network block storage in-real-life.


I've also seen large scale iSCSI deployments be fraught with pain and 
peril simply because network engineers can't be trusted to leave the VLANs 
it runs on alone and can't be bothered to put it on discrete switches.


Of course AoE runs on top of layer-2 and iSCSI is a layer-5 protocol. The 
extra layers underneath iSCSI make it routable, but destroy performance. 
With AoE you don't have to tune TCP/IP (but it's non-routable).


I also remember hearing about HyperSCSI which is supposed to be hybrid 
strategy that uses SCSI CDB's over Ethernet frames like AoE does. My guess 
is, based on AoE's good-showing, that approach would rock if they got it 
off the ground.


I guess I should also point out that iSCSI is widely supported across a 
larger number of operating systems than AoE and has much more vendor 
acceptance since AoE is seen as the domain of the CORAID (or whatever they 
are called now) folks.


Anyhow, based on my bad experience, I wouldn't recommend iSCSI for anyone 
unless they simply had no other choice. I have seen it be workable, 
especially with dedicated hardware (Equallogic gear seems to work okay, 
and it's got NetBSD bits in there too!), but overall, I'd run screaming 
away.


iSCSI does give a block device to use with CGD, though. I bet it would 
work fine with CGD, despite being kind of a poor idea in general (iSCSI 
not CGD).


-Swift

Just my opinions here. If you use iSCSI and love it, YMMV, and more power 
to you.

Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Tue, Jun 13, 2017 at 10:31:14AM +0200, Martin Husemann wrote:
> No, just an iscssi device. Many NAS devices can export a "partition" via
> iscssi.

Just curious. How does iscsi compare with NFS? Guess even NFS has a notion
of block size, that would help optimize io.

Mayuresh.

Re: Can NetBSD cgd be used for encrypted backup?

[Stephen Borrill]

On Wed, 14 Jun 2017, Michael van Elst wrote:

On Wed, Jun 14, 2017 at 10:32:18AM +0530, Mayuresh wrote:

On Wed, Jun 14, 2017 at 04:23:08AM -, Michael van Elst wrote:

An ISCSI target on the other hand doesn't require any privileges if
you just export a file as a disk image.


I mean, even to export it once from the target side, you'd need root. No?
If not, can you please provide more details such as name of the software
on the target side etc.?


iscsi_target is in base.


net/istgt is much better...

"better" meaning resilient, compatible, deals with SIGHUP, etc.

--
Stephen

Re: Can NetBSD cgd be used for encrypted backup?

[Michael van Elst]

On Wed, Jun 14, 2017 at 10:32:18AM +0530, Mayuresh wrote:
> On Wed, Jun 14, 2017 at 04:23:08AM -, Michael van Elst wrote:
> > An ISCSI target on the other hand doesn't require any privileges if
> > you just export a file as a disk image.
> 
> I mean, even to export it once from the target side, you'd need root. No?
> If not, can you please provide more details such as name of the software
> on the target side etc.?

iscsi_target is in base.

Linux has TGT and probably a couple of other packages. I've never used
them.

> I have got one VPS where I have root, but not much disk space. I have
> another shell account form another provider where I have space, but no
> root. It's the latter where I wish to use for backup.

No idea if iscsi is a good solution for that, but it should work.


Greetings,
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."

Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Wed, Jun 14, 2017 at 04:23:08AM -, Michael van Elst wrote:
> An ISCSI target on the other hand doesn't require any privileges if
> you just export a file as a disk image.

I mean, even to export it once from the target side, you'd need root. No?
If not, can you please provide more details such as name of the software
on the target side etc.?

I have got one VPS where I have root, but not much disk space. I have
another shell account form another provider where I have space, but no
root. It's the latter where I wish to use for backup.

Mayuresh.

Re: Can NetBSD cgd be used for encrypted backup?

[Michael van Elst]

mayur...@acm.org (Mayuresh) writes:

>I think creating iscssi device would need root.

>I searched for user space iscssi, but couldn't get much concrete
>information.

An ISCSI initiator attaches device drivers, and that usually requires
root permission.

An ISCSI target on the other hand doesn't require any privileges if
you just export a file as a disk image.

-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."

Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Tue, Jun 13, 2017 at 10:31:14AM +0200, Martin Husemann wrote:
> On Mon, Jun 12, 2017 at 12:19:33PM +0530, Mayuresh wrote:
> > That's quite interesting. Would like to try out. But I guess, you'd need
> > root access at remote? No?
> 
> No, just an iscssi device. Many NAS devices can export a "partition" via
> iscssi.

I think creating iscssi device would need root.

I searched for user space iscssi, but couldn't get much concrete
information.

Mayuresh

Re: Can NetBSD cgd be used for encrypted backup?

[Martin Husemann]

On Mon, Jun 12, 2017 at 12:19:33PM +0530, Mayuresh wrote:
> That's quite interesting. Would like to try out. But I guess, you'd need
> root access at remote? No?

No, just an iscssi device. Many NAS devices can export a "partition" via
iscssi.

Martin

Re: Can NetBSD cgd be used for encrypted backup?

[Jan Danielsson]

On 06/12/17 15:46, Kamil Rytarowski wrote:
[---]
> I'm thinking about reuploading block device image for each change. I
> noted in the past that people were trying to do the same with TrueCrypt,
> and they give up - it's good for one-time upload of something, but it's
> not usable in scenarios that the volume has to be altered even for few bits.

   I have no idea how these "cloud services" work, but can one use rsync
against any of them?

   For many years I had an backup image file which I vnd'd and cgd'd and
then copied files to -- then I unconfigured the cgd and vnd and pushed
the file using rsync --checksum to a remote system.  For a time I did
this with a TrueCrypt image on a Windows XP laptop as well, and it
worked great.  (Again, thanks to "rsync --checksum").


-- 
Kind regards,
Jan Danielsson

Re: Can NetBSD cgd be used for encrypted backup?

[Kamil Rytarowski]

On 12.06.2017 15:03, Thor Lancelot Simon wrote:
> On Mon, Jun 12, 2017 at 02:55:48PM +0200, Kamil Rytarowski wrote:
>> On 12.06.2017 14:45, Thor Lancelot Simon wrote:
>>> On Sun, Jun 11, 2017 at 06:41:56PM +0200, Kamil Rytarowski wrote:

 You would need to push large part image of an encrypted volume for every
 change to files.
>>>
>>> That doesn't make sense to me.  Why would you need to push more blocks
>>> than actually changed?
>>>
>>
>> Cloud hosting would require partial updates of files. But I think we can
>> agree that using cgd raw image is not the best tool for this task.
> 
> I still don't understand what you're getting at.  Are you suggesting backing
> the image with S3 or something?  If so, I don't actually see why the write
> amplification problem is any worse for block-based or file-based storage,
> though it's bad for a small-write workload either way.
> 

I'm thinking about reuploading block device image for each change. I
noted in the past that people were trying to do the same with TrueCrypt,
and they give up - it's good for one-time upload of something, but it's
not usable in scenarios that the volume has to be altered even for few bits.



signature.asc
Description: OpenPGP digital signature

Re: Can NetBSD cgd be used for encrypted backup?

[Thor Lancelot Simon]

On Mon, Jun 12, 2017 at 02:55:48PM +0200, Kamil Rytarowski wrote:
> On 12.06.2017 14:45, Thor Lancelot Simon wrote:
> > On Sun, Jun 11, 2017 at 06:41:56PM +0200, Kamil Rytarowski wrote:
> >>
> >> You would need to push large part image of an encrypted volume for every
> >> change to files.
> > 
> > That doesn't make sense to me.  Why would you need to push more blocks
> > than actually changed?
> > 
> 
> Cloud hosting would require partial updates of files. But I think we can
> agree that using cgd raw image is not the best tool for this task.

I still don't understand what you're getting at.  Are you suggesting backing
the image with S3 or something?  If so, I don't actually see why the write
amplification problem is any worse for block-based or file-based storage,
though it's bad for a small-write workload either way.

-- 
  Thor Lancelot Simont...@panix.com

  "We cannot usually in social life pursue a single value or a single moral
   aim, untroubled by the need to compromise with others."  - H.L.A. Hart

Re: Can NetBSD cgd be used for encrypted backup?

[Kamil Rytarowski]

On 12.06.2017 14:45, Thor Lancelot Simon wrote:
> On Sun, Jun 11, 2017 at 06:41:56PM +0200, Kamil Rytarowski wrote:
>>
>> You would need to push large part image of an encrypted volume for every
>> change to files.
> 
> That doesn't make sense to me.  Why would you need to push more blocks
> than actually changed?
> 

Cloud hosting would require partial updates of files. But I think we can
agree that using cgd raw image is not the best tool for this task.



signature.asc
Description: OpenPGP digital signature

Re: Can NetBSD cgd be used for encrypted backup?

[Thor Lancelot Simon]

On Sun, Jun 11, 2017 at 06:41:56PM +0200, Kamil Rytarowski wrote:
> 
> You would need to push large part image of an encrypted volume for every
> change to files.

That doesn't make sense to me.  Why would you need to push more blocks
than actually changed?

-- 
  Thor Lancelot Simont...@panix.com

  "We cannot usually in social life pursue a single value or a single moral
   aim, untroubled by the need to compromise with others."  - H.L.A. Hart

Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Mon, Jun 12, 2017 at 07:51:00AM +0200, Martin Husemann wrote:
> On Sun, Jun 11, 2017 at 09:59:17PM +0530, Mayuresh wrote:
> > On Sun, Jun 11, 2017 at 06:12:58PM +0200, Martin Husemann wrote:
> > > I have used cgd for remote encrypted backups in the past:
> > > 
> > >  - remote offers a "partition" as iscsi device
> > 
> > Not over network, such as cloud backup, right?
> 
> The "remote" in my case was a another office, a few 100 km away. But not
> a generic "cloud provider".

That's quite interesting. Would like to try out. But I guess, you'd need
root access at remote? No?

Mayuresh

Re: Can NetBSD cgd be used for encrypted backup?

[Martin Husemann]

On Sun, Jun 11, 2017 at 09:59:17PM +0530, Mayuresh wrote:
> On Sun, Jun 11, 2017 at 06:12:58PM +0200, Martin Husemann wrote:
> > I have used cgd for remote encrypted backups in the past:
> > 
> >  - remote offers a "partition" as iscsi device
> 
> Not over network, such as cloud backup, right?

The "remote" in my case was a another office, a few 100 km away. But not
a generic "cloud provider".

Martin

Re: Can NetBSD cgd be used for encrypted backup?

[Kamil Rytarowski]

On 11.06.2017 18:26, Mayuresh wrote:
> On Sun, Jun 11, 2017 at 06:16:50PM +0200, Kamil Rytarowski wrote:
>>> Ok, you mean, I can mount it such that it shows encrypted files?
>>>
>>
>> Yes, use cgdconfig(8).
> 
> It won't really show files in encrypted form, I guess.
> 

You need to create a regular partition on a cgd(4) device.

>> Putting image of encrypted partition to cloud sounds less trivial. You
>> might need a lot of bandwidth to use it.
> 
> Right, this is where encfs/cryfs is better.
> 

You would need to push large part image of an encrypted volume for every
change to files.

I cannot speak fore encfs/cryfs.

> Mayuresh.
> 




signature.asc
Description: OpenPGP digital signature

Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Sun, Jun 11, 2017 at 06:12:58PM +0200, Martin Husemann wrote:
> I have used cgd for remote encrypted backups in the past:
> 
>  - remote offers a "partition" as iscsi device

Not over network, such as cloud backup, right?

encfs/cryfs like approach shows the encrypted directory which can be
backed up over network easily, due to incremental nature of rsync.

I understand, something like that won't be possible with cgd.

Mayuresh.

Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Sun, Jun 11, 2017 at 06:16:50PM +0200, Kamil Rytarowski wrote:
> > Ok, you mean, I can mount it such that it shows encrypted files?
> > 
> 
> Yes, use cgdconfig(8).

It won't really show files in encrypted form, I guess.

> Putting image of encrypted partition to cloud sounds less trivial. You
> might need a lot of bandwidth to use it.

Right, this is where encfs/cryfs is better.

Mayuresh.

Re: Can NetBSD cgd be used for encrypted backup?

[Kamil Rytarowski]

On 11.06.2017 17:57, Mayuresh wrote:
> On Sun, Jun 11, 2017 at 04:32:02PM +0200, Kamil Rytarowski wrote:
>>> - Can the native cgd of NetBSD be used for the purpose of encrypted
>>>   backup? Basically can I mount such filesystem in a way that it shows
>>>   encrypted files?
>>>
>>
>> I use cgd(4) devices for encrypted backup.
> 
> Ok, you mean, I can mount it such that it shows encrypted files?
> 

Yes, use cgdconfig(8).

> And are the changes to files incremental (not necessarily one one one for
> encryption reasons), so that rsync can be effectively used for backup over
> cloud? (i.e. the amount of data movement when using rsync is somewhat - if
> not exactly - proportional to amount of change since last backup.)
> 
> Alternatively one could backup the virtual file, but that's not so useful
> for cloud backup.
> 

I don't use incremental or cloud backup, I just use a regular block
device on a portable storage (USB disk).

Putting image of encrypted partition to cloud sounds less trivial. You
might need a lot of bandwidth to use it.

> Mayuresh.
> 




signature.asc
Description: OpenPGP digital signature

Re: Can NetBSD cgd be used for encrypted backup?

[Martin Husemann]

On Sun, Jun 11, 2017 at 09:27:25PM +0530, Mayuresh wrote:
> On Sun, Jun 11, 2017 at 04:32:02PM +0200, Kamil Rytarowski wrote:
> > > - Can the native cgd of NetBSD be used for the purpose of encrypted
> > >   backup? Basically can I mount such filesystem in a way that it shows
> > >   encrypted files?
> > > 
> > 
> > I use cgd(4) devices for encrypted backup.
> 
> Ok, you mean, I can mount it such that it shows encrypted files?

cgd is an encrypted disk, not a file system. I encrypts/decrypts disk
blocks when reading/writing, it does not know about files at all.

I have used cgd for remote encrypted backups in the past:

 - remote offers a "partition" as iscsi device
 - via iscsi the remote partition shows up as (say) sd0 on my machine
 - I (locally) configure cgd to use sd0c (or sd0d)
 - all crypto setup stays local, remote has no way to decrypt the data
 - when doing a backup I bring up iscsi, configure cgd, mount the cgd
   disk and rsync all changes over, then unconfigre cgd and disconnect
   iscssi

In my case it was a company setup, I had to comply with "need to have
automatic backups at *this* facility" policy, but I did not trust admis
at that facility. I kept a printout of the cgd setup in a off-site safe.

There are certainly various other ways to do something similar.

Martin

Re: Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

On Sun, Jun 11, 2017 at 04:32:02PM +0200, Kamil Rytarowski wrote:
> > - Can the native cgd of NetBSD be used for the purpose of encrypted
> >   backup? Basically can I mount such filesystem in a way that it shows
> >   encrypted files?
> > 
> 
> I use cgd(4) devices for encrypted backup.

Ok, you mean, I can mount it such that it shows encrypted files?

And are the changes to files incremental (not necessarily one one one for
encryption reasons), so that rsync can be effectively used for backup over
cloud? (i.e. the amount of data movement when using rsync is somewhat - if
not exactly - proportional to amount of change since last backup.)

Alternatively one could backup the virtual file, but that's not so useful
for cloud backup.

Mayuresh.

Re: Can NetBSD cgd be used for encrypted backup?

[Kamil Rytarowski]

On 11.06.2017 16:22, Mayuresh wrote:
> I am coming from encfs/cryfs on Linux, which allow encryption at directory
> level. A user space mount shows the unencrypted contents, while the
> physical disk has encrypted contents, which can be easily backed up.
> 
> encfs on NetBSD seems broken and dated to me, while cryfs doesn't seem to
> be available (contrary to wikipedia page
> https://en.wikipedia.org/wiki/Comparison_of_disk_encryption_software#Operating_systems
> )
> 
> Have a few questions about cgd in this context:
> 
> - Can the native cgd of NetBSD be used for the purpose of encrypted
>   backup? Basically can I mount such filesystem in a way that it shows
>   encrypted files?
> 

I use cgd(4) devices for encrypted backup.

> - Can I use a virtual filesystem (loop device/vnode disk) with cgd?
> 

vnd(4) can be used.

> - Can I dynamically grow such virtual filesystem as the space requirement
>   grows?
> 

I've never tried growing, in theory this should work, if not directly
with userland tools than with external ones (like qemu disk format).

> Mayuresh
> 




signature.asc
Description: OpenPGP digital signature

Can NetBSD cgd be used for encrypted backup?

[Mayuresh]

I am coming from encfs/cryfs on Linux, which allow encryption at directory
level. A user space mount shows the unencrypted contents, while the
physical disk has encrypted contents, which can be easily backed up.

encfs on NetBSD seems broken and dated to me, while cryfs doesn't seem to
be available (contrary to wikipedia page
https://en.wikipedia.org/wiki/Comparison_of_disk_encryption_software#Operating_systems
)

Have a few questions about cgd in this context:

- Can the native cgd of NetBSD be used for the purpose of encrypted
  backup? Basically can I mount such filesystem in a way that it shows
  encrypted files?

- Can I use a virtual filesystem (loop device/vnode disk) with cgd?

- Can I dynamically grow such virtual filesystem as the space requirement
  grows?

Mayuresh