Re: Looking for advice on tools (or libraries) for unsupervised, bulk symmetric encryption/decryption of files

[John Crawley]

On 13/01/2019 12.46, Celejar wrote:

On Fri, 11 Jan 2019 21:45:57 +

I believe that the most commonly used software for file level
encryption is EncFS. I haven't really used it much, and can't speak to
its long term stablity.


EncFS should not be used for any new file encryption project, IMHO.
There was the following report in 2014:
https://defuse.ca/audits/encfs.htm
This is referenced in the NEWS file in the EncFS package
https://salsa.debian.org/debian/encfs/blob/debian/sid/debian/NEWS

Both the report and the NEWS file are 5 years sold so I am not sure of
its current status but I'd want to seek positive assurance.


Huh - good to know. But I was wondering, along similar (but less
informed) lines, how good some of the other suggestions were, e.g.
ccrypt. I know very little about ccrypt, but has it even been
audited at all? Is it sufficiently widely used that any vulnerablities
or misimplementations of the sort discovered by the EncFS audit would
have been looked for or turned up?


Looking at encfs, gocryptfs showed up, which claims "This project was  
inspired by EncFS and strives to fix its security issues while providing  
good performance":

https://packages.debian.org/stretch/gocryptfs
https://github.com/rfjakob/gocryptfs

No personal experience (yet) of using it though.
--
John

Re: Looking for advice on tools (or libraries) for unsupervised, bulk symmetric encryption/decryption of files

[Celejar]

On Fri, 11 Jan 2019 21:45:57 +
Jonathan Dowland  wrote:

> On Wed, Jan 09, 2019 at 10:18:47PM -0500, Celejar wrote:
> >The standard encryption technology for linux is LUKS. It works on the
> >block device level, not the file level.
> 
> LUKS would be no good if the user wants to move/copy/share the encrypted
> files, encrypted, elsewhere: they didn't say so explicitly but that's
> the impression I got reading their message.

You're probably right; I realized after I wrote my reply (and read some
of the other replies) that my solutions likely weren't really what the
OP was looking for.

> >I believe that the most commonly used software for file level
> >encryption is EncFS. I haven't really used it much, and can't speak to
> >its long term stablity.
> 
> EncFS should not be used for any new file encryption project, IMHO.
> There was the following report in 2014:
> https://defuse.ca/audits/encfs.htm
> This is referenced in the NEWS file in the EncFS package
> https://salsa.debian.org/debian/encfs/blob/debian/sid/debian/NEWS
> 
> Both the report and the NEWS file are 5 years sold so I am not sure of
> its current status but I'd want to seek positive assurance.

Huh - good to know. But I was wondering, along similar (but less
informed) lines, how good some of the other suggestions were, e.g.
ccrypt. I know very little about ccrypt, but has it even been
audited at all? Is it sufficiently widely used that any vulnerablities
or misimplementations of the sort discovered by the EncFS audit would
have been looked for or turned up?

Celejar

Re: Looking for advice on tools (or libraries) for unsupervised, bulk symmetric encryption/decryption of files

[Stefan Pietsch]

On 11.01.19 22:45, Jonathan Dowland wrote:

> EncFS should not be used for any new file encryption project, IMHO.
> There was the following report in 2014:
> https://defuse.ca/audits/encfs.htm
> This is referenced in the NEWS file in the EncFS package
> https://salsa.debian.org/debian/encfs/blob/debian/sid/debian/NEWS
> 
> Both the report and the NEWS file are 5 years sold so I am not sure of
> its current status but I'd want to seek positive assurance.

See GitHub issue #314 that lists the open security issues in EncFS:
https://github.com/vgough/encfs/issues/314

Re: Looking for advice on tools (or libraries) for unsupervised, bulk symmetric encryption/decryption of files

[Jonathan Dowland]

On Wed, Jan 09, 2019 at 10:18:47PM -0500, Celejar wrote:

The standard encryption technology for linux is LUKS. It works on the
block device level, not the file level.


LUKS would be no good if the user wants to move/copy/share the encrypted
files, encrypted, elsewhere: they didn't say so explicitly but that's
the impression I got reading their message.


I believe that the most commonly used software for file level
encryption is EncFS. I haven't really used it much, and can't speak to
its long term stablity.


EncFS should not be used for any new file encryption project, IMHO.
There was the following report in 2014:
https://defuse.ca/audits/encfs.htm
This is referenced in the NEWS file in the EncFS package
https://salsa.debian.org/debian/encfs/blob/debian/sid/debian/NEWS

Both the report and the NEWS file are 5 years sold so I am not sure of
its current status but I'd want to seek positive assurance.

--

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Jonathan Dowland
⢿⡄⠘⠷⠚⠋⠀ https://jmtd.net
⠈⠳⣄ Please do not CC me, I am subscribed to the list.

Re: Looking for advice on tools (or libraries) for unsupervised, bulk symmetric encryption/decryption of files

[Ben Caradoc-Davies]

On 10/01/2019 03:05, Kynn Jones wrote:

The only encryption tool I have used for encrypting files on my hard drive
is gpg2, which I have used for small, interactive encryption tasks
(half-dozen files, at most).
Therefore, my initial attempt was to use gpg2 for this new bulk-encryption
task, but I found myself constantly fighting with it, and finally had to
recognize that I was trying to use gpg2 for something it is not primarily
designed for.  (I am also a bit concerned with gpg2's future stability.
AFAICT, It's design has varied significantly over the years, and as a
result there's a lot of confusion on its use.  That has been my experience,
in any case.)


I use a pipe with gpg2 as one component for symmetric encryption:

gpg --batch --symmetric --cipher-algo AES256 --s2k-digest-algo SHA512 
--compress-algo none --passphrase-file $PASSPHRASE_FILE


My pipe input is usually a tar file gzipped with pigz for parallel 
compression, hence the "--compress-algo none". I then add another "pigz 
-0" wrapper to get a cryptographically weak checksum to allow testing 
for media failures without the passphrase. I like tar because it 
preserves file metadata and filesystem structure and is a very stable 
format. Other formats may be better for random access.


Recently I used gpg2 to decrypt files that were encrypted over 15 years 
ago; note that these were much smaller files and a simpler invocation of 
gpg1 (the then default cipher was CAST5 IIRC). The gpg file format seems 
well-documented and stable. Regular decryption tests are prudent to 
catch problems after gpg upgrade. Yes, the new interactive predilections 
of gpg2 were a pain at first when compared to gpg1, but "--batch" and 
"--passphrase-file" seem sufficient for batch symmetric encryption, if 
you do not mind your passphrase being in plain text on your filesystem.


Kind regards,

--
Ben Caradoc-Davies 
Director
Transient Software Limited 
New Zealand

Re: Looking for advice on tools (or libraries) for unsupervised, bulk symmetric encryption/decryption of files

[Celejar]

On Wed, 9 Jan 2019 09:05:32 -0500
Kynn Jones  wrote:

> I am looking for software to symmetric-encrypt large numbers of files on
> disk (terabytes' worth of data), and would appreciate some advice.
> 
> My basic requirements:
> 
>- It should be open source and no-cost (though, since I'm asking this
>question here, this goes without saying);
>- I should be able to program scripts (shell, Python, Perl, or Ruby) to
>run this software without human intervention; this rules out tools that are
>designed for interactive use.
>- It should be stable; I should be able to decrypt encrypted files that
>were encrypted several years earlier; (how much earlier?  hard to say;
>let's say 10 years, as a rough ballpark)
> 
> In addition, the following would be nice:
> 
>- good documentation;
>- good performance;
>- bindings for a high-level language (preferably Python).

The standard encryption technology for linux is LUKS. It works on the
block device level, not the file level. [I may be using the terminology
inaccurately.] You'll find the best compatibility, stability, and
documentation with LUKS, but you don't use it to create encrypted
copies of files - rather, you create a LUKS encrypted device, and
copy / move your files there. The LUKS tools (cryptsetup) are only
necessary for the creation and management of the encrypted volume; once
it's set up, you use the ondinary filesystem tools (cp, mv, ls, etc.)
to access files and move them to and from encrypted storage.

I believe that the most commonly used software for file level
encryption is EncFS. I haven't really used it much, and can't speak to
its long term stablity.



Celejar

Re: Looking for advice on tools (or libraries) for unsupervised, bulk symmetric encryption/decryption of files

[Linux-Fan]

Kynn Jones writes:

I am looking for software to symmetric-encrypt large numbers of files on disk  
(terabytes' worth of data), and would appreciate some advice.


My basic requirements:

• It should be open source and no-cost (though, since I'm asking this  
question here, this goes without saying);


• I should be able to program scripts (shell, Python, Perl, or Ruby) to run  
this software without human intervention; this rules out tools that are  
designed for interactive use.


• It should be stable; I should be able to decrypt encrypted files that were  
encrypted several years earlier; (how much earlier?  hard to say; let's say  
10 years, as a rough ballpark)

In addition, the following would be nice:

• good documentation;

• good performance;

• bindings for a high-level language (preferably Python).

The only encryption tool I have used for encrypting files on my hard drive is  
gpg2, which I have used for small, interactive encryption tasks (half-dozen  
files, at most).


[...]


What Debian packages would you recommend?


I actually tend to use 7-Zip for symmetric file encryption a lot because it  
ensures cross-platform compatibility and many users have 7-Zip already  
installed (I am always afraid that one day decryption software might not be  
available because then data would be close to being lost). The Debian  
package is `p7zip-full`.


Another program which I like is AESCrypt. Unfortunately it seems it is not  
included in Debian, but one might be able to install it via PIP (I have only  
ever used the Java and C version). I have actually read the Java  
implementation (which is also available as a very minimalistic commandline  
utility) and found the code understandable which is always a bonus when it  
comes to security :) Additionally, it was simple to adapt the library to  
provide a slightly different API and the result is still compatible with the  
AESCrypt commandline. As a result, you can use the API to automatically  
process data and the commandline utility to manually extract the data should  
something go wrong with the automatism. AESCrypt also seems to have Python  
bindings (but I have not used them). See https://www.aescrypt.com.


HTH
Linux-Fan


pgpXbZBrkpsDQ.pgp
Description: PGP signature

Re: Looking for advice on tools (or libraries) for unsupervised, bulk symmetric encryption/decryption of files

[David Christensen]

On 1/9/19 6:05 AM, Kynn Jones wrote:

I am looking for software to symmetric-encrypt large numbers of files on
disk (terabytes' worth of data), and would appreciate some advice.

My basic requirements:

- It should be open source and no-cost (though, since I'm asking this
question here, this goes without saying);
- I should be able to program scripts (shell, Python, Perl, or Ruby) to
run this software without human intervention; this rules out tools that are
designed for interactive use.
- It should be stable; I should be able to decrypt encrypted files that
were encrypted several years earlier; (how much earlier?  hard to say;
let's say 10 years, as a rough ballpark)

In addition, the following would be nice:

- good documentation;
- good performance;
- bindings for a high-level language (preferably Python).

The only encryption tool I have used for encrypting files on my hard drive
is gpg2, which I have used for small, interactive encryption tasks
(half-dozen files, at most).

Therefore, my initial attempt was to use gpg2 for this new bulk-encryption
task, but I found myself constantly fighting with it, and finally had to
recognize that I was trying to use gpg2 for something it is not primarily
designed for.  (I am also a bit concerned with gpg2's future stability.
AFAICT, It's design has varied significantly over the years, and as a
result there's a lot of confusion on its use.  That has been my experience,
in any case.)

So I am back to square one.

I stress that I am interested only in symmetric encryption.  The issues
that asymmetric encryption addresses are not at all part of the problem I
am dealing with, and therefore I don't want to have to deal with the
complexities of asymmetric encryption.

What Debian packages would you recommend?

Many thanks in advance!

kj


I use the ccrypt(1) suite for encrypting files using only a password 
(e.g. not PKI).  I wrap the CLI tools with Bash and/or Perl scripts to 
automate repetitive tasks (Python, Ruby, and other scripting languages 
should also work).  It is available via the Debian package 'ccrypt'.


http://ccrypt.sourceforge.net/


David

Looking for advice on tools (or libraries) for unsupervised, bulk symmetric encryption/decryption of files

[Kynn Jones]

I am looking for software to symmetric-encrypt large numbers of files on
disk (terabytes' worth of data), and would appreciate some advice.

My basic requirements:

   - It should be open source and no-cost (though, since I'm asking this
   question here, this goes without saying);
   - I should be able to program scripts (shell, Python, Perl, or Ruby) to
   run this software without human intervention; this rules out tools that are
   designed for interactive use.
   - It should be stable; I should be able to decrypt encrypted files that
   were encrypted several years earlier; (how much earlier?  hard to say;
   let's say 10 years, as a rough ballpark)

In addition, the following would be nice:

   - good documentation;
   - good performance;
   - bindings for a high-level language (preferably Python).

The only encryption tool I have used for encrypting files on my hard drive
is gpg2, which I have used for small, interactive encryption tasks
(half-dozen files, at most).

Therefore, my initial attempt was to use gpg2 for this new bulk-encryption
task, but I found myself constantly fighting with it, and finally had to
recognize that I was trying to use gpg2 for something it is not primarily
designed for.  (I am also a bit concerned with gpg2's future stability.
AFAICT, It's design has varied significantly over the years, and as a
result there's a lot of confusion on its use.  That has been my experience,
in any case.)

So I am back to square one.

I stress that I am interested only in symmetric encryption.  The issues
that asymmetric encryption addresses are not at all part of the problem I
am dealing with, and therefore I don't want to have to deal with the
complexities of asymmetric encryption.

What Debian packages would you recommend?

Many thanks in advance!

kj