Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Doug Cutting
negrinv wrote: there is a third way Doug, and it's for me to stop trying to be polite by answering all the questions that I am being asked, then nobody will get upset by my replies. If the decision is for no encryption at field level, I accept it, but I don't believe it should be externalised. P

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Michael Busch
negrinv wrote: there is a third way Doug, and it's for me to stop trying to be polite by answering all the questions that I am being asked, then nobody will get upset by my replies. If the decision is for no encryption at field level, I accept it, but I don't believe it should be externalised. P

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread negrinv
there is a third way Doug, and it's for me to stop trying to be polite by answering all the questions that I am being asked, then nobody will get upset by my replies. If the decision is for no encryption at field level, I accept it, but I don't believe it should be externalised. Perhaps someone e

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread negrinv
there is absolutely no suggestion to make any changes to the index format. the index format would not change, whether you use encryption or not. Chuck Williams-2 wrote: > > > > Mike Klaas wrote on 12/05/2006 11:38 AM: >> On 12/5/06, negrinv <[EMAIL PROTECTED]> wrote: >> >>> Chris Hostetter wr

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Chuck Williams
Mike Klaas wrote on 12/05/2006 11:38 AM: > On 12/5/06, negrinv <[EMAIL PROTECTED]> wrote: > >> Chris Hostetter wrote: > >> > If the code was not already in the core, and someone asked about >> adding it >> > I would argue against doing so on the grounds that some helpfull >> utility >> > methods

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Doug Cutting
Doug Cutting wrote: So, Victor, do you think this functionality could be reasonably packaged as an add-on package to Lucene? Doug, for an answer to most of your questions could you please refer to my answer to Chris Hostetter [ ... ] Let me be more direct. Encryption of Lucene fields may be

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread negrinv
Doug Cutting wrote: > > > Some utilities for encrypting and decrypting binary fields might make a > useful contrib module, but I see no compelling reason to add this to the > core at this point. If such a contrib module becomes widely used, and > it becomes clear that it would work better i

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread robert engels
If it is only meant to protect from "prying eyes" a simple field level analyzer that does a simple xor/rotation should suffice. It will be much faster and simpler. Going beyond that, your solution is not very secure as has been pointed out, so you might as well just uses the simplest soluti

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Mike Klaas
On 12/5/06, negrinv <[EMAIL PROTECTED]> wrote: Chris Hostetter wrote: > If the code was not already in the core, and someone asked about adding it > I would argue against doing so on the grounds that some helpfull utility > methods (possibly in a contrib) would be just as usefull, and would h

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread negrinv
Chris Hostetter wrote: > > > Compression of stored fields is a feature that the Lucene "core" currently > supports out of the box -- but it does so in a very limited maner that > doesn't allow for much configuration. There is no advantage for users in > using compressed fields over compressing

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-05 Thread Doug Cutting
robert engels wrote: I would counter that it is not a compelling feature for MOST users of Lucene, but it can still be implemented externally using binary fields for those that require it, and or even easier (and maybe even faster) using a encrypted filesystem with proper security. +1 Some u

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-04 Thread Chris Hostetter
(For the record: I have delibierately avoided looking at your patch so far, because i didn't want my opinion on the question of "should Lucene offer encryption services" to be clouded by any specifics of your implimentation. That said...) As it's already been pointed out, an apples to apples com

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-02 Thread robert engels
-- From: Nicolas Lalev�e <[EMAIL PROTECTED]> Sent: Dec 1, 2006 4:49 AM To: java-dev@lucene.apache.org Subject: Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted Le Vendredi 1 D�cembre 2006 11:10, negrinv a �crit�: Nicolas Lalev�e-2 wrote: Le Vendredi 1 D�cem

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-02 Thread negrinv
gt; Victor >> >> >> Robert Engels wrote: >>> >>> Not if running under OSX with encrypted swap turned on ! :) >>> >>> -----Original Message- >>>> From: Nicolas Lalev�e <[EMAIL PROTECTED]> >>>> Sent: Dec 1, 2006 4:49 AM

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread robert engels
ct: Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted Le Vendredi 1 D�cembre 2006 11:10, negrinv a �crit�: Nicolas Lalev�e-2 wrote: Le Vendredi 1 D�cembre 2006 01:33, negrinv a �crit : Thank you Robert for your commnets. I am inclined to agree with you, but I

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Chris Hostetter
: dismay I noticed that JIRA assigned licence to the ASF for the provider : software. something which I did not intend and which cannot be valid. Can it : be reversed please?) the flag can't be modified, but attachments can be deleted, which i have done for the jar in question. feel free to rea

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Chris Hostetter
: That is a valid consideration Doron, which brings the discussion back to the : difference between encrypton and security. I believe that security is an end : application responsability, not Lucene's. For instance, is it possible to : write the end application so that those stats are hidden from

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
> From negrinv <[EMAIL PROTECTED]> > Sent Fri 12/1/2006 1:22 PM > To java-dev@lucene.apache.org > Subject Re: Attached proposed modifications to Lucene 2.0 to support > Field.Store.Encrypted > > > That is a valid consideration Doron, which brings the discuss

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
n't do it > from Eclipse, do it from the command-line: svn diff src, or some such. > > Otis > > - Original Message > From: negrinv <[EMAIL PROTECTED]> > To: java-dev@lucene.apache.org > Sent: Friday, December 1, 2006 5:10:47 AM > Subject: Re: Attach

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread JOAQUIN DELGADO
ple ;-) That's my two cents. -- Joaquin -Original Message- >From negrinv <[EMAIL PROTECTED]> Sent Fri 12/1/2006 1:22 PM To java-dev@lucene.apache.org Subject Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted That is a valid consider

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
That is a valid consideration Doron, which brings the discussion back to the difference between encrypton and security. I believe that security is an end application responsability, not Lucene's. For instance, is it possible to write the end application so that those stats are hidden from or inacc

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Erik Hatcher
Cryptonomicon, anyone?! On Dec 1, 2006, at 1:02 PM, Doron Cohen wrote: Robert Engels <[EMAIL PROTECTED]> wrote on 01/12/2006 09:34:12: ... decrypting such small payloads .. I think it is also subject to an easy attack, In addition, index statistics are still available, right? So one can

Re: Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Mike Klaas
On 12/1/06, negrinv <[EMAIL PROTECTED]> wrote: I think we should not make too many assumptions about performance until we can test alternative solutions. <> The small payload overhead will be amply offset in my opinion by the ability to be very selective about what is being encrypted, as opp

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Otis Gospodnetic
If you can't do it from Eclipse, do it from the command-line: svn diff src, or some such. Otis - Original Message From: negrinv <[EMAIL PROTECTED]> To: java-dev@lucene.apache.org Sent: Friday, December 1, 2006 5:10:47 AM Subject: Re: Attached proposed modification

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
as Lalev�e <[EMAIL PROTECTED]> >>Sent: Dec 1, 2006 4:49 AM >>To: java-dev@lucene.apache.org >>Subject: Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted >> >>Le Vendredi 1 D�cembre 2006 11:10, negrinv a �crit�: >>> Nicolas Lale

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
-Original Message- >>From: Nicolas Lalev�e <[EMAIL PROTECTED]> >>Sent: Dec 1, 2006 2:20 AM >>To: java-dev@lucene.apache.org >>Subject: Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted >> >>Le Vendredi 1 D�cembre 2006 01:33,

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Doron Cohen
Robert Engels <[EMAIL PROTECTED]> wrote on 01/12/2006 09:34:12: > ... decrypting such small payloads .. I think it is also subject to an easy attack, In addition, index statistics are still available, right? So one can know how many docs, which (encrypted) words appear in which docs and exactly w

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Robert Engels
Not if running under OSX with encrypted swap turned on ! :) -Original Message- >From: Nicolas Lalev�e <[EMAIL PROTECTED]> >Sent: Dec 1, 2006 4:49 AM >To: java-dev@lucene.apache.org >Subject: Re: Attached proposed modifications to Lucene 2.0 to support >Field.

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Robert Engels
and store the actual payloads elsewhere, so in our case your solution is not optimal for us. -Original Message- >From: Nicolas Lalev�e <[EMAIL PROTECTED]> >Sent: Dec 1, 2006 2:20 AM >To: java-dev@lucene.apache.org >Subject: Re: Attached proposed modifications to Luc

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Nicolas Lalevée
Le Vendredi 1 Décembre 2006 11:10, negrinv a écrit : > Nicolas Lalevée-2 wrote: > > Le Vendredi 1 Décembre 2006 01:33, negrinv a écrit : > >> Thank you Robert for your commnets. I am inclined to agree with you, but > >> I > >> would like to establish first of all if simplicity of implementation is

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
Nicolas Lalevée-2 wrote: > > Le Vendredi 1 Décembre 2006 01:33, negrinv a écrit : >> Thank you Robert for your commnets. I am inclined to agree with you, but >> I >> would like to establish first of all if simplicity of implementation is >> the >> overriding consideration. But before I dwell on

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Nicolas Lalevée
rectory. > > > > Far simpler, and if yuou are using encryption to begin with, you are > > probably encrypting most of the data anyway. > > > > -Original Message- > > > >>From: negrinv <[EMAIL PROTECTED]> > >>Sent: Nov 29, 2006 9:4

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-11-30 Thread negrinv
Luke, I should have mentioned in my earlier posting that what I am proposing uses password based encrytpion, where the password is NOT stored anywhere within Lucene. I avoided on purpose to make any references to security (as opposed to encryption) because I believe security to be the responsabi

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-11-30 Thread negrinv
egrinv <[EMAIL PROTECTED]> >>Sent: Nov 29, 2006 9:45 PM >>To: java-dev@lucene.apache.org >>Subject: Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted >> >> >>Thank you Luke for your comments and the references you supplied.

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-11-30 Thread Robert Engels
- >From: negrinv <[EMAIL PROTECTED]> >Sent: Nov 29, 2006 9:45 PM >To: java-dev@lucene.apache.org >Subject: Re: Attached proposed modifications to Lucene 2.0 to support >Field.Store.Encrypted > > >Thank you Luke for your comments and the references you supplied. I read

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-11-30 Thread Robert Engels
Agreed. -Original Message- >From: Luke Nezda <[EMAIL PROTECTED]> >Sent: Nov 29, 2006 8:30 PM >To: java-dev@lucene.apache.org >Subject: Re: Attached proposed modifications to Lucene 2.0 to support >Field.Store.Encrypted > >I think that adding encryption suppor

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-11-30 Thread Luke Nezda
Victor- Your point is well taken that a comprehensive encryption strategy is not quite analogous to compression which is involves more than a transformation of field values to a more compact form since it requires (at a minimum) all data structures which comprise the index be encrypted too. Maybe

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-11-29 Thread negrinv
Thank you Luke for your comments and the references you supplied. I read through them and reached the following conclusions. There seems to be a philosophical issue about the boundary between a user application and the Lucene API, where should one start and the other stop. The other issue is the s

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-11-29 Thread Luke Nezda
I think that adding encryption support to Lucene fields is a bad idea for the same reasons adding compression was a bad idea (conclusive comments on the tail of this issue http://issues.apache.org/jira/browse/LUCENE-648?page=all). Binary fields can be used by users to achieve this end. Maybe a