Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread robert engels
I think you misunderstood. If you do not have encrypted swap (like OSX provides for) then you encryption is pointless as anyone can inspect the data as it it loaded into the heap by lucene - bypassing the encryption. I also think you underestimated the impact on the size of the indexes, a

[jira] Updated: (LUCENE-737) Provision of encryption/decryption services API to support Field.Store.Encrypted

2006-12-01 Thread victor negrin (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-737?page=all ] victor negrin updated LUCENE-737: - Attachment: BouncyCastle Licence & disclaimer BouncyCastle Licence, to be read before using the BouncyCastle .jar file > Provision of encryption/decryption s

[jira] Commented: (LUCENE-707) Lucene Java Site docs

2006-12-01 Thread Marvin Humphrey (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-707?page=comments#action_12455076 ] Marvin Humphrey commented on LUCENE-707: One thing you might want to fix is the breadcrumb trail. The javascript that generates it automatically inserts a

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Chris Hostetter
: dismay I noticed that JIRA assigned licence to the ASF for the provider : software. something which I did not intend and which cannot be valid. Can it : be reversed please?) the flag can't be modified, but attachments can be deleted, which i have done for the jar in question. feel free to rea

[jira] Commented: (LUCENE-737) Provision of encryption/decryption services API to support Field.Store.Encrypted

2006-12-01 Thread Hoss Man (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-737?page=comments#action_12455070 ] Hoss Man commented on LUCENE-737: - deleted encryption provider jar which mistakenly had apache liscene "ok" info set on it per the request of the issue submitter

[jira] Updated: (LUCENE-737) Provision of encryption/decryption services API to support Field.Store.Encrypted

2006-12-01 Thread Hoss Man (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-737?page=all ] Hoss Man updated LUCENE-737: Attachment: (was: bcprov-jdk14-133.jar) > Provision of encryption/decryption services API to support > Field.Store.Encrypted >

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Chris Hostetter
: That is a valid consideration Doron, which brings the discussion back to the : difference between encrypton and security. I believe that security is an end : application responsability, not Lucene's. For instance, is it possible to : write the end application so that those stats are hidden from

Re: [jira] Commented: (LUCENE-707) Lucene Java Site docs

2006-12-01 Thread Doug Cutting
Grant Ingersoll wrote: Does forrest not copy unused images? All the images live under the xdocs/images directory properly in the source tree and I did a recursive copy from the build. No, it seems that Forrest does not copy unused images into the build. But projects probably shouldn't rely o

Re: [jira] Commented: (LUCENE-707) Lucene Java Site docs

2006-12-01 Thread Grant Ingersoll
Does forrest not copy unused images? All the images live under the xdocs/images directory properly in the source tree and I did a recursive copy from the build. -Grant On Dec 1, 2006, at 5:18 PM, Doug Cutting (JIRA) wrote: [ http://issues.apache.org/jira/browse/LUCENE-707? page=commen

[jira] Commented: (LUCENE-707) Lucene Java Site docs

2006-12-01 Thread Sami Siren (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-707?page=comments#action_12455045 ] Sami Siren commented on LUCENE-707: --- > We should probably do the same for Nutch. I'll do that > Lucene Java Site docs > - > >

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
I am delighted I am no longer a voice in the wilderness. I couldn't agree more with you Joaquin! Victor Joaquin Delgado-2 wrote: > > Security should be responsibility of the application. However let's make > it clear that field level encryption is more a "means of" implementing > security and he

[jira] Commented: (LUCENE-707) Lucene Java Site docs

2006-12-01 Thread Doug Cutting (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-707?page=comments#action_12455020 ] Doug Cutting commented on LUCENE-707: - Lucene Java, Hadoop & Nutch all pointed to the image: http://lucene.apache.org/java/docs/images/lucene_green_150.gif wh

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
Otis, I have raised JIRA Lucene-737. Please note a couple of points. The text of the issue is a cut and paste of my original posting, which did not include the encryption provider software, and attached the Lucene modifications in diff form. The modifications attached to the JIRA are not in diff f

[jira] Created: (LUCENE-737) Provision of encryption/decryption services API to support Field.Store.Encrypted

2006-12-01 Thread victor negrin (JIRA)
Provision of encryption/decryption services API to support Field.Store.Encrypted Key: LUCENE-737 URL: http://issues.apache.org/jira/browse/LUCENE-737 Project: Lucene - Ja

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread JOAQUIN DELGADO
Security should be responsibility of the application. However let's make it clear that field level encryption is more a "means of" implementing security and herefore an infrastructure functionality that in my opinion Lucene should optionally provide. In the same way relational databases provide

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
That is a valid consideration Doron, which brings the discussion back to the difference between encrypton and security. I believe that security is an end application responsability, not Lucene's. For instance, is it possible to write the end application so that those stats are hidden from or inacc

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Erik Hatcher
Cryptonomicon, anyone?! On Dec 1, 2006, at 1:02 PM, Doron Cohen wrote: Robert Engels <[EMAIL PROTECTED]> wrote on 01/12/2006 09:34:12: ... decrypting such small payloads .. I think it is also subject to an easy attack, In addition, index statistics are still available, right? So one can

Re: Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Mike Klaas
On 12/1/06, negrinv <[EMAIL PROTECTED]> wrote: I think we should not make too many assumptions about performance until we can test alternative solutions. <> The small payload overhead will be amply offset in my opinion by the ability to be very selective about what is being encrypted, as opp

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Otis Gospodnetic
Victor, I haven't looked at your patch/ZIP. It would be best to attach to a new JIRA issue. That will be easier for others to look at (I already don't have the email with your ZIP, for example). Also, a patch is strngly preferred if you've made changes to existing code. If you can't do

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
Good news for OSX users! but what about all the others, should I say the majority?? One more reason for encrypting at field level. Victor Robert Engels wrote: > > Not if running under OSX with encrypted swap turned on ! :) > > -Original Message- >>From: Nicolas Lalev�e <[EMAIL PROTECT

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
I think we should not make too many assumptions about performance until we can test alternative solutions. The small payload overhead will be amply offset in my opinion by the ability to be very selective about what is being encrypted, as opposed to wholesale encryption and decryption. Also we sho

[jira] Commented: (LUCENE-736) Sloppy Phrase Scoring Misbehavior

2006-12-01 Thread Paul Elschot (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-736?page=comments#action_12454971 ] Paul Elschot commented on LUCENE-736: - I have had similar concerns when I implemented NearSpansOrdered.java and NearSpansUnordered.java, which are in the trunk

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Doron Cohen
Robert Engels <[EMAIL PROTECTED]> wrote on 01/12/2006 09:34:12: > ... decrypting such small payloads .. I think it is also subject to an easy attack, In addition, index statistics are still available, right? So one can know how many docs, which (encrypted) words appear in which docs and exactly w

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Robert Engels
Not if running under OSX with encrypted swap turned on ! :) -Original Message- >From: Nicolas Lalev�e <[EMAIL PROTECTED]> >Sent: Dec 1, 2006 4:49 AM >To: java-dev@lucene.apache.org >Subject: Re: Attached proposed modifications to Lucene 2.0 to support >Field.Store.Encrypted > >Le Vendredi

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Robert Engels
I agree with Nicolas. I think the overhead of decrypting such small payloads (I think it is also subject to an easy attack, and/or will increase index size dramtically in order to prevent such small encryption blocks) will have a serious impact on performance. We use Lucene for indexing only a

[jira] Commented: (LUCENE-736) Sloppy Phrase Scoring Misbehavior

2006-12-01 Thread Yonik Seeley (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-736?page=comments#action_12454955 ] Yonik Seeley commented on LUCENE-736: - Great investigations Doron! Personally I'm more concerned with (1) than (2). Was the fix for one issue more responsible

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Nicolas Lalevée
Le Vendredi 1 Décembre 2006 11:10, negrinv a écrit : > Nicolas Lalevée-2 wrote: > > Le Vendredi 1 Décembre 2006 01:33, negrinv a écrit : > >> Thank you Robert for your commnets. I am inclined to agree with you, but > >> I > >> would like to establish first of all if simplicity of implementation is

[jira] Updated: (LUCENE-736) Sloppy Phrase Scoring Misbehavior

2006-12-01 Thread Doron Cohen (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-736?page=all ] Doron Cohen updated LUCENE-736: --- Attachment: sloppy_phrase_java.patch.txt perf-search-new.log perf-search-orig.log Attached sloppy_phrase_java.patch.txt is fixing

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread negrinv
Nicolas Lalevée-2 wrote: > > Le Vendredi 1 Décembre 2006 01:33, negrinv a écrit : >> Thank you Robert for your commnets. I am inclined to agree with you, but >> I >> would like to establish first of all if simplicity of implementation is >> the >> overriding consideration. But before I dwell on

[jira] Updated: (LUCENE-736) Sloppy Phrase Scoring Misbehavior

2006-12-01 Thread Doron Cohen (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-736?page=all ] Doron Cohen updated LUCENE-736: --- Attachment: sloppy_phrase_tests.patch.txt sloppy_phrase_tests.patch.txt contains: - two test cases added in TestPhraseQuery. These new tests currently fail.

[jira] Commented: (LUCENE-697) Scorer.skipTo affects sloppyPhrase scoring

2006-12-01 Thread Doron Cohen (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-697?page=comments#action_12454844 ] Doron Cohen commented on LUCENE-697: I went on documenting sloppy phrase scorer and phrase scorer, so that the fix above can make it in more comfortably. Howev

[jira] Created: (LUCENE-736) Sloppy Phrase Scoring Misbehavior

2006-12-01 Thread Doron Cohen (JIRA)
Sloppy Phrase Scoring Misbehavior - Key: LUCENE-736 URL: http://issues.apache.org/jira/browse/LUCENE-736 Project: Lucene - Java Issue Type: Bug Components: Search Reporter: Doron Cohen

Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted

2006-12-01 Thread Nicolas Lalevée
Le Vendredi 1 Décembre 2006 01:33, negrinv a écrit : > Thank you Robert for your commnets. I am inclined to agree with you, but I > would like to establish first of all if simplicity of implementation is the > overriding consideration. But before I dwell on that let me say that i have > discovered