Re: Solrj possible deadlock

2009-09-24 Thread pof

Well, in the same processes I am using a jdbc connection to get all the
relative paths to the documents I want to index, then I parse the documents
to plain text using tones of open source libraries like POI, PFDBox
etc.(which might account for java2d) then I add them to the index and commit
every 2000 documents.

I write a db row for each row I index so I can resume where I left off after
a crash or exception. My app will happily index for hours then after it
hangs, the resumed indexing run will only last a few additional minutes! The
thread dumps look the same.

Cheers.


ryantxu wrote:
 
 do you have anything custom going on?
 
 The fact that the lock is in java2d seems suspicious...
 
 
 On Sep 23, 2009, at 7:01 PM, pof wrote:
 

 I had the same problem again yesterday except the process halted  
 after about
 20mins this time.


 pof wrote:

 Hello, I was running a batch index the other day using the Solrj
 EmbeddedSolrServer when the process abruptly froze in it's tracks  
 after
 running for about 4-5 hours and indexing ~400K documents. There  
 were no
 document locks so it would seem likely that there was some kind of  
 thread
 deadlock. I was hoping someone might be able to tell me some  
 information
 about the following thread dump taken at the time:

 Full thread dump OpenJDK Client VM (1.6.0-b09 mixed mode):

 DestroyJavaVM prio=10 tid=0x9322a800 nid=0xcef waiting on condition
 [0x..0x0018a044]
   java.lang.Thread.State: RUNNABLE

 Java2D Disposer daemon prio=10 tid=0x0a28cc00 nid=0xf1c in  
 Object.wait()
 [0x0311d000..0x0311def4]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0x97a96840 (a java.lang.ref.ReferenceQueue 
 $Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: 
 133)
- locked 0x97a96840 (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: 
 149)
at sun.java2d.Disposer.run(Disposer.java:143)
at java.lang.Thread.run(Thread.java:636)

 pool-1-thread-1 prio=10 tid=0x93a26c00 nid=0xcf7 waiting on  
 condition
 [0x08a6a000..0x08a6b074]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  0x967acfd0 (a
 java.util.concurrent.locks.AbstractQueuedSynchronizer 
 $ConditionObject)
at
 java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
 java.util.concurrent.locks.AbstractQueuedSynchronizer 
 $ConditionObject.await(AbstractQueuedSynchronizer.java:1978)
at
 java 
 .util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java: 
 386)
at
 java 
 .util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java: 
 1043)
at
 java 
 .util 
 .concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: 
 1103)
at
 java.util.concurrent.ThreadPoolExecutor 
 $Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)

 Low Memory Detector daemon prio=10 tid=0x93a00c00 nid=0xcf5  
 runnable
 [0x..0x]
   java.lang.Thread.State: RUNNABLE

 CompilerThread0 daemon prio=10 tid=0x09fe9800 nid=0xcf4 waiting on
 condition [0x..0x096a7af4]
   java.lang.Thread.State: RUNNABLE

 Signal Dispatcher daemon prio=10 tid=0x09fe8800 nid=0xcf3 waiting  
 on
 condition [0x..0x]
   java.lang.Thread.State: RUNNABLE

 Finalizer daemon prio=10 tid=0x09fd7000 nid=0xcf2 in Object.wait()
 [0x005ca000..0x005caef4]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0x966e6d40 (a java.lang.ref.ReferenceQueue 
 $Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: 
 133)
- locked 0x966e6d40 (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java: 
 149)
at java.lang.ref.Finalizer 
 $FinalizerThread.run(Finalizer.java:177)

 Reference Handler daemon prio=10 tid=0x09fd2c00 nid=0xcf1 in
 Object.wait() [0x00579000..0x00579d74]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0x966e6dc8 (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at
 java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked 0x966e6dc8 (a java.lang.ref.Reference$Lock)

 VM Thread prio=10 tid=0x09fcf800 nid=0xcf0 runnable

 VM Periodic Task Thread prio=10 tid=0x93a02400 nid=0xcf6 waiting on
 condition

 JNI global references: 1072

 Heap
 def new generation   total 36288K, used 23695K [0x93f1,  
 0x9667,
 0x9667)
  eden space 32256K,  73% used [0x93f1, 0x95633f60, 0x95e9)
  from space 4032K,   0% used [0x95e9, 0x95e9, 0x9628)
  to   space 4032K,   0% used [0x9628, 0x9628, 0x9667)
 tenured generation   total 483968K, used 72129K

Re: Solrj possible deadlock

2009-09-23 Thread pof

I had the same problem again yesterday except the process halted after about
20mins this time. 


pof wrote:
 
 Hello, I was running a batch index the other day using the Solrj
 EmbeddedSolrServer when the process abruptly froze in it's tracks after
 running for about 4-5 hours and indexing ~400K documents. There were no
 document locks so it would seem likely that there was some kind of thread
 deadlock. I was hoping someone might be able to tell me some information
 about the following thread dump taken at the time:
 
 Full thread dump OpenJDK Client VM (1.6.0-b09 mixed mode):
 
 DestroyJavaVM prio=10 tid=0x9322a800 nid=0xcef waiting on condition
 [0x..0x0018a044]
java.lang.Thread.State: RUNNABLE
 
 Java2D Disposer daemon prio=10 tid=0x0a28cc00 nid=0xf1c in Object.wait()
 [0x0311d000..0x0311def4]
java.lang.Thread.State: WAITING (on object monitor)
 at java.lang.Object.wait(Native Method)
 - waiting on 0x97a96840 (a java.lang.ref.ReferenceQueue$Lock)
 at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
 - locked 0x97a96840 (a java.lang.ref.ReferenceQueue$Lock)
 at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
 at sun.java2d.Disposer.run(Disposer.java:143)
 at java.lang.Thread.run(Thread.java:636)
 
 pool-1-thread-1 prio=10 tid=0x93a26c00 nid=0xcf7 waiting on condition
 [0x08a6a000..0x08a6b074]
java.lang.Thread.State: WAITING (parking)
 at sun.misc.Unsafe.park(Native Method)
 - parking to wait for  0x967acfd0 (a
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
 at
 java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
 at
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1978)
 at
 java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
 at
 java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
 at
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
 at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
 at java.lang.Thread.run(Thread.java:636)
 
 Low Memory Detector daemon prio=10 tid=0x93a00c00 nid=0xcf5 runnable
 [0x..0x]
java.lang.Thread.State: RUNNABLE
 
 CompilerThread0 daemon prio=10 tid=0x09fe9800 nid=0xcf4 waiting on
 condition [0x..0x096a7af4]
java.lang.Thread.State: RUNNABLE
 
 Signal Dispatcher daemon prio=10 tid=0x09fe8800 nid=0xcf3 waiting on
 condition [0x..0x]
java.lang.Thread.State: RUNNABLE
 
 Finalizer daemon prio=10 tid=0x09fd7000 nid=0xcf2 in Object.wait()
 [0x005ca000..0x005caef4]
java.lang.Thread.State: WAITING (on object monitor)
 at java.lang.Object.wait(Native Method)
 - waiting on 0x966e6d40 (a java.lang.ref.ReferenceQueue$Lock)
 at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
 - locked 0x966e6d40 (a java.lang.ref.ReferenceQueue$Lock)
 at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
 at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
 
 Reference Handler daemon prio=10 tid=0x09fd2c00 nid=0xcf1 in
 Object.wait() [0x00579000..0x00579d74]
java.lang.Thread.State: WAITING (on object monitor)
 at java.lang.Object.wait(Native Method)
 - waiting on 0x966e6dc8 (a java.lang.ref.Reference$Lock)
 at java.lang.Object.wait(Object.java:502)
 at
 java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
 - locked 0x966e6dc8 (a java.lang.ref.Reference$Lock)
 
 VM Thread prio=10 tid=0x09fcf800 nid=0xcf0 runnable
 
 VM Periodic Task Thread prio=10 tid=0x93a02400 nid=0xcf6 waiting on
 condition
 
 JNI global references: 1072
 
 Heap
  def new generation   total 36288K, used 23695K [0x93f1, 0x9667,
 0x9667)
   eden space 32256K,  73% used [0x93f1, 0x95633f60, 0x95e9)
   from space 4032K,   0% used [0x95e9, 0x95e9, 0x9628)
   to   space 4032K,   0% used [0x9628, 0x9628, 0x9667)
  tenured generation   total 483968K, used 72129K [0x9667, 0xb3f1,
 0xb3f1)
the space 483968K,  14% used [0x9667, 0x9ace04b8, 0x9ace0600,
 0xb3f1)
  compacting perm gen  total 23040K, used 22983K [0xb3f1, 0xb559,
 0xb7f1)
the space 23040K,  99% used [0xb3f1, 0xb5581ff8, 0xb5582000,
 0xb559)
 No shared spaces configured.
 
 Cheers. Brett.
 

-- 
View this message in context: 
http://www.nabble.com/Solrj-possible-deadlock-tp25530146p25531321.html
Sent from the Solr - User mailing list archive at Nabble.com.



Solrj possible deadlock

2009-09-20 Thread pof

Hello, I was running a batch index the other day using the Solrj
EmbeddedSolrServer when the process abruptly froze in it's tracks after
running for about 4-5 hours and indexing ~400K documents. There were no
document locks so it would seem likely that there was some kind of thread
deadlock. I was hoping someone might be able to tell me some information
about the following thread dump taken at the time:

Full thread dump OpenJDK Client VM (1.6.0-b09 mixed mode):

DestroyJavaVM prio=10 tid=0x9322a800 nid=0xcef waiting on condition
[0x..0x0018a044]
   java.lang.Thread.State: RUNNABLE

Java2D Disposer daemon prio=10 tid=0x0a28cc00 nid=0xf1c in Object.wait()
[0x0311d000..0x0311def4]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0x97a96840 (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
- locked 0x97a96840 (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
at sun.java2d.Disposer.run(Disposer.java:143)
at java.lang.Thread.run(Thread.java:636)

pool-1-thread-1 prio=10 tid=0x93a26c00 nid=0xcf7 waiting on condition
[0x08a6a000..0x08a6b074]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  0x967acfd0 (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1978)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)

Low Memory Detector daemon prio=10 tid=0x93a00c00 nid=0xcf5 runnable
[0x..0x]
   java.lang.Thread.State: RUNNABLE

CompilerThread0 daemon prio=10 tid=0x09fe9800 nid=0xcf4 waiting on
condition [0x..0x096a7af4]
   java.lang.Thread.State: RUNNABLE

Signal Dispatcher daemon prio=10 tid=0x09fe8800 nid=0xcf3 waiting on
condition [0x..0x]
   java.lang.Thread.State: RUNNABLE

Finalizer daemon prio=10 tid=0x09fd7000 nid=0xcf2 in Object.wait()
[0x005ca000..0x005caef4]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0x966e6d40 (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
- locked 0x966e6d40 (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)

Reference Handler daemon prio=10 tid=0x09fd2c00 nid=0xcf1 in Object.wait()
[0x00579000..0x00579d74]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0x966e6dc8 (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked 0x966e6dc8 (a java.lang.ref.Reference$Lock)

VM Thread prio=10 tid=0x09fcf800 nid=0xcf0 runnable

VM Periodic Task Thread prio=10 tid=0x93a02400 nid=0xcf6 waiting on
condition

JNI global references: 1072

Heap
 def new generation   total 36288K, used 23695K [0x93f1, 0x9667,
0x9667)
  eden space 32256K,  73% used [0x93f1, 0x95633f60, 0x95e9)
  from space 4032K,   0% used [0x95e9, 0x95e9, 0x9628)
  to   space 4032K,   0% used [0x9628, 0x9628, 0x9667)
 tenured generation   total 483968K, used 72129K [0x9667, 0xb3f1,
0xb3f1)
   the space 483968K,  14% used [0x9667, 0x9ace04b8, 0x9ace0600,
0xb3f1)
 compacting perm gen  total 23040K, used 22983K [0xb3f1, 0xb559,
0xb7f1)
   the space 23040K,  99% used [0xb3f1, 0xb5581ff8, 0xb5582000,
0xb559)
No shared spaces configured.

Cheers. Brett.
-- 
View this message in context: 
http://www.nabble.com/Solrj-possible-deadlock-tp25530146p25530146.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Regarding Response Builder

2009-07-21 Thread pof

I would just do something like this:

String myParam = req.getParams().get(xparam);

where xparam is:

http://localhost:8983/solr/select/?q=dogxparam=somethingstart=0rows=10indent=on


Kartik1 wrote:
 
 The responsebuiilder class has SolrQueryRequest as public type. Using
 SolrQueryRequest we can get a list of SolrParams like
 
 SolrParams params = req.getParams();
 
 Now I want to get the values of those params. What should be the approach
 as SolrParams is an abstract class and its get(String) method is abstract?
 
 Best regards,
 Amandeep Singh
 
 
 

-- 
View this message in context: 
http://www.nabble.com/Regarding-Response-Builder-tp24456722p24600481.html
Sent from the Solr - User mailing list archive at Nabble.com.



DocList Pagination

2009-07-20 Thread pof

Hi, I am try to get the next DocList page in my custom search component.
Could I get a code example of this?

Cheers.
-- 
View this message in context: 
http://www.nabble.com/DocList-Pagination-tp24581850p24581850.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Solr 1.4 Release Date

2009-07-13 Thread pof

Any updates on this?

Cheers.

Gurjot Singh wrote:
 
 Hi, I am curious to know when is the scheduled/tentative release date of
 Solr 1.4.
 
 Thanks,
 Gurjot
 
 

-- 
View this message in context: 
http://www.nabble.com/Solr-1.4-Release-Date-tp23260381p24473570.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Deleting from SolrQueryResponse

2009-07-02 Thread pof



hossman wrote:
 
 one thing to keep in mind however is that post-processing a DocList to 
 filter stuff out is almost never a good idea -- things get really 
 convoluted when you think about dealing with pagination and except for 
 some really trivial use cases you can never know what your upper bound 
 should be when deciding how many hits to request from underlying 
 IndexSearcher.  You're usually better off restructuring your problem so 
 that you can construct a Query/Filter/DocSet that you want to filter by 
 first and *then* executing the search to generate the DocList in a single 
 pass.
 
I was wanting to edit the DocList in a custom SearchComponent to be executed
after the QueryComponent. I do not require facetting etc. If do not want
facetted results, will I still need to take any special steps not to break
the doclist?

hossman wrote:
 
 PS: replying to your own message (or reposting it) to bump it up 
 generally doesn't encourage replies any faster -- it just increases the 
 volume of traffic on the list, and if anything antagonizes people and 
 makes them less interested in responding.
 
Okay, sorry I wasn't certain on the protocol on that.


Thanks, Brett.
-- 
View this message in context: 
http://www.nabble.com/Deleting-from-SolrQueryResponse-tp24266686p24315607.html
Sent from the Solr - User mailing list archive at Nabble.com.



Deleting from SolrQueryResponse

2009-06-30 Thread pof

Hi, I was wondering if any has had luck deleting added documents to
SolrQueryResponse? I am subclassing StandardRequestHandler and after I run
the handle request body method (super.handleRequestBody(req, rsp);) I won't
to filter out some of the hits.

Thanks. Brett
-- 
View this message in context: 
http://www.nabble.com/Deleting-from-SolrQueryResponse-tp24266686p24266686.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Deleting from SolrQueryResponse

2009-06-30 Thread pof

This thread got buried last night, so... bump.

pof wrote:
 
 Hi, I was wondering if any has had luck deleting added documents from
 SolrQueryResponse? I am subclassing StandardRequestHandler and after I run
 the handle request body method (super.handleRequestBody(req, rsp);) I want
 to filter out some of the hits.
 
 Thanks. Brett
 

-- 
View this message in context: 
http://www.nabble.com/Deleting-from-SolrQueryResponse-tp24266686p24281972.html
Sent from the Solr - User mailing list archive at Nabble.com.



Getting hits in RequestHandler

2009-06-29 Thread pof

Hi, I am writing my own request handler and I was wondering how I go about
get a list of hits back. Thanks.
-- 
View this message in context: 
http://www.nabble.com/Getting-hits-in-RequestHandler-tp24248810p24248810.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Solr document security

2009-06-28 Thread pof

I want to use filtering or similar, any help?


Otis Gospodnetic wrote:
 
 
 That URL to your Solr Admin page should never be exposed to the outside
 world.  You can play with network, routing, DNS and other similar things
 to make sure one can't get to this from the outside even if the URL is
 know.
 
  Otis
 --
 Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
 
 
 
 - Original Message 
 From: pof melbournebeerba...@gmail.com
 To: solr-user@lucene.apache.org
 Sent: Thursday, June 25, 2009 7:40:12 PM
 Subject: Re: Solr document security
 
 
 Thats what I was going to do originally, however what is stopping a user
 from
 simply running a search through http://localhost:8983/solr/admin/ of the
 index server?
 
 
 Norberto Meijome-6 wrote:
  
  On Wed, 24 Jun 2009 23:20:26 -0700 (PDT)
  pof wrote:
  
  
  Hi, I am wanting to add document-level security that works as
 following:
  An
  external process makes a query to the index, depending on their
 security
  allowences based of a login id a list of hits are returned minus any
 the
  user are meant to know even exist. I was thinking maybe a custom
 filter
  with
  a JDBC connection to check security of the user vs. the document. I'm
 not
  sure how I would add the filter or how to write the filter or how to
 get
  the
  login id from a GET parameter. Any suggestions, comments etc.?
  
  Hi Brett,
  (keeping in mind that i've been away from SOLR for 8 months, but i
  dont think this was added of late)
  
  standard approach is to manage security @ your
  application layer, not @ SOLR. ie, search, return documents (which
 should
  contain some kind of data to identify their ACL ) and then you can
 decide
  whether to show it or not. 
  
  HIH
  _
  {Beto|Norberto|Numard} Meijome
  
  They never open their mouths without subtracting from the sum of human
  knowledge. Thomas Brackett Reed
  
  I speak for myself, not my employer. Contents may be hot. Slippery when
  wet.
  Reading disclaimers makes you go blind. Writing them is worse. You have
  been
  Warned.
  
  
 
 -- 
 View this message in context: 
 http://www.nabble.com/Solr-document-security-tp24197620p24212752.html
 Sent from the Solr - User mailing list archive at Nabble.com.
 
 
 

-- 
View this message in context: 
http://www.nabble.com/Solr-document-security-tp24197620p24246624.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Solr document security

2009-06-28 Thread pof

The custom dispatcher filter example on
http://wiki.apache.org/solr/SolrSecurity seems to be what I need however I
don't understand the code. Could someone break it down for me? Thanks.

pof wrote:
 
 I want to use filtering or similar, any help?
 
 
 Otis Gospodnetic wrote:
 
 
 That URL to your Solr Admin page should never be exposed to the outside
 world.  You can play with network, routing, DNS and other similar things
 to make sure one can't get to this from the outside even if the URL is
 know.
 
  Otis
 --
 Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
 
 
 
 - Original Message 
 From: pof melbournebeerba...@gmail.com
 To: solr-user@lucene.apache.org
 Sent: Thursday, June 25, 2009 7:40:12 PM
 Subject: Re: Solr document security
 
 
 Thats what I was going to do originally, however what is stopping a user
 from
 simply running a search through http://localhost:8983/solr/admin/ of the
 index server?
 
 
 Norberto Meijome-6 wrote:
  
  On Wed, 24 Jun 2009 23:20:26 -0700 (PDT)
  pof wrote:
  
  
  Hi, I am wanting to add document-level security that works as
 following:
  An
  external process makes a query to the index, depending on their
 security
  allowences based of a login id a list of hits are returned minus any
 the
  user are meant to know even exist. I was thinking maybe a custom
 filter
  with
  a JDBC connection to check security of the user vs. the document. I'm
 not
  sure how I would add the filter or how to write the filter or how to
 get
  the
  login id from a GET parameter. Any suggestions, comments etc.?
  
  Hi Brett,
  (keeping in mind that i've been away from SOLR for 8 months, but i
  dont think this was added of late)
  
  standard approach is to manage security @ your
  application layer, not @ SOLR. ie, search, return documents (which
 should
  contain some kind of data to identify their ACL ) and then you can
 decide
  whether to show it or not. 
  
  HIH
  _
  {Beto|Norberto|Numard} Meijome
  
  They never open their mouths without subtracting from the sum of
 human
  knowledge. Thomas Brackett Reed
  
  I speak for myself, not my employer. Contents may be hot. Slippery
 when
  wet.
  Reading disclaimers makes you go blind. Writing them is worse. You
 have
  been
  Warned.
  
  
 
 -- 
 View this message in context: 
 http://www.nabble.com/Solr-document-security-tp24197620p24212752.html
 Sent from the Solr - User mailing list archive at Nabble.com.
 
 
 
 
 

-- 
View this message in context: 
http://www.nabble.com/Solr-document-security-tp24197620p24246685.html
Sent from the Solr - User mailing list archive at Nabble.com.



Solr document security

2009-06-25 Thread pof

Hi, I am wanting to add document-level security that works as following: An
external process makes a query to the index, depending on their security
allowences based of a login id a list of hits are returned minus any the
user are meant to know even exist. I was thinking maybe a custom filter with
a JDBC connection to check security of the user vs. the document. I'm not
sure how I would add the filter or how to write the filter or how to get the
login id from a GET parameter. Any suggestions, comments etc.?

Thanks. Brett. 
-- 
View this message in context: 
http://www.nabble.com/Solr-document-security-tp24197620p24197620.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Solr document security

2009-06-25 Thread pof

Thats what I was going to do originally, however what is stopping a user from
simply running a search through http://localhost:8983/solr/admin/ of the
index server?


Norberto Meijome-6 wrote:
 
 On Wed, 24 Jun 2009 23:20:26 -0700 (PDT)
 pof melbournebeerba...@gmail.com wrote:
 
 
 Hi, I am wanting to add document-level security that works as following:
 An
 external process makes a query to the index, depending on their security
 allowences based of a login id a list of hits are returned minus any the
 user are meant to know even exist. I was thinking maybe a custom filter
 with
 a JDBC connection to check security of the user vs. the document. I'm not
 sure how I would add the filter or how to write the filter or how to get
 the
 login id from a GET parameter. Any suggestions, comments etc.?
 
 Hi Brett,
 (keeping in mind that i've been away from SOLR for 8 months, but i
 dont think this was added of late)
 
 standard approach is to manage security @ your
 application layer, not @ SOLR. ie, search, return documents (which should
 contain some kind of data to identify their ACL ) and then you can decide
 whether to show it or not. 
 
 HIH
 _
 {Beto|Norberto|Numard} Meijome
 
 They never open their mouths without subtracting from the sum of human
 knowledge. Thomas Brackett Reed
 
 I speak for myself, not my employer. Contents may be hot. Slippery when
 wet.
 Reading disclaimers makes you go blind. Writing them is worse. You have
 been
 Warned.
 
 

-- 
View this message in context: 
http://www.nabble.com/Solr-document-security-tp24197620p24212752.html
Sent from the Solr - User mailing list archive at Nabble.com.



Solrj no search results

2009-06-24 Thread pof

Hi, I'm using an EmbeddedSolrServer. Adding documents with the example jetty
server example using this method worked fine:
doc1.addField( id, id1);
doc1.addField( name, doc1);
doc1.addField( price, 10);
server.add(doc1)

However now I have changed the schema.xml so I can use my own fields and the
documents will add to index (no compilation errors), however I will not get
any search results back what so ever. Any Ideas?

Thanks.
-- 
View this message in context: 
http://www.nabble.com/Solrj-no-search-results-tp24179484p24179484.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Solr Jetty confusion

2009-06-22 Thread pof

Hi, the problem was my xml configurations. I am now using a java process
(seperate from the webserver) using the EmbeddedSolrServer method and post a
commit to the webserver at the end to sync the two cores which seems to work
well for me. Thanks for the help.


Development Team wrote:
 
 Hi Brett,
  Well, I'm running Solr in Jetty with JBoss, so I used the JBoss
 method
 of specifying properties (properties-service.xml). However, you can supply
 the solr-home to the command-line when you start Jetty by using a
 parameter
 like, -Dsolr.solr.home=C:\solr. You can do it like how they do it for
 Tomcat: http://wiki.apache.org/solr/SolrTomcat?highlight=(solr.home)
 
  You mention your code is not compiling... the code should be able to
 compile whether or not you can actually start solr with the right
 solr-home.
 It should also compile regardless of how what container you deploy Solr
 into. What exactly are you trying to do besides getting Solr to start in
 Jetty?
 
 - Daryl.
 
 
 
 On Thu, Jun 18, 2009 at 9:58 PM, pof melbournebeerba...@gmail.com wrote:
 


 Development Team wrote:
 
  To specify the
  solr-home I use a Java system property (instead of the JNDI way) since
 I
  already have other necessary system properties for my apps.
 

 Could you please give me a concrete example of how you did this? There is
 no
 example code or commandline examples to be found.

 Cheers, Brett.

 --
 View this message in context:
 http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24104378.html
 Sent from the Solr - User mailing list archive at Nabble.com.


 
 

-- 
View this message in context: 
http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24142282.html
Sent from the Solr - User mailing list archive at Nabble.com.



Sorlj when to commit?

2009-06-22 Thread pof

Hi, I am doing a large batch (thousands) of insertions to my index using an
EmbeddedSolrServer. I was wondering how often should I use server.commit()
as I am trying to avoid unecessary bottlenecks. 

Thanks, Brett.
-- 
View this message in context: 
http://www.nabble.com/Sorlj-when-to-commit--tp24142326p24142326.html
Sent from the Solr - User mailing list archive at Nabble.com.



Solr Jetty confusion

2009-06-18 Thread pof

Hi, I am currently trying to write a Jetty embedded java app that implements
SOLR and uses SOLRJ by excepting posts telling it to do a batch index, or a
deletion or what have you. At this point I am completely lost trying to
follow http://wiki.apache.org/solr/SolrJetty . In my constructor I am doing
the following call:

Server server = new Server();
XmlConfiguration configuration = new XmlConfiguration(new
FileInputStream(solrjetty.xml));

My xml has two calls, an addConnector to configure the port etc. and the
addWebApplication as specified on the solr wiki. When running the app I get
this:

Exception in thread main java.lang.IllegalStateException: No Method: Call
name=addWebApplicationArg/solr/*/ArgArg/webapps/solr.war/ArgSet
name=extractWARtrue/SetSet
name=defaultsDescriptororg/mortbay/jetty/servlet/webdefault.xml/SetCall
name=addEnvEntryArg/solr/home/ArgArg
type=String/solr/home/Arg/Call/Call on class
org.mortbay.jetty.Server

Can anyone point me in the right direction? Thanks. 
-- 
View this message in context: 
http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24087264.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Solr Jetty confusion

2009-06-18 Thread pof

My problem is that my project doesn't compile and I have know way of knowing
if I'm on the right track code wise. There just isn't any comprehensive
guide out there for having a solr/jetty app.


Development Team wrote:
 
 Hey,
  So... I'm assuming your problem is that you're having trouble
 deploying
 Solr in Jetty? Or is your problem that it's deploying just fine but your
 code throws an exception when you try to run it?
  I am running Solr in Jetty, and I just copied the war into the
 webapps
 directory and it worked. It was accessible under /solr, and it was
 accessible under the port that Jetty has as its HTTP listener (which is
 probably 8080 by default, but probably won't be 8983). To specify the
 solr-home I use a Java system property (instead of the JNDI way) since I
 already have other necessary system properties for my apps. So if your
 problem turns out to be with the JNDI, sorry I won't be of much help.
  Hope that helps...
 
 - Daryl.
 
 
 On Thu, Jun 18, 2009 at 2:44 AM, pof melbournebeerba...@gmail.com wrote:
 

 Hi, I am currently trying to write a Jetty embedded java app that
 implements
 SOLR and uses SOLRJ by excepting posts telling it to do a batch index, or
 a
 deletion or what have you. At this point I am completely lost trying to
 follow http://wiki.apache.org/solr/SolrJetty . In my constructor I am
 doing
 the following call:

 Server server = new Server();
 XmlConfiguration configuration = new XmlConfiguration(new
 FileInputStream(solrjetty.xml));

 My xml has two calls, an addConnector to configure the port etc. and the
 addWebApplication as specified on the solr wiki. When running the app I
 get
 this:

 Exception in thread main java.lang.IllegalStateException: No Method:
 Call
 name=addWebApplicationArg/solr/*/ArgArg/webapps/solr.war/ArgSet
 name=extractWARtrue/SetSet

 name=defaultsDescriptororg/mortbay/jetty/servlet/webdefault.xml/SetCall
 name=addEnvEntryArg/solr/home/ArgArg
 type=String/solr/home/Arg/Call/Call on class
 org.mortbay.jetty.Server

 Can anyone point me in the right direction? Thanks.
 --
 View this message in context:
 http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24087264.html
 Sent from the Solr - User mailing list archive at Nabble.com.


 
 

-- 
View this message in context: 
http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24099696.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: Solr Jetty confusion

2009-06-18 Thread pof


Development Team wrote:
 
 To specify the
 solr-home I use a Java system property (instead of the JNDI way) since I
 already have other necessary system properties for my apps.
 

Could you please give me a concrete example of how you did this? There is no
example code or commandline examples to be found.

Cheers, Brett.

-- 
View this message in context: 
http://www.nabble.com/Solr-Jetty-confusion-tp24087264p24104378.html
Sent from the Solr - User mailing list archive at Nabble.com.



EmbeddedSolrServer seperate process

2009-06-15 Thread pof

Hi, one question: Will EmbeddedSolrServer work on a seperate java process
than the solr servlet(start.jar) as long as it is run on the same machine?

Thanks. 
-- 
View this message in context: 
http://www.nabble.com/EmbeddedSolrServer-seperate-process-tp24046680p24046680.html
Sent from the Solr - User mailing list archive at Nabble.com.



Re: EmbeddedSolrServer seperate process

2009-06-15 Thread pof

It certain does, thank you.


Shalin Shekhar Mangar wrote:
 
 On Tue, Jun 16, 2009 at 8:15 AM, pof melbournebeerba...@gmail.com wrote:
 

 Hi, one question: Will EmbeddedSolrServer work on a seperate java process
 than the solr servlet(start.jar) as long as it is run on the same
 machine?


 EmbeddedSolrServer is run in the same process as the application which
 uses
 it. CommonsHttpSolrServer is used by an application to communicate with
 Solr
 running in a separate host (or JVM) using HTTP. The start.jar is the jetty
 servlet container which can be used to host solr.
 
 Does that answer your question?
 
 -- 
 Regards,
 Shalin Shekhar Mangar.
 
 

-- 
View this message in context: 
http://www.nabble.com/EmbeddedSolrServer-seperate-process-tp24046680p24047849.html
Sent from the Solr - User mailing list archive at Nabble.com.