Re: Newbie question - Error loading an existing config file

[Erick Erickson]

I have absolutely no idea when it comes to Drupal, the Drupal folks would be
much better equipped to answer.

Best,
Erick

> On Feb 21, 2019, at 8:16 AM, Greg Robinson  wrote:
> 
> Thanks for the feedback.
> 
> So here is where I'm at.
> 
> I first went ahead and deleted the existing core that was returning the
> error using the following command: bin/solr delete -c new_solr_core
> 
> Now when I access the admin panel, there are no errors.
> 
> I then referred to the large "warning" box on the CREATE action
> documentation:
> 
> "While it’s possible to create a core for a non-existent collection, this
> approach is not supported and not recommended. Always create a collection
> using the *Collections API* before creating a core directly for it."
> 
> I then tried to create a collection using the following "Collections API"
> command:
> 
> http://localhost:8983/solr/admin/collections?action=CREATE=newCollection=2=1=xml
> 
> This was the response:
> 
> 
> 
> 400
> 2
> 
> 
> 
> org.apache.solr.common.SolrException
> org.apache.solr.common.SolrException
> 
> Solr instance is not running in SolrCloud mode.
> 400
> 
> 
> 
> I guess my main question is, do I need to be running in "SolrCloud mode" if
> my intention is to use Solr Server to index a Drupal 7 website? We're
> currently using opensolr.com which is working fine but we're trying to
> avoid the monthly costs associated with their "Shared Solr Cloud" plan.
> 
> Thanks!
> 
> 
> 
> 
> 
> On Wed, Feb 20, 2019 at 8:34 PM Shawn Heisey  wrote:
> 
>> On 2/20/2019 11:07 AM, Greg Robinson wrote:
>>> Lets try this: https://imgur.com/a/z5OzbLW
>>> 
>>> What I'm trying to do seems pretty straightforward:
>>> 
>>> 1. Install Solr Server 7.4 on Linux (Completed)
>>> 2. Connect my Drupal 7 site to the Solr Server and use it for indexing
>>> content
>>> 
>>> My understanding is that I must first create a core in order to connect
>> my
>>> drupal site to Solr Server. This is where I'm currently stuck.
>> 
>> The assertion in your screenshot that the dataDir must exist is
>> incorrect.  If current versions of Solr say this also, that is something
>> we will need to change.  This is what actually happens:  If all the
>> other requirements are met and the dataDir does not exist, it will be
>> created automatically when the core starts, if the process has
>> sufficient permissions.
>> 
>> See the large "warning" box on the CREATE action documentation for
>> details on what you need:
>> 
>> 
>> https://lucene.apache.org/solr/guide/7_4/coreadmin-api.html#coreadmin-create
>> 
>> The warning box is the one that has a red triangle to the left of it.
>> The red triangle contains an exclamation point.
>> 
>> The essence of what it says there is that the core's instance directory
>> must exist, that directory must contain a "conf" directory, and all
>> required config files must be in the conf directory.
>> 
>> If you're running in SolrCloud mode, then you're using the wrong API.
>> 
>> Thanks,
>> Shawn
>> 
> 
> 
> -- 
> Greg Robinson
> CEO - Mobile*Enhanced*
> www.mobileenhanced.com
> g...@mobileenhanced.com
> 303-598-1865

Re: Newbie question - Error loading an existing config file

[Greg Robinson]

Thanks for the feedback.

So here is where I'm at.

I first went ahead and deleted the existing core that was returning the
error using the following command: bin/solr delete -c new_solr_core

Now when I access the admin panel, there are no errors.

I then referred to the large "warning" box on the CREATE action
documentation:

"While it’s possible to create a core for a non-existent collection, this
approach is not supported and not recommended. Always create a collection
using the *Collections API* before creating a core directly for it."

I then tried to create a collection using the following "Collections API"
command:

http://localhost:8983/solr/admin/collections?action=CREATE=newCollection=2=1=xml

This was the response:



400
2



org.apache.solr.common.SolrException
org.apache.solr.common.SolrException

Solr instance is not running in SolrCloud mode.
400



I guess my main question is, do I need to be running in "SolrCloud mode" if
my intention is to use Solr Server to index a Drupal 7 website? We're
currently using opensolr.com which is working fine but we're trying to
avoid the monthly costs associated with their "Shared Solr Cloud" plan.

Thanks!





On Wed, Feb 20, 2019 at 8:34 PM Shawn Heisey  wrote:

> On 2/20/2019 11:07 AM, Greg Robinson wrote:
> > Lets try this: https://imgur.com/a/z5OzbLW
> >
> > What I'm trying to do seems pretty straightforward:
> >
> > 1. Install Solr Server 7.4 on Linux (Completed)
> > 2. Connect my Drupal 7 site to the Solr Server and use it for indexing
> > content
> >
> > My understanding is that I must first create a core in order to connect
> my
> > drupal site to Solr Server. This is where I'm currently stuck.
>
> The assertion in your screenshot that the dataDir must exist is
> incorrect.  If current versions of Solr say this also, that is something
> we will need to change.  This is what actually happens:  If all the
> other requirements are met and the dataDir does not exist, it will be
> created automatically when the core starts, if the process has
> sufficient permissions.
>
> See the large "warning" box on the CREATE action documentation for
> details on what you need:
>
>
> https://lucene.apache.org/solr/guide/7_4/coreadmin-api.html#coreadmin-create
>
> The warning box is the one that has a red triangle to the left of it.
> The red triangle contains an exclamation point.
>
> The essence of what it says there is that the core's instance directory
> must exist, that directory must contain a "conf" directory, and all
> required config files must be in the conf directory.
>
> If you're running in SolrCloud mode, then you're using the wrong API.
>
> Thanks,
> Shawn
>


-- 
Greg Robinson
CEO - Mobile*Enhanced*
www.mobileenhanced.com
g...@mobileenhanced.com
303-598-1865

Re: Newbie question - Error loading an existing config file

[Shawn Heisey]

On 2/20/2019 11:07 AM, Greg Robinson wrote:

Lets try this: https://imgur.com/a/z5OzbLW

What I'm trying to do seems pretty straightforward:

1. Install Solr Server 7.4 on Linux (Completed)
2. Connect my Drupal 7 site to the Solr Server and use it for indexing
content

My understanding is that I must first create a core in order to connect my
drupal site to Solr Server. This is where I'm currently stuck.


The assertion in your screenshot that the dataDir must exist is 
incorrect.  If current versions of Solr say this also, that is something 
we will need to change.  This is what actually happens:  If all the 
other requirements are met and the dataDir does not exist, it will be 
created automatically when the core starts, if the process has 
sufficient permissions.


See the large "warning" box on the CREATE action documentation for 
details on what you need:


https://lucene.apache.org/solr/guide/7_4/coreadmin-api.html#coreadmin-create

The warning box is the one that has a red triangle to the left of it. 
The red triangle contains an exclamation point.


The essence of what it says there is that the core's instance directory 
must exist, that directory must contain a "conf" directory, and all 
required config files must be in the conf directory.


If you're running in SolrCloud mode, then you're using the wrong API.

Thanks,
Shawn

Re: Newbie question - Error loading an existing config file

[Greg Robinson]

Gotcha.

Lets try this: https://imgur.com/a/z5OzbLW

What I'm trying to do seems pretty straightforward:

1. Install Solr Server 7.4 on Linux (Completed)
2. Connect my Drupal 7 site to the Solr Server and use it for indexing
content

My understanding is that I must first create a core in order to connect my
drupal site to Solr Server. This is where I'm currently stuck.

Thanks for your help!

On Wed, Feb 20, 2019 at 10:43 AM Erick Erickson 
wrote:

> Attachments generally are stripped by the mail server.
>
> Are you trying to create a core as part of a SolrCloud _collection_? If
> so, this
> is an anti-pattern, use the collection API commands. Shot in the dark.
>
> Best,
> Erick
>
> > On Feb 19, 2019, at 3:05 PM, Greg Robinson 
> wrote:
> >
> > I used the front end admin (see attached)
> >
> > thanks
> >
> > On Tue, Feb 19, 2019 at 3:54 PM Erick Erickson 
> wrote:
> > Hmmm, that’s not very helpful…..
> >
> > Don’t quite know what to say. There should be something more helpful
> > in the logs.
> >
> > Hmmm, How did you create the core?
> >
> > Best,
> > Erick
> >
> >
> > > On Feb 19, 2019, at 1:29 PM, Greg Robinson 
> wrote:
> > >
> > > Thanks for your direction regarding the log.
> > >
> > > I was able to locate it and these two lines stood out:
> > >
> > > Caused by: org.apache.solr.common.SolrException: Could not load conf
> for
> > > core new_solr_core: Error loading solr config from
> > > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> > >
> > > Caused by: org.apache.solr.common.SolrException: Error loading solr
> config
> > > from /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> > >
> > > which seems to point to the same issue.
> > >
> > > I also went ahead and updated permissions/owner to "solr" on all
> > > directories and files within "/home/solr/server/solr/new_solr_core".
> > >
> > > Still no luck. This is currently the same message that I'm getting on
> the
> > > admin front end:
> > >
> > > new_solr_core:
> > >
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> > > Could not load conf for core new_solr_core: Error loading solr config
> from
> > > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml.
> > >
> > > thanks!
> > >
> > >
> > >
> > > On Tue, Feb 19, 2019 at 1:55 PM Erick Erickson <
> erickerick...@gmail.com>
> > > wrote:
> > >
> > >> do a recursive seach for “solr.log" under SOLR_HOME…….
> > >>
> > >> Best,
> > >> ERick
> > >>
> > >>> On Feb 19, 2019, at 8:08 AM, Greg Robinson 
> > >> wrote:
> > >>>
> > >>> Hi Erick,
> > >>>
> > >>> Thanks for the quick response.
> > >>>
> > >>> Here is what is currently contained within  the conf dir:
> > >>>
> > >>> drwxr-xr-x 2 root root  4096 Feb 18 17:51 lang
> > >>> -rw-r--r-- 1 root root 54513 Feb 18 17:51 managed-schema
> > >>> -rw-r--r-- 1 root root   329 Feb 18 17:51 params.json
> > >>> -rw-r--r-- 1 root root   894 Feb 18 17:51 protwords.txt
> > >>> -rwxrwxrwx 1 root root 55323 Feb 18 17:51 solrconfig.xml
> > >>> -rw-r--r-- 1 root root   795 Feb 18 17:51 stopwords.txt
> > >>> -rw-r--r-- 1 root root  1153 Feb 18 17:51 synonyms.txt
> > >>>
> > >>> As far as the log, where exactly might I find the specific log that
> would
> > >>> give more info in regards to this error?
> > >>>
> > >>> thanks again!
> > >>>
> > >>> On Tue, Feb 19, 2019 at 9:06 AM Erick Erickson <
> erickerick...@gmail.com>
> > >>> wrote:
> > >>>
> >  Are all the other files there in your conf dir? Solrconfig.xml
> > >> references
> >  things like nanaged-schema etc.
> > 
> >  Also, your log file might contain more clues...
> > 
> >  On Tue, Feb 19, 2019, 08:03 Greg Robinson  > >> wrote:
> > 
> > > Hello,
> > >
> > > We have Solr 7.4 up and running on a Linux machine.
> > >
> > > I'm just trying to add a new core so that I can eventually point a
> > >> Drupal
> > > site to the Solr Server for indexing.
> > >
> > > When attempting to add a core, I'm getting the following error:
> > >
> > > new_solr_core:
> > >
> > 
> > >>
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> > > Could not load conf for core new_solr_core: Error loading solr
> config
> >  from
> > > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> > >
> > > I've confirmed that
> > > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml exists
> but I'm
> > > still getting the error.
> > >
> > > Any direction is appreciated.
> > >
> > > Thanks!
> > >
> > 
> > >>>
> > >>>
> > >>> --
> > >>> Greg Robinson
> > >>> CEO - Mobile*Enhanced*
> > >>> www.mobileenhanced.com
> > >>> g...@mobileenhanced.com
> > >>> 303-598-1865
> > >>
> > >>
> > >
> > > --
> > > Greg Robinson
> > > CEO - Mobile*Enhanced*
> > > www.mobileenhanced.com
> > > g...@mobileenhanced.com
> > > 303-598-1865
> >
> >
> >
> > --
> > Greg Robinson
> > CEO - MobileEnhanced
> > www.mobileenhanced.com
> > g...@mobileenhanced.com
> > 

Re: Newbie question - Error loading an existing config file

[Erick Erickson]

Attachments generally are stripped by the mail server.

Are you trying to create a core as part of a SolrCloud _collection_? If so, this
is an anti-pattern, use the collection API commands. Shot in the dark.

Best,
Erick

> On Feb 19, 2019, at 3:05 PM, Greg Robinson  wrote:
> 
> I used the front end admin (see attached)
> 
> thanks
> 
> On Tue, Feb 19, 2019 at 3:54 PM Erick Erickson  
> wrote:
> Hmmm, that’s not very helpful…..
> 
> Don’t quite know what to say. There should be something more helpful
> in the logs.
> 
> Hmmm, How did you create the core?
> 
> Best,
> Erick
> 
> 
> > On Feb 19, 2019, at 1:29 PM, Greg Robinson  wrote:
> > 
> > Thanks for your direction regarding the log.
> > 
> > I was able to locate it and these two lines stood out:
> > 
> > Caused by: org.apache.solr.common.SolrException: Could not load conf for
> > core new_solr_core: Error loading solr config from
> > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> > 
> > Caused by: org.apache.solr.common.SolrException: Error loading solr config
> > from /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> > 
> > which seems to point to the same issue.
> > 
> > I also went ahead and updated permissions/owner to "solr" on all
> > directories and files within "/home/solr/server/solr/new_solr_core".
> > 
> > Still no luck. This is currently the same message that I'm getting on the
> > admin front end:
> > 
> > new_solr_core:
> > org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> > Could not load conf for core new_solr_core: Error loading solr config from
> > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml.
> > 
> > thanks!
> > 
> > 
> > 
> > On Tue, Feb 19, 2019 at 1:55 PM Erick Erickson 
> > wrote:
> > 
> >> do a recursive seach for “solr.log" under SOLR_HOME…….
> >> 
> >> Best,
> >> ERick
> >> 
> >>> On Feb 19, 2019, at 8:08 AM, Greg Robinson 
> >> wrote:
> >>> 
> >>> Hi Erick,
> >>> 
> >>> Thanks for the quick response.
> >>> 
> >>> Here is what is currently contained within  the conf dir:
> >>> 
> >>> drwxr-xr-x 2 root root  4096 Feb 18 17:51 lang
> >>> -rw-r--r-- 1 root root 54513 Feb 18 17:51 managed-schema
> >>> -rw-r--r-- 1 root root   329 Feb 18 17:51 params.json
> >>> -rw-r--r-- 1 root root   894 Feb 18 17:51 protwords.txt
> >>> -rwxrwxrwx 1 root root 55323 Feb 18 17:51 solrconfig.xml
> >>> -rw-r--r-- 1 root root   795 Feb 18 17:51 stopwords.txt
> >>> -rw-r--r-- 1 root root  1153 Feb 18 17:51 synonyms.txt
> >>> 
> >>> As far as the log, where exactly might I find the specific log that would
> >>> give more info in regards to this error?
> >>> 
> >>> thanks again!
> >>> 
> >>> On Tue, Feb 19, 2019 at 9:06 AM Erick Erickson 
> >>> wrote:
> >>> 
>  Are all the other files there in your conf dir? Solrconfig.xml
> >> references
>  things like nanaged-schema etc.
>  
>  Also, your log file might contain more clues...
>  
>  On Tue, Feb 19, 2019, 08:03 Greg Robinson  >> wrote:
>  
> > Hello,
> > 
> > We have Solr 7.4 up and running on a Linux machine.
> > 
> > I'm just trying to add a new core so that I can eventually point a
> >> Drupal
> > site to the Solr Server for indexing.
> > 
> > When attempting to add a core, I'm getting the following error:
> > 
> > new_solr_core:
> > 
>  
> >> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> > Could not load conf for core new_solr_core: Error loading solr config
>  from
> > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> > 
> > I've confirmed that
> > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml exists but I'm
> > still getting the error.
> > 
> > Any direction is appreciated.
> > 
> > Thanks!
> > 
>  
> >>> 
> >>> 
> >>> --
> >>> Greg Robinson
> >>> CEO - Mobile*Enhanced*
> >>> www.mobileenhanced.com
> >>> g...@mobileenhanced.com
> >>> 303-598-1865
> >> 
> >> 
> > 
> > -- 
> > Greg Robinson
> > CEO - Mobile*Enhanced*
> > www.mobileenhanced.com
> > g...@mobileenhanced.com
> > 303-598-1865
> 
> 
> 
> -- 
> Greg Robinson
> CEO - MobileEnhanced
> www.mobileenhanced.com
> g...@mobileenhanced.com
> 303-598-1865

Re: Newbie question - Error loading an existing config file

[Greg Robinson]

I used the front end admin (see attached)

thanks

On Tue, Feb 19, 2019 at 3:54 PM Erick Erickson 
wrote:

> Hmmm, that’s not very helpful…..
>
> Don’t quite know what to say. There should be something more helpful
> in the logs.
>
> Hmmm, How did you create the core?
>
> Best,
> Erick
>
>
> > On Feb 19, 2019, at 1:29 PM, Greg Robinson 
> wrote:
> >
> > Thanks for your direction regarding the log.
> >
> > I was able to locate it and these two lines stood out:
> >
> > Caused by: org.apache.solr.common.SolrException: Could not load conf for
> > core new_solr_core: Error loading solr config from
> > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> >
> > Caused by: org.apache.solr.common.SolrException: Error loading solr
> config
> > from /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> >
> > which seems to point to the same issue.
> >
> > I also went ahead and updated permissions/owner to "solr" on all
> > directories and files within "/home/solr/server/solr/new_solr_core".
> >
> > Still no luck. This is currently the same message that I'm getting on the
> > admin front end:
> >
> > new_solr_core:
> >
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> > Could not load conf for core new_solr_core: Error loading solr config
> from
> > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml.
> >
> > thanks!
> >
> >
> >
> > On Tue, Feb 19, 2019 at 1:55 PM Erick Erickson 
> > wrote:
> >
> >> do a recursive seach for “solr.log" under SOLR_HOME…….
> >>
> >> Best,
> >> ERick
> >>
> >>> On Feb 19, 2019, at 8:08 AM, Greg Robinson 
> >> wrote:
> >>>
> >>> Hi Erick,
> >>>
> >>> Thanks for the quick response.
> >>>
> >>> Here is what is currently contained within  the conf dir:
> >>>
> >>> drwxr-xr-x 2 root root  4096 Feb 18 17:51 lang
> >>> -rw-r--r-- 1 root root 54513 Feb 18 17:51 managed-schema
> >>> -rw-r--r-- 1 root root   329 Feb 18 17:51 params.json
> >>> -rw-r--r-- 1 root root   894 Feb 18 17:51 protwords.txt
> >>> -rwxrwxrwx 1 root root 55323 Feb 18 17:51 solrconfig.xml
> >>> -rw-r--r-- 1 root root   795 Feb 18 17:51 stopwords.txt
> >>> -rw-r--r-- 1 root root  1153 Feb 18 17:51 synonyms.txt
> >>>
> >>> As far as the log, where exactly might I find the specific log that
> would
> >>> give more info in regards to this error?
> >>>
> >>> thanks again!
> >>>
> >>> On Tue, Feb 19, 2019 at 9:06 AM Erick Erickson <
> erickerick...@gmail.com>
> >>> wrote:
> >>>
>  Are all the other files there in your conf dir? Solrconfig.xml
> >> references
>  things like nanaged-schema etc.
> 
>  Also, your log file might contain more clues...
> 
>  On Tue, Feb 19, 2019, 08:03 Greg Robinson  >> wrote:
> 
> > Hello,
> >
> > We have Solr 7.4 up and running on a Linux machine.
> >
> > I'm just trying to add a new core so that I can eventually point a
> >> Drupal
> > site to the Solr Server for indexing.
> >
> > When attempting to add a core, I'm getting the following error:
> >
> > new_solr_core:
> >
> 
> >>
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> > Could not load conf for core new_solr_core: Error loading solr config
>  from
> > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> >
> > I've confirmed that
> > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml exists but
> I'm
> > still getting the error.
> >
> > Any direction is appreciated.
> >
> > Thanks!
> >
> 
> >>>
> >>>
> >>> --
> >>> Greg Robinson
> >>> CEO - Mobile*Enhanced*
> >>> www.mobileenhanced.com
> >>> g...@mobileenhanced.com
> >>> 303-598-1865
> >>
> >>
> >
> > --
> > Greg Robinson
> > CEO - Mobile*Enhanced*
> > www.mobileenhanced.com
> > g...@mobileenhanced.com
> > 303-598-1865
>
>

-- 
Greg Robinson
CEO - Mobile*Enhanced*
www.mobileenhanced.com
g...@mobileenhanced.com
303-598-1865

Re: Newbie question - Error loading an existing config file

[Erick Erickson]

Hmmm, that’s not very helpful…..

Don’t quite know what to say. There should be something more helpful
in the logs.

Hmmm, How did you create the core?

Best,
Erick


> On Feb 19, 2019, at 1:29 PM, Greg Robinson  wrote:
> 
> Thanks for your direction regarding the log.
> 
> I was able to locate it and these two lines stood out:
> 
> Caused by: org.apache.solr.common.SolrException: Could not load conf for
> core new_solr_core: Error loading solr config from
> /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> 
> Caused by: org.apache.solr.common.SolrException: Error loading solr config
> from /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> 
> which seems to point to the same issue.
> 
> I also went ahead and updated permissions/owner to "solr" on all
> directories and files within "/home/solr/server/solr/new_solr_core".
> 
> Still no luck. This is currently the same message that I'm getting on the
> admin front end:
> 
> new_solr_core:
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Could not load conf for core new_solr_core: Error loading solr config from
> /home/solr/server/solr/new_solr_core/conf/solrconfig.xml.
> 
> thanks!
> 
> 
> 
> On Tue, Feb 19, 2019 at 1:55 PM Erick Erickson 
> wrote:
> 
>> do a recursive seach for “solr.log" under SOLR_HOME…….
>> 
>> Best,
>> ERick
>> 
>>> On Feb 19, 2019, at 8:08 AM, Greg Robinson 
>> wrote:
>>> 
>>> Hi Erick,
>>> 
>>> Thanks for the quick response.
>>> 
>>> Here is what is currently contained within  the conf dir:
>>> 
>>> drwxr-xr-x 2 root root  4096 Feb 18 17:51 lang
>>> -rw-r--r-- 1 root root 54513 Feb 18 17:51 managed-schema
>>> -rw-r--r-- 1 root root   329 Feb 18 17:51 params.json
>>> -rw-r--r-- 1 root root   894 Feb 18 17:51 protwords.txt
>>> -rwxrwxrwx 1 root root 55323 Feb 18 17:51 solrconfig.xml
>>> -rw-r--r-- 1 root root   795 Feb 18 17:51 stopwords.txt
>>> -rw-r--r-- 1 root root  1153 Feb 18 17:51 synonyms.txt
>>> 
>>> As far as the log, where exactly might I find the specific log that would
>>> give more info in regards to this error?
>>> 
>>> thanks again!
>>> 
>>> On Tue, Feb 19, 2019 at 9:06 AM Erick Erickson 
>>> wrote:
>>> 
 Are all the other files there in your conf dir? Solrconfig.xml
>> references
 things like nanaged-schema etc.
 
 Also, your log file might contain more clues...
 
 On Tue, Feb 19, 2019, 08:03 Greg Robinson > wrote:
 
> Hello,
> 
> We have Solr 7.4 up and running on a Linux machine.
> 
> I'm just trying to add a new core so that I can eventually point a
>> Drupal
> site to the Solr Server for indexing.
> 
> When attempting to add a core, I'm getting the following error:
> 
> new_solr_core:
> 
 
>> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Could not load conf for core new_solr_core: Error loading solr config
 from
> /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> 
> I've confirmed that
> /home/solr/server/solr/new_solr_core/conf/solrconfig.xml exists but I'm
> still getting the error.
> 
> Any direction is appreciated.
> 
> Thanks!
> 
 
>>> 
>>> 
>>> --
>>> Greg Robinson
>>> CEO - Mobile*Enhanced*
>>> www.mobileenhanced.com
>>> g...@mobileenhanced.com
>>> 303-598-1865
>> 
>> 
> 
> -- 
> Greg Robinson
> CEO - Mobile*Enhanced*
> www.mobileenhanced.com
> g...@mobileenhanced.com
> 303-598-1865

Re: Newbie question - Error loading an existing config file

[Greg Robinson]

Thanks for your direction regarding the log.

I was able to locate it and these two lines stood out:

Caused by: org.apache.solr.common.SolrException: Could not load conf for
core new_solr_core: Error loading solr config from
/home/solr/server/solr/new_solr_core/conf/solrconfig.xml

Caused by: org.apache.solr.common.SolrException: Error loading solr config
from /home/solr/server/solr/new_solr_core/conf/solrconfig.xml

which seems to point to the same issue.

I also went ahead and updated permissions/owner to "solr" on all
directories and files within "/home/solr/server/solr/new_solr_core".

Still no luck. This is currently the same message that I'm getting on the
admin front end:

new_solr_core:
org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
Could not load conf for core new_solr_core: Error loading solr config from
/home/solr/server/solr/new_solr_core/conf/solrconfig.xml.

thanks!



On Tue, Feb 19, 2019 at 1:55 PM Erick Erickson 
wrote:

> do a recursive seach for “solr.log" under SOLR_HOME…….
>
> Best,
> ERick
>
> > On Feb 19, 2019, at 8:08 AM, Greg Robinson 
> wrote:
> >
> > Hi Erick,
> >
> > Thanks for the quick response.
> >
> > Here is what is currently contained within  the conf dir:
> >
> > drwxr-xr-x 2 root root  4096 Feb 18 17:51 lang
> > -rw-r--r-- 1 root root 54513 Feb 18 17:51 managed-schema
> > -rw-r--r-- 1 root root   329 Feb 18 17:51 params.json
> > -rw-r--r-- 1 root root   894 Feb 18 17:51 protwords.txt
> > -rwxrwxrwx 1 root root 55323 Feb 18 17:51 solrconfig.xml
> > -rw-r--r-- 1 root root   795 Feb 18 17:51 stopwords.txt
> > -rw-r--r-- 1 root root  1153 Feb 18 17:51 synonyms.txt
> >
> > As far as the log, where exactly might I find the specific log that would
> > give more info in regards to this error?
> >
> > thanks again!
> >
> > On Tue, Feb 19, 2019 at 9:06 AM Erick Erickson 
> > wrote:
> >
> >> Are all the other files there in your conf dir? Solrconfig.xml
> references
> >> things like nanaged-schema etc.
> >>
> >> Also, your log file might contain more clues...
> >>
> >> On Tue, Feb 19, 2019, 08:03 Greg Robinson  wrote:
> >>
> >>> Hello,
> >>>
> >>> We have Solr 7.4 up and running on a Linux machine.
> >>>
> >>> I'm just trying to add a new core so that I can eventually point a
> Drupal
> >>> site to the Solr Server for indexing.
> >>>
> >>> When attempting to add a core, I'm getting the following error:
> >>>
> >>> new_solr_core:
> >>>
> >>
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> >>> Could not load conf for core new_solr_core: Error loading solr config
> >> from
> >>> /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> >>>
> >>> I've confirmed that
> >>> /home/solr/server/solr/new_solr_core/conf/solrconfig.xml exists but I'm
> >>> still getting the error.
> >>>
> >>> Any direction is appreciated.
> >>>
> >>> Thanks!
> >>>
> >>
> >
> >
> > --
> > Greg Robinson
> > CEO - Mobile*Enhanced*
> > www.mobileenhanced.com
> > g...@mobileenhanced.com
> > 303-598-1865
>
>

-- 
Greg Robinson
CEO - Mobile*Enhanced*
www.mobileenhanced.com
g...@mobileenhanced.com
303-598-1865

Re: Newbie question - Error loading an existing config file

[Erick Erickson]

do a recursive seach for “solr.log" under SOLR_HOME…….

Best,
ERick

> On Feb 19, 2019, at 8:08 AM, Greg Robinson  wrote:
> 
> Hi Erick,
> 
> Thanks for the quick response.
> 
> Here is what is currently contained within  the conf dir:
> 
> drwxr-xr-x 2 root root  4096 Feb 18 17:51 lang
> -rw-r--r-- 1 root root 54513 Feb 18 17:51 managed-schema
> -rw-r--r-- 1 root root   329 Feb 18 17:51 params.json
> -rw-r--r-- 1 root root   894 Feb 18 17:51 protwords.txt
> -rwxrwxrwx 1 root root 55323 Feb 18 17:51 solrconfig.xml
> -rw-r--r-- 1 root root   795 Feb 18 17:51 stopwords.txt
> -rw-r--r-- 1 root root  1153 Feb 18 17:51 synonyms.txt
> 
> As far as the log, where exactly might I find the specific log that would
> give more info in regards to this error?
> 
> thanks again!
> 
> On Tue, Feb 19, 2019 at 9:06 AM Erick Erickson 
> wrote:
> 
>> Are all the other files there in your conf dir? Solrconfig.xml references
>> things like nanaged-schema etc.
>> 
>> Also, your log file might contain more clues...
>> 
>> On Tue, Feb 19, 2019, 08:03 Greg Robinson > 
>>> Hello,
>>> 
>>> We have Solr 7.4 up and running on a Linux machine.
>>> 
>>> I'm just trying to add a new core so that I can eventually point a Drupal
>>> site to the Solr Server for indexing.
>>> 
>>> When attempting to add a core, I'm getting the following error:
>>> 
>>> new_solr_core:
>>> 
>> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
>>> Could not load conf for core new_solr_core: Error loading solr config
>> from
>>> /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
>>> 
>>> I've confirmed that
>>> /home/solr/server/solr/new_solr_core/conf/solrconfig.xml exists but I'm
>>> still getting the error.
>>> 
>>> Any direction is appreciated.
>>> 
>>> Thanks!
>>> 
>> 
> 
> 
> -- 
> Greg Robinson
> CEO - Mobile*Enhanced*
> www.mobileenhanced.com
> g...@mobileenhanced.com
> 303-598-1865

Re: Newbie question - Error loading an existing config file

[Greg Robinson]

Hi Erick,

Thanks for the quick response.

Here is what is currently contained within  the conf dir:

drwxr-xr-x 2 root root  4096 Feb 18 17:51 lang
-rw-r--r-- 1 root root 54513 Feb 18 17:51 managed-schema
-rw-r--r-- 1 root root   329 Feb 18 17:51 params.json
-rw-r--r-- 1 root root   894 Feb 18 17:51 protwords.txt
-rwxrwxrwx 1 root root 55323 Feb 18 17:51 solrconfig.xml
-rw-r--r-- 1 root root   795 Feb 18 17:51 stopwords.txt
-rw-r--r-- 1 root root  1153 Feb 18 17:51 synonyms.txt

As far as the log, where exactly might I find the specific log that would
give more info in regards to this error?

thanks again!

On Tue, Feb 19, 2019 at 9:06 AM Erick Erickson 
wrote:

> Are all the other files there in your conf dir? Solrconfig.xml references
> things like nanaged-schema etc.
>
> Also, your log file might contain more clues...
>
> On Tue, Feb 19, 2019, 08:03 Greg Robinson 
> > Hello,
> >
> > We have Solr 7.4 up and running on a Linux machine.
> >
> > I'm just trying to add a new core so that I can eventually point a Drupal
> > site to the Solr Server for indexing.
> >
> > When attempting to add a core, I'm getting the following error:
> >
> > new_solr_core:
> >
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> > Could not load conf for core new_solr_core: Error loading solr config
> from
> > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
> >
> > I've confirmed that
> > /home/solr/server/solr/new_solr_core/conf/solrconfig.xml exists but I'm
> > still getting the error.
> >
> > Any direction is appreciated.
> >
> > Thanks!
> >
>


-- 
Greg Robinson
CEO - Mobile*Enhanced*
www.mobileenhanced.com
g...@mobileenhanced.com
303-598-1865

Re: Newbie question - Error loading an existing config file

[Erick Erickson]

Are all the other files there in your conf dir? Solrconfig.xml references
things like nanaged-schema etc.

Also, your log file might contain more clues...

On Tue, Feb 19, 2019, 08:03 Greg Robinson  Hello,
>
> We have Solr 7.4 up and running on a Linux machine.
>
> I'm just trying to add a new core so that I can eventually point a Drupal
> site to the Solr Server for indexing.
>
> When attempting to add a core, I'm getting the following error:
>
> new_solr_core:
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Could not load conf for core new_solr_core: Error loading solr config from
> /home/solr/server/solr/new_solr_core/conf/solrconfig.xml
>
> I've confirmed that
> /home/solr/server/solr/new_solr_core/conf/solrconfig.xml exists but I'm
> still getting the error.
>
> Any direction is appreciated.
>
> Thanks!
>

Re: Newbie Question

[Deepak Goel]

*Hello*

*The code which worked for me:*

SolrClient client = new HttpSolrClient.Builder("
http://localhost:8983/solr/shakespeare;).build();

SolrQuery query = new SolrQuery();
query.setRequestHandler("/select");
query.setQuery("text_entry:henry");
query.setFields("text_entry");

QueryResponse queryResponse = null;
try
{
queryResponse = client.query(query);
}
catch (Exception e)
{

}

System.out.println("Query Response: " +queryResponse.toString());

if (queryResponse!=null && queryResponse.getResponse().size()>0)
{
SolrDocumentList results = queryResponse.getResults();
for (int i = 0; i < results.size(); ++i) {
SolrDocument document = results.get(i);
System.out.println("The result is: " +results.get(i));
System.out.println("The Document field names are: "
+document.getFieldNames());
}
}

*The data:*

{"index":{"_index":"shakespeare","_id":0}}
{"type":"act","line_id":1,"play_name":"Henry IV",
"speech_number":"","line_number":"","speaker":"","text_entry":"ACT I"}
{"index":{"_index":"shakespeare","_id":1}}
{"type":"scene","line_id":2,"play_name":"Henry
IV","speech_number":"","line_number":"","speaker":"","text_entry":"SCENE I.
London. The palace."}

*Deepak*



Virus-free.
www.avg.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>



Deepak
"Please stop cruelty to Animals, help by becoming a Vegan"
+91 73500 12833
deic...@gmail.com

Facebook: https://www.facebook.com/deicool
LinkedIn: www.linkedin.com/in/deicool

"Plant a Tree, Go Green"

On Tue, Jan 9, 2018 at 8:09 PM, Shawn Heisey  wrote:

> On 1/8/2018 10:23 AM, Deepak Goel wrote:
> > *I am trying to search for documents in my collection (Shakespeare). The
> > code is as follows:*
> >
> > SolrClient client = new HttpSolrClient.Builder("
> > http://localhost:8983/solr/shakespeare;).build();
> >
> > SolrDocument doc = client.getById("2");
> > *However this does not return any document. What mistake am I making?*
>
> The getById method accesses the handler named "/get", normally defined
> with the RealTimeGetHandler class.  In recent Solr versions, the /get
> handler is defined implicitly and does not have to be configured, but in
> older versions (not sure which ones) you do need to have it in
> solrconfig.xml.
>
> I didn't expect your code to work because getById method returns a
> SolrDocumentList and you have SolrDocument, but apparently this actually
> does work.  I have tried code very similar to yours against the
> techproducts example in version 7.1, and it works perfectly.  I will
> share the exact code I tried and what results I got below.
>
> What code have you tried after the code you've shared?  How are you
> determining that no document is returned?  Are there any error messages
> logged by the client code or Solr?  If there are, can you share them?
>
> Do you have a document in the shakespeare index that has the value "2"
> in whatever field is the uniqueKey?  Does the schema have a uniqueKey
> defined?
>
> Can you find the entry in solr.log that logs the query and share that
> entire log entry?
>
> Code:
>
> public static void main(String[] args) throws SolrServerException,
> IOException
> {
>   String baseUrl = "http://localhost:8983/solr/techproducts;;
>   SolrClient client = new HttpSolrClient.Builder(baseUrl).build();
>   SolrDocument doc = client.getById("SP2514N");
>   System.out.println(doc.getFieldValue("name"));
> }
>
> Console log from that code:
>
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
> further details.
> Samsung SpinPoint P120 SP2514N - hard drive - 250 GB - ATA-133
>
>
> Including the collection/core name in the URL is an older way of writing
> SolrJ code.  It works well, but multiple collections can be accessed
> through one client object if you change it and your SolrJ version is new
> enough.
>
> Thanks,
> Shawn
>
>

Re: Newbie Question

[Shawn Heisey]

On 1/8/2018 10:23 AM, Deepak Goel wrote:
> *I am trying to search for documents in my collection (Shakespeare). The
> code is as follows:*
>
> SolrClient client = new HttpSolrClient.Builder("
> http://localhost:8983/solr/shakespeare;).build();
>
> SolrDocument doc = client.getById("2");
> *However this does not return any document. What mistake am I making?*

The getById method accesses the handler named "/get", normally defined
with the RealTimeGetHandler class.  In recent Solr versions, the /get
handler is defined implicitly and does not have to be configured, but in
older versions (not sure which ones) you do need to have it in
solrconfig.xml.

I didn't expect your code to work because getById method returns a
SolrDocumentList and you have SolrDocument, but apparently this actually
does work.  I have tried code very similar to yours against the
techproducts example in version 7.1, and it works perfectly.  I will
share the exact code I tried and what results I got below.

What code have you tried after the code you've shared?  How are you
determining that no document is returned?  Are there any error messages
logged by the client code or Solr?  If there are, can you share them?

Do you have a document in the shakespeare index that has the value "2"
in whatever field is the uniqueKey?  Does the schema have a uniqueKey
defined?

Can you find the entry in solr.log that logs the query and share that
entire log entry?

Code:

public static void main(String[] args) throws SolrServerException,
IOException
{
  String baseUrl = "http://localhost:8983/solr/techproducts;;
  SolrClient client = new HttpSolrClient.Builder(baseUrl).build();
  SolrDocument doc = client.getById("SP2514N");
  System.out.println(doc.getFieldValue("name"));
}

Console log from that code:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for
further details.
Samsung SpinPoint P120 SP2514N - hard drive - 250 GB - ATA-133


Including the collection/core name in the URL is an older way of writing
SolrJ code.  It works well, but multiple collections can be accessed
through one client object if you change it and your SolrJ version is new
enough.

Thanks,
Shawn

Re: Newbie Question

[Deepak Goel]

Got it . Thank You for your help



Deepak
"Please stop cruelty to Animals, help by becoming a Vegan"
+91 73500 12833
deic...@gmail.com

Facebook: https://www.facebook.com/deicool
LinkedIn: www.linkedin.com/in/deicool

"Plant a Tree, Go Green"

On Mon, Jan 8, 2018 at 11:48 PM, Deepak Goel  wrote:

> *Is this right?*
>
> SolrClient client = new HttpSolrClient.Builder("http:/
> /localhost:8983/solr/shakespeare/select").build();
>
> SolrQuery query = new SolrQuery();
> query.setQuery("henry");
> query.setFields("text_entry");
> query.setStart(0);
>
> queryResponse = client.query(query);
>
> *This is still returning NULL*
>
>
>
> 
>  Virus-free.
> www.avg.com
> 
> <#m_-1646772333528808550_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
>
>
> Deepak
> "Please stop cruelty to Animals, help by becoming a Vegan"
> +91 73500 12833
> deic...@gmail.com
>
> Facebook: https://www.facebook.com/deicool
> LinkedIn: www.linkedin.com/in/deicool
>
> "Plant a Tree, Go Green"
>
> On Mon, Jan 8, 2018 at 10:55 PM, Alexandre Rafalovitch  > wrote:
>
>> I think you are missing /query handler endpoint in the URL. Plus actual
>> search parameters.
>>
>> You may try using the admin UI to build your queries first.
>>
>> Regards,
>> Alex
>>
>> On Jan 8, 2018 12:23 PM, "Deepak Goel"  wrote:
>>
>> > Hello
>> >
>> > *I am trying to search for documents in my collection (Shakespeare). The
>> > code is as follows:*
>> >
>> > SolrClient client = new HttpSolrClient.Builder("
>> > http://localhost:8983/solr/shakespeare;).build();
>> >
>> > SolrDocument doc = client.getById("2");
>> > *However this does not return any document. What mistake am I making?*
>> >
>> > Thank You
>> > Deepak
>> >
>> > Deepak
>> > "Please stop cruelty to Animals, help by becoming a Vegan"
>> > +91 73500 12833
>> > deic...@gmail.com
>> >
>> > Facebook: https://www.facebook.com/deicool
>> > LinkedIn: www.linkedin.com/in/deicool
>> >
>> > "Plant a Tree, Go Green"
>> >
>> > > > utm_source=link_campaign=sig-email_content=webmail>
>> > Virus-free.
>> > www.avg.com
>> > > > utm_source=link_campaign=sig-email_content=webmail>
>> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>> >
>>
>
>

Re: Newbie Question

[Deepak Goel]

*Is this right?*

SolrClient client = new HttpSolrClient.Builder("
http://localhost:8983/solr/shakespeare/select;).build();

SolrQuery query = new SolrQuery();
query.setQuery("henry");
query.setFields("text_entry");
query.setStart(0);

queryResponse = client.query(query);

*This is still returning NULL*



Virus-free.
www.avg.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>



Deepak
"Please stop cruelty to Animals, help by becoming a Vegan"
+91 73500 12833
deic...@gmail.com

Facebook: https://www.facebook.com/deicool
LinkedIn: www.linkedin.com/in/deicool

"Plant a Tree, Go Green"

On Mon, Jan 8, 2018 at 10:55 PM, Alexandre Rafalovitch 
wrote:

> I think you are missing /query handler endpoint in the URL. Plus actual
> search parameters.
>
> You may try using the admin UI to build your queries first.
>
> Regards,
> Alex
>
> On Jan 8, 2018 12:23 PM, "Deepak Goel"  wrote:
>
> > Hello
> >
> > *I am trying to search for documents in my collection (Shakespeare). The
> > code is as follows:*
> >
> > SolrClient client = new HttpSolrClient.Builder("
> > http://localhost:8983/solr/shakespeare;).build();
> >
> > SolrDocument doc = client.getById("2");
> > *However this does not return any document. What mistake am I making?*
> >
> > Thank You
> > Deepak
> >
> > Deepak
> > "Please stop cruelty to Animals, help by becoming a Vegan"
> > +91 73500 12833
> > deic...@gmail.com
> >
> > Facebook: https://www.facebook.com/deicool
> > LinkedIn: www.linkedin.com/in/deicool
> >
> > "Plant a Tree, Go Green"
> >
> >  > utm_source=link_campaign=sig-email_content=webmail>
> > Virus-free.
> > www.avg.com
> >  > utm_source=link_campaign=sig-email_content=webmail>
> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> >
>

Re: Newbie Question

[Alexandre Rafalovitch]

I think you are missing /query handler endpoint in the URL. Plus actual
search parameters.

You may try using the admin UI to build your queries first.

Regards,
Alex

On Jan 8, 2018 12:23 PM, "Deepak Goel"  wrote:

> Hello
>
> *I am trying to search for documents in my collection (Shakespeare). The
> code is as follows:*
>
> SolrClient client = new HttpSolrClient.Builder("
> http://localhost:8983/solr/shakespeare;).build();
>
> SolrDocument doc = client.getById("2");
> *However this does not return any document. What mistake am I making?*
>
> Thank You
> Deepak
>
> Deepak
> "Please stop cruelty to Animals, help by becoming a Vegan"
> +91 73500 12833
> deic...@gmail.com
>
> Facebook: https://www.facebook.com/deicool
> LinkedIn: www.linkedin.com/in/deicool
>
> "Plant a Tree, Go Green"
>
>  utm_source=link_campaign=sig-email_content=webmail>
> Virus-free.
> www.avg.com
>  utm_source=link_campaign=sig-email_content=webmail>
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>

Re: Newbie question about why represent timestamps as "float" values

[Erick Erickson]

Hold it. "date", "tdate", "pdate" _are_ primitive types. Under the
covers date/tdate are just a tlong type, newer Solrs have a "pdate"
which is a point numeric type. All that these types do is some parsing
up front so you can send human-readable data (and get it back). But
under the covers it's still a primitive.

And the idea of making it a float is _certainly_ worse than a long.
Last time I checked, floats were more expensive to work with than
longs. If this  was done for "efficiency" it wasn't done correctly.

It's vaguely possible that if this was done for efficiency, it was
done lng ago when dates could be strings. Certainly there's a
performance argument there, but that hasn't been the case for a very
long time.


Erick

On Tue, Oct 10, 2017 at 2:24 AM, Michael Kuhlmann  wrote:
> While you're generally right, in this case it might make sense to stick
> to a primitive type.
>
> I see "unixtime" as a technical information, probably from
> System.currentTimeMillis(). As long as it's not used as a "real world"
> date but only for sorting based on latest updates, or chosing which
> document is more recent, it's totally okay to index it as a long value.
>
> But definitely not as a float.
>
> -Michael
>
> Am 10.10.2017 um 10:55 schrieb alessandro.benedetti:
>> There was time ago a Solr installation which had the same problem, and the
>> author explained me that the choice was made for performance reasons.
>> Apparently he was sure that handling everything as primitive types would
>> give a boost to the Solr searching/faceting performance.
>> I never agreed ( and one of the reasons is that you need to transform back
>> from float to dates to actually render them in a readable format).
>>
>> Furthermore I tend to rely on standing on the shoulders of giants, so if a
>> community ( not just a single developer) spent time implementing a date type
>> ( with the different available implementations) to manage specifically date
>> information, I tend to thrust them and believe that the best approach to
>> manage dates is to use that ad hoc date type ( in its variants, depending on
>> the use cases).
>>
>> As a plus, using the right data type gives you immense power in debugging
>> and understanding better your data.
>> For proper maintenance , it is another good reason to stick with standards.
>>
>>
>>
>> -
>> ---
>> Alessandro Benedetti
>> Search Consultant, R Software Engineer, Director
>> Sease Ltd. - www.sease.io
>> --
>> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>>
>

Re: Newbie question about why represent timestamps as "float" values

[Michael Kuhlmann]

While you're generally right, in this case it might make sense to stick
to a primitive type.

I see "unixtime" as a technical information, probably from
System.currentTimeMillis(). As long as it's not used as a "real world"
date but only for sorting based on latest updates, or chosing which
document is more recent, it's totally okay to index it as a long value.

But definitely not as a float.

-Michael

Am 10.10.2017 um 10:55 schrieb alessandro.benedetti:
> There was time ago a Solr installation which had the same problem, and the
> author explained me that the choice was made for performance reasons.
> Apparently he was sure that handling everything as primitive types would
> give a boost to the Solr searching/faceting performance.
> I never agreed ( and one of the reasons is that you need to transform back
> from float to dates to actually render them in a readable format).
> 
> Furthermore I tend to rely on standing on the shoulders of giants, so if a
> community ( not just a single developer) spent time implementing a date type
> ( with the different available implementations) to manage specifically date
> information, I tend to thrust them and believe that the best approach to
> manage dates is to use that ad hoc date type ( in its variants, depending on
> the use cases).
> 
> As a plus, using the right data type gives you immense power in debugging
> and understanding better your data.
> For proper maintenance , it is another good reason to stick with standards.
> 
> 
> 
> -
> ---
> Alessandro Benedetti
> Search Consultant, R Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> 

Re: Newbie question about why represent timestamps as "float" values

[alessandro.benedetti]

There was time ago a Solr installation which had the same problem, and the
author explained me that the choice was made for performance reasons.
Apparently he was sure that handling everything as primitive types would
give a boost to the Solr searching/faceting performance.
I never agreed ( and one of the reasons is that you need to transform back
from float to dates to actually render them in a readable format).

Furthermore I tend to rely on standing on the shoulders of giants, so if a
community ( not just a single developer) spent time implementing a date type
( with the different available implementations) to manage specifically date
information, I tend to thrust them and believe that the best approach to
manage dates is to use that ad hoc date type ( in its variants, depending on
the use cases).

As a plus, using the right data type gives you immense power in debugging
and understanding better your data.
For proper maintenance , it is another good reason to stick with standards.



-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Newbie question about why represent timestamps as "float" values

[Erick Erickson]

What Hoss said, and in addition somewhere some
custom code has to be translating things back and
forth. For dates, Solr wants -MM-DDTHH:MM:SSZ
as a date string it knows how to deal with. That simply
couldn't parse as a float type so there's some custom
code that transforms dates into a float at ingest
time and converts from float to something recognizable
as a date on output.



On Mon, Oct 9, 2017 at 2:06 PM, Chris Hostetter
 wrote:
>
> : Here is my question.  In schema.xml, there is this field:
> :
> : 
> :
> : Question:  why is this declared as a float datatype?  I'm just looking
> : for an explanation of what is there – any changes come later, after I
> : understand things better.
>
> You would hvae to ask the creator of that schema.xml file why they made
> that choice ... to the best of my knowledge, no sample/example schema that
> has ever shipped with any version of solr has ever included a "unixdate"
> field -- let alone one that suggested "float" would be a logically correct
> data type for storing that type of information.
>
>
> -Hoss
> http://www.lucidworks.com/

Re: Newbie question about why represent timestamps as "float" values

[Chris Hostetter]

: Here is my question.  In schema.xml, there is this field:
: 
: 
: 
: Question:  why is this declared as a float datatype?  I'm just looking 
: for an explanation of what is there – any changes come later, after I 
: understand things better.

You would hvae to ask the creator of that schema.xml file why they made 
that choice ... to the best of my knowledge, no sample/example schema that 
has ever shipped with any version of solr has ever included a "unixdate" 
field -- let alone one that suggested "float" would be a logically correct 
data type for storing that type of information.


-Hoss
http://www.lucidworks.com/

Re: newbie question re solr.PatternReplaceFilterFactory

[Erick Erickson]

First use PatternReplaceCharFilterFactory. The difference is that
PatternReplaceCharFilterFactoryworks on the entire input whereas
PatternReplaceFilterFactory works only on the tokens emitted by the
tokenizer. Concrete example using WhitespeceTokenizerFactory would be
this [is some ] text
PatternReplaceFilterFactory would see 5 tokens, "this", "[is", "some",
"]", and "text". So it would be very hard to do what you want.

patternReplaceCharFilterFactory will see the entire input as one
string and operate on it, _then" send it through the tokenizer.

And also don't be fooled by the fact that the _stored_ data will still
contain the removed words. So when you get the doc back from solr
you'll see the original input, brackets and all. In the above example,
if you returned the field you'd still see

this [is some ] text

when the doc matched. This doc would be found when searching for
"this" or "text", but _not_ when searching for "is" or "some".

You want some pattern like
  

Best,
Erick

On Wed, May 10, 2017 at 6:08 PM, Michael Tobias  wrote:
> I am sure this is very simple but I cannot get the pattern right.
>
> How can I use solr.PatternReplaceFilterFactory to remove all words in 
> brackets from being indexed?
>
> eg [ignore this]
>
> thanks
>
> Michael
>

Re: newbie question

[John Bickerstaff]

the /solr is a "chroot" -- if used, everything for solr goes into
zookeeper's /solr "directory"

It isn't required, but is very useful for keeping things separated.  I use
it to handle different Solr versions for upgrading (/solr5_4 or /solr6_2)

If not used, everything you put into Zookeeper (some microservices, Kafka,
etc) will all end up at the "root" and it'll be a cluster...

On Wed, Sep 7, 2016 at 1:26 PM, Erick Erickson 
wrote:

> Well, first off the ZK ensemble string is usually specified as
> dayrhegapd016.enterprisenet.org:2181,host2:2181,host3:2181/solr
> (note that the /solr is only at the end, not every node).
>
> Second, I always get confused whether the /solr is necessary or not.
>
> Again, though, the Cloudera user's list is probably a place to get better
> answers.
>
> Best,
> Erick
>
> On Wed, Sep 7, 2016 at 12:15 PM, Darshan Pandya 
> wrote:
> > Thank Erik,
> > So seems like the problem is that when I upload the configs to zookeeper
> > and then inspect zookeeper-client and ls /solr/configs it is showing to
> be
> > empty.
> >
> > I executed the following command to upload the config
> >
> > solrctl --zk
> > dayrhegapd016.enterprisenet.org:2181/solr,host2:2181/solr,
> host3:2181/solr
> > --solr host4:8983/solr/ instancedir --create config1 $HOME/config1/
> >
> >
> > Zookeeper-client ls /solr/configs/
> >
> > does not show this configuration present there.
> >
> >
> >
> > On Wed, Sep 7, 2016 at 2:02 PM, Erick Erickson 
> > wrote:
> >
> >> I'm a bit rusty on solrctl (and you might get faster/more up-to-date
> >> responses on the Cloudera lists). But to create a collection, you
> >> first need to have uploaded the configs to Zookeeper, things like
> >> schema.xml, solrconfig.xml etc. I forget
> >> what the solrctl command is, but something like "upconfig" IIRC.
> >>
> >> Once that's done, you either
> >> 1> specify the collection name exactly the same as the config name
> >> you uploaded
> >> or
> >> 2> use one of the other parameters to tell collectionX to use configsetY
> >> with the collection create command. solrctl help should show you all
> these
> >> options...
> >>
> >> Best,
> >> Erick
> >>
> >> On Wed, Sep 7, 2016 at 11:43 AM, Darshan Pandya <
> darshanpan...@gmail.com>
> >> wrote:
> >> > Gonzalo,
> >> > Thanks for responding,
> >> > executed the parameters you suggested, it still shows me the same
> error.
> >> > Sincerely,
> >> > darshan
> >> >
> >> > On Wed, Sep 7, 2016 at 1:13 PM, Gonzalo Rodriguez <
> >> > grodrig...@searchtechnologies.com> wrote:
> >> >
> >> >> Hi Darshan,
> >> >>
> >> >> It looks like you are listing the instanceDir's name twice in the
> create
> >> >> collection command, it should be
> >> >>
> >> >> $ solrctl --zk  host:2181/solr --solr host:8983/solr/ collection
> >> --create
> >> >> Catalog_search_index -s 10 -c Catalog_search_index
> >> >>
> >> >> Without the extra ". Catalog_search_index" at the end. Also, because
> >> your
> >> >> new collection's name is the same as the instanceDir's, you could
> just
> >> omit
> >> >> that parameter and it should work ok.
> >> >>
> >> >> Try that and see if it works.
> >> >>
> >> >> Good luck,
> >> >> Gonzalo
> >> >>
> >> >> -Original Message-
> >> >> From: Darshan Pandya [mailto:darshanpan...@gmail.com]
> >> >> Sent: Wednesday, September 7, 2016 12:02 PM
> >> >> To: solr-user@lucene.apache.org
> >> >> Subject: newbie question
> >> >>
> >> >> hello,
> >> >>
> >> >> I am using solr cloud with cloudera. When I try to create a
> collection,
> >> it
> >> >> fails with the following error.
> >> >> Any hints / answers will be helpful.
> >> >>
> >> >>
> >> >> $ solrctl --zk  host:2181/solr instancedir --list
> >> >>
> >> >> Catalog_search_index
> >> >>
> >> >> $ solrctl --zk  shot:2181/solr --solr host:8983/solr/ collection
> >> --create
> >> >> Catalog_search_index -s 10 -c Catalog_search_index.Catalog_
> search_index
> >> >>
> >> >> Error: A call to SolrCloud WEB APIs failed: HTTP/1.1 200 OK
> >> >>
> >> >> Server: Apache-Coyote/1.1
> >> >>
> >> >> Content-Type: application/xml;charset=UTF-8
> >> >>
> >> >> Transfer-Encoding: chunked
> >> >>
> >> >> Date: Wed, 07 Sep 2016 17:58:13 GMT
> >> >>
> >> >>
> >> >> 
> >> >>
> >> >>
> >> >> 
> >> >>
> >> >>
> >> >> 
> >> >>
> >> >> 
> >> >>
> >> >> 0
> >> >>
> >> >> 
> >> >>
> >> >> 1165
> >> >>
> >> >> 
> >> >>
> >> >> 
> >> >>
> >> >> 
> >> >>
> >> >> org.apache.solr.client.solrj.impl.HttpSolrServer$
> >> RemoteSolrException:Error
> >> >> CREATEing SolrCore 'Catalog_search_index_shard1_replica1': Unable to
> >> >> create core [Catalog_search_index_shard1_replica1] Caused by:
> Specified
> >> >> config does not exist in ZooKeeper:Catalog_search_
> >> >> index.dataCatalog_search_index
> >> >>
> >> >>
> >> >> --
> >> >> Sincerely,
> >> >> Darshan
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Sincerely,
> >> > Darshan
> >>
> >
> >
> >
> > --
> > Sincerely,
> > Darshan
>

Re: newbie question

[Darshan Pandya]

Many Thanks! I will move this to a cloudera list.

On Wed, Sep 7, 2016 at 2:26 PM, Erick Erickson 
wrote:

> Well, first off the ZK ensemble string is usually specified as
> dayrhegapd016.enterprisenet.org:2181,host2:2181,host3:2181/solr
> (note that the /solr is only at the end, not every node).
>
> Second, I always get confused whether the /solr is necessary or not.
>
> Again, though, the Cloudera user's list is probably a place to get better
> answers.
>
> Best,
> Erick
>
> On Wed, Sep 7, 2016 at 12:15 PM, Darshan Pandya 
> wrote:
> > Thank Erik,
> > So seems like the problem is that when I upload the configs to zookeeper
> > and then inspect zookeeper-client and ls /solr/configs it is showing to
> be
> > empty.
> >
> > I executed the following command to upload the config
> >
> > solrctl --zk
> > dayrhegapd016.enterprisenet.org:2181/solr,host2:2181/solr,
> host3:2181/solr
> > --solr host4:8983/solr/ instancedir --create config1 $HOME/config1/
> >
> >
> > Zookeeper-client ls /solr/configs/
> >
> > does not show this configuration present there.
> >
> >
> >
> > On Wed, Sep 7, 2016 at 2:02 PM, Erick Erickson 
> > wrote:
> >
> >> I'm a bit rusty on solrctl (and you might get faster/more up-to-date
> >> responses on the Cloudera lists). But to create a collection, you
> >> first need to have uploaded the configs to Zookeeper, things like
> >> schema.xml, solrconfig.xml etc. I forget
> >> what the solrctl command is, but something like "upconfig" IIRC.
> >>
> >> Once that's done, you either
> >> 1> specify the collection name exactly the same as the config name
> >> you uploaded
> >> or
> >> 2> use one of the other parameters to tell collectionX to use configsetY
> >> with the collection create command. solrctl help should show you all
> these
> >> options...
> >>
> >> Best,
> >> Erick
> >>
> >> On Wed, Sep 7, 2016 at 11:43 AM, Darshan Pandya <
> darshanpan...@gmail.com>
> >> wrote:
> >> > Gonzalo,
> >> > Thanks for responding,
> >> > executed the parameters you suggested, it still shows me the same
> error.
> >> > Sincerely,
> >> > darshan
> >> >
> >> > On Wed, Sep 7, 2016 at 1:13 PM, Gonzalo Rodriguez <
> >> > grodrig...@searchtechnologies.com> wrote:
> >> >
> >> >> Hi Darshan,
> >> >>
> >> >> It looks like you are listing the instanceDir's name twice in the
> create
> >> >> collection command, it should be
> >> >>
> >> >> $ solrctl --zk  host:2181/solr --solr host:8983/solr/ collection
> >> --create
> >> >> Catalog_search_index -s 10 -c Catalog_search_index
> >> >>
> >> >> Without the extra ". Catalog_search_index" at the end. Also, because
> >> your
> >> >> new collection's name is the same as the instanceDir's, you could
> just
> >> omit
> >> >> that parameter and it should work ok.
> >> >>
> >> >> Try that and see if it works.
> >> >>
> >> >> Good luck,
> >> >> Gonzalo
> >> >>
> >> >> -Original Message-
> >> >> From: Darshan Pandya [mailto:darshanpan...@gmail.com]
> >> >> Sent: Wednesday, September 7, 2016 12:02 PM
> >> >> To: solr-user@lucene.apache.org
> >> >> Subject: newbie question
> >> >>
> >> >> hello,
> >> >>
> >> >> I am using solr cloud with cloudera. When I try to create a
> collection,
> >> it
> >> >> fails with the following error.
> >> >> Any hints / answers will be helpful.
> >> >>
> >> >>
> >> >> $ solrctl --zk  host:2181/solr instancedir --list
> >> >>
> >> >> Catalog_search_index
> >> >>
> >> >> $ solrctl --zk  shot:2181/solr --solr host:8983/solr/ collection
> >> --create
> >> >> Catalog_search_index -s 10 -c Catalog_search_index.Catalog_
> search_index
> >> >>
> >> >> Error: A call to SolrCloud WEB APIs failed: HTTP/1.1 200 OK
> >> >>
> >> >> Server: Apache-Coyote/1.1
> >> >>
> >> >> Content-Type: application/xml;charset=UTF-8
> >> >>
> >> >> Transfer-Encoding: chunked
> >> >>
> >> >> Date: Wed, 07 Sep 2016 17:58:13 GMT
> >> >>
> >> >>
> >> >> 
> >> >>
> >> >>
> >> >> 
> >> >>
> >> >>
> >> >> 
> >> >>
> >> >> 
> >> >>
> >> >> 0
> >> >>
> >> >> 
> >> >>
> >> >> 1165
> >> >>
> >> >> 
> >> >>
> >> >> 
> >> >>
> >> >> 
> >> >>
> >> >> org.apache.solr.client.solrj.impl.HttpSolrServer$
> >> RemoteSolrException:Error
> >> >> CREATEing SolrCore 'Catalog_search_index_shard1_replica1': Unable to
> >> >> create core [Catalog_search_index_shard1_replica1] Caused by:
> Specified
> >> >> config does not exist in ZooKeeper:Catalog_search_
> >> >> index.dataCatalog_search_index
> >> >>
> >> >>
> >> >> --
> >> >> Sincerely,
> >> >> Darshan
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Sincerely,
> >> > Darshan
> >>
> >
> >
> >
> > --
> > Sincerely,
> > Darshan
>



-- 
Sincerely,
Darshan

Re: newbie question

[Erick Erickson]

Well, first off the ZK ensemble string is usually specified as
dayrhegapd016.enterprisenet.org:2181,host2:2181,host3:2181/solr
(note that the /solr is only at the end, not every node).

Second, I always get confused whether the /solr is necessary or not.

Again, though, the Cloudera user's list is probably a place to get better
answers.

Best,
Erick

On Wed, Sep 7, 2016 at 12:15 PM, Darshan Pandya  wrote:
> Thank Erik,
> So seems like the problem is that when I upload the configs to zookeeper
> and then inspect zookeeper-client and ls /solr/configs it is showing to be
> empty.
>
> I executed the following command to upload the config
>
> solrctl --zk
> dayrhegapd016.enterprisenet.org:2181/solr,host2:2181/solr,host3:2181/solr
> --solr host4:8983/solr/ instancedir --create config1 $HOME/config1/
>
>
> Zookeeper-client ls /solr/configs/
>
> does not show this configuration present there.
>
>
>
> On Wed, Sep 7, 2016 at 2:02 PM, Erick Erickson 
> wrote:
>
>> I'm a bit rusty on solrctl (and you might get faster/more up-to-date
>> responses on the Cloudera lists). But to create a collection, you
>> first need to have uploaded the configs to Zookeeper, things like
>> schema.xml, solrconfig.xml etc. I forget
>> what the solrctl command is, but something like "upconfig" IIRC.
>>
>> Once that's done, you either
>> 1> specify the collection name exactly the same as the config name
>> you uploaded
>> or
>> 2> use one of the other parameters to tell collectionX to use configsetY
>> with the collection create command. solrctl help should show you all these
>> options...
>>
>> Best,
>> Erick
>>
>> On Wed, Sep 7, 2016 at 11:43 AM, Darshan Pandya 
>> wrote:
>> > Gonzalo,
>> > Thanks for responding,
>> > executed the parameters you suggested, it still shows me the same error.
>> > Sincerely,
>> > darshan
>> >
>> > On Wed, Sep 7, 2016 at 1:13 PM, Gonzalo Rodriguez <
>> > grodrig...@searchtechnologies.com> wrote:
>> >
>> >> Hi Darshan,
>> >>
>> >> It looks like you are listing the instanceDir's name twice in the create
>> >> collection command, it should be
>> >>
>> >> $ solrctl --zk  host:2181/solr --solr host:8983/solr/ collection
>> --create
>> >> Catalog_search_index -s 10 -c Catalog_search_index
>> >>
>> >> Without the extra ". Catalog_search_index" at the end. Also, because
>> your
>> >> new collection's name is the same as the instanceDir's, you could just
>> omit
>> >> that parameter and it should work ok.
>> >>
>> >> Try that and see if it works.
>> >>
>> >> Good luck,
>> >> Gonzalo
>> >>
>> >> -Original Message-
>> >> From: Darshan Pandya [mailto:darshanpan...@gmail.com]
>> >> Sent: Wednesday, September 7, 2016 12:02 PM
>> >> To: solr-user@lucene.apache.org
>> >> Subject: newbie question
>> >>
>> >> hello,
>> >>
>> >> I am using solr cloud with cloudera. When I try to create a collection,
>> it
>> >> fails with the following error.
>> >> Any hints / answers will be helpful.
>> >>
>> >>
>> >> $ solrctl --zk  host:2181/solr instancedir --list
>> >>
>> >> Catalog_search_index
>> >>
>> >> $ solrctl --zk  shot:2181/solr --solr host:8983/solr/ collection
>> --create
>> >> Catalog_search_index -s 10 -c Catalog_search_index.Catalog_search_index
>> >>
>> >> Error: A call to SolrCloud WEB APIs failed: HTTP/1.1 200 OK
>> >>
>> >> Server: Apache-Coyote/1.1
>> >>
>> >> Content-Type: application/xml;charset=UTF-8
>> >>
>> >> Transfer-Encoding: chunked
>> >>
>> >> Date: Wed, 07 Sep 2016 17:58:13 GMT
>> >>
>> >>
>> >> 
>> >>
>> >>
>> >> 
>> >>
>> >>
>> >> 
>> >>
>> >> 
>> >>
>> >> 0
>> >>
>> >> 
>> >>
>> >> 1165
>> >>
>> >> 
>> >>
>> >> 
>> >>
>> >> 
>> >>
>> >> org.apache.solr.client.solrj.impl.HttpSolrServer$
>> RemoteSolrException:Error
>> >> CREATEing SolrCore 'Catalog_search_index_shard1_replica1': Unable to
>> >> create core [Catalog_search_index_shard1_replica1] Caused by: Specified
>> >> config does not exist in ZooKeeper:Catalog_search_
>> >> index.dataCatalog_search_index
>> >>
>> >>
>> >> --
>> >> Sincerely,
>> >> Darshan
>> >>
>> >
>> >
>> >
>> > --
>> > Sincerely,
>> > Darshan
>>
>
>
>
> --
> Sincerely,
> Darshan

Re: newbie question

[Darshan Pandya]

Thank Erik,
So seems like the problem is that when I upload the configs to zookeeper
and then inspect zookeeper-client and ls /solr/configs it is showing to be
empty.

I executed the following command to upload the config

solrctl --zk
dayrhegapd016.enterprisenet.org:2181/solr,host2:2181/solr,host3:2181/solr
--solr host4:8983/solr/ instancedir --create config1 $HOME/config1/


Zookeeper-client ls /solr/configs/

does not show this configuration present there.



On Wed, Sep 7, 2016 at 2:02 PM, Erick Erickson 
wrote:

> I'm a bit rusty on solrctl (and you might get faster/more up-to-date
> responses on the Cloudera lists). But to create a collection, you
> first need to have uploaded the configs to Zookeeper, things like
> schema.xml, solrconfig.xml etc. I forget
> what the solrctl command is, but something like "upconfig" IIRC.
>
> Once that's done, you either
> 1> specify the collection name exactly the same as the config name
> you uploaded
> or
> 2> use one of the other parameters to tell collectionX to use configsetY
> with the collection create command. solrctl help should show you all these
> options...
>
> Best,
> Erick
>
> On Wed, Sep 7, 2016 at 11:43 AM, Darshan Pandya 
> wrote:
> > Gonzalo,
> > Thanks for responding,
> > executed the parameters you suggested, it still shows me the same error.
> > Sincerely,
> > darshan
> >
> > On Wed, Sep 7, 2016 at 1:13 PM, Gonzalo Rodriguez <
> > grodrig...@searchtechnologies.com> wrote:
> >
> >> Hi Darshan,
> >>
> >> It looks like you are listing the instanceDir's name twice in the create
> >> collection command, it should be
> >>
> >> $ solrctl --zk  host:2181/solr --solr host:8983/solr/ collection
> --create
> >> Catalog_search_index -s 10 -c Catalog_search_index
> >>
> >> Without the extra ". Catalog_search_index" at the end. Also, because
> your
> >> new collection's name is the same as the instanceDir's, you could just
> omit
> >> that parameter and it should work ok.
> >>
> >> Try that and see if it works.
> >>
> >> Good luck,
> >> Gonzalo
> >>
> >> -Original Message-
> >> From: Darshan Pandya [mailto:darshanpan...@gmail.com]
> >> Sent: Wednesday, September 7, 2016 12:02 PM
> >> To: solr-user@lucene.apache.org
> >> Subject: newbie question
> >>
> >> hello,
> >>
> >> I am using solr cloud with cloudera. When I try to create a collection,
> it
> >> fails with the following error.
> >> Any hints / answers will be helpful.
> >>
> >>
> >> $ solrctl --zk  host:2181/solr instancedir --list
> >>
> >> Catalog_search_index
> >>
> >> $ solrctl --zk  shot:2181/solr --solr host:8983/solr/ collection
> --create
> >> Catalog_search_index -s 10 -c Catalog_search_index.Catalog_search_index
> >>
> >> Error: A call to SolrCloud WEB APIs failed: HTTP/1.1 200 OK
> >>
> >> Server: Apache-Coyote/1.1
> >>
> >> Content-Type: application/xml;charset=UTF-8
> >>
> >> Transfer-Encoding: chunked
> >>
> >> Date: Wed, 07 Sep 2016 17:58:13 GMT
> >>
> >>
> >> 
> >>
> >>
> >> 
> >>
> >>
> >> 
> >>
> >> 
> >>
> >> 0
> >>
> >> 
> >>
> >> 1165
> >>
> >> 
> >>
> >> 
> >>
> >> 
> >>
> >> org.apache.solr.client.solrj.impl.HttpSolrServer$
> RemoteSolrException:Error
> >> CREATEing SolrCore 'Catalog_search_index_shard1_replica1': Unable to
> >> create core [Catalog_search_index_shard1_replica1] Caused by: Specified
> >> config does not exist in ZooKeeper:Catalog_search_
> >> index.dataCatalog_search_index
> >>
> >>
> >> --
> >> Sincerely,
> >> Darshan
> >>
> >
> >
> >
> > --
> > Sincerely,
> > Darshan
>



-- 
Sincerely,
Darshan

Re: newbie question

[Erick Erickson]

I'm a bit rusty on solrctl (and you might get faster/more up-to-date
responses on the Cloudera lists). But to create a collection, you
first need to have uploaded the configs to Zookeeper, things like
schema.xml, solrconfig.xml etc. I forget
what the solrctl command is, but something like "upconfig" IIRC.

Once that's done, you either
1> specify the collection name exactly the same as the config name
you uploaded
or
2> use one of the other parameters to tell collectionX to use configsetY
with the collection create command. solrctl help should show you all these
options...

Best,
Erick

On Wed, Sep 7, 2016 at 11:43 AM, Darshan Pandya  wrote:
> Gonzalo,
> Thanks for responding,
> executed the parameters you suggested, it still shows me the same error.
> Sincerely,
> darshan
>
> On Wed, Sep 7, 2016 at 1:13 PM, Gonzalo Rodriguez <
> grodrig...@searchtechnologies.com> wrote:
>
>> Hi Darshan,
>>
>> It looks like you are listing the instanceDir's name twice in the create
>> collection command, it should be
>>
>> $ solrctl --zk  host:2181/solr --solr host:8983/solr/ collection --create
>> Catalog_search_index -s 10 -c Catalog_search_index
>>
>> Without the extra ". Catalog_search_index" at the end. Also, because your
>> new collection's name is the same as the instanceDir's, you could just omit
>> that parameter and it should work ok.
>>
>> Try that and see if it works.
>>
>> Good luck,
>> Gonzalo
>>
>> -Original Message-
>> From: Darshan Pandya [mailto:darshanpan...@gmail.com]
>> Sent: Wednesday, September 7, 2016 12:02 PM
>> To: solr-user@lucene.apache.org
>> Subject: newbie question
>>
>> hello,
>>
>> I am using solr cloud with cloudera. When I try to create a collection, it
>> fails with the following error.
>> Any hints / answers will be helpful.
>>
>>
>> $ solrctl --zk  host:2181/solr instancedir --list
>>
>> Catalog_search_index
>>
>> $ solrctl --zk  shot:2181/solr --solr host:8983/solr/ collection --create
>> Catalog_search_index -s 10 -c Catalog_search_index.Catalog_search_index
>>
>> Error: A call to SolrCloud WEB APIs failed: HTTP/1.1 200 OK
>>
>> Server: Apache-Coyote/1.1
>>
>> Content-Type: application/xml;charset=UTF-8
>>
>> Transfer-Encoding: chunked
>>
>> Date: Wed, 07 Sep 2016 17:58:13 GMT
>>
>>
>> 
>>
>>
>> 
>>
>>
>> 
>>
>> 
>>
>> 0
>>
>> 
>>
>> 1165
>>
>> 
>>
>> 
>>
>> 
>>
>> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
>> CREATEing SolrCore 'Catalog_search_index_shard1_replica1': Unable to
>> create core [Catalog_search_index_shard1_replica1] Caused by: Specified
>> config does not exist in ZooKeeper:Catalog_search_
>> index.dataCatalog_search_index
>>
>>
>> --
>> Sincerely,
>> Darshan
>>
>
>
>
> --
> Sincerely,
> Darshan

Re: newbie question

[Darshan Pandya]

Gonzalo,
Thanks for responding,
executed the parameters you suggested, it still shows me the same error.
Sincerely,
darshan

On Wed, Sep 7, 2016 at 1:13 PM, Gonzalo Rodriguez <
grodrig...@searchtechnologies.com> wrote:

> Hi Darshan,
>
> It looks like you are listing the instanceDir's name twice in the create
> collection command, it should be
>
> $ solrctl --zk  host:2181/solr --solr host:8983/solr/ collection --create
> Catalog_search_index -s 10 -c Catalog_search_index
>
> Without the extra ". Catalog_search_index" at the end. Also, because your
> new collection's name is the same as the instanceDir's, you could just omit
> that parameter and it should work ok.
>
> Try that and see if it works.
>
> Good luck,
> Gonzalo
>
> -Original Message-
> From: Darshan Pandya [mailto:darshanpan...@gmail.com]
> Sent: Wednesday, September 7, 2016 12:02 PM
> To: solr-user@lucene.apache.org
> Subject: newbie question
>
> hello,
>
> I am using solr cloud with cloudera. When I try to create a collection, it
> fails with the following error.
> Any hints / answers will be helpful.
>
>
> $ solrctl --zk  host:2181/solr instancedir --list
>
> Catalog_search_index
>
> $ solrctl --zk  shot:2181/solr --solr host:8983/solr/ collection --create
> Catalog_search_index -s 10 -c Catalog_search_index.Catalog_search_index
>
> Error: A call to SolrCloud WEB APIs failed: HTTP/1.1 200 OK
>
> Server: Apache-Coyote/1.1
>
> Content-Type: application/xml;charset=UTF-8
>
> Transfer-Encoding: chunked
>
> Date: Wed, 07 Sep 2016 17:58:13 GMT
>
>
> 
>
>
> 
>
>
> 
>
> 
>
> 0
>
> 
>
> 1165
>
> 
>
> 
>
> 
>
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'Catalog_search_index_shard1_replica1': Unable to
> create core [Catalog_search_index_shard1_replica1] Caused by: Specified
> config does not exist in ZooKeeper:Catalog_search_
> index.dataCatalog_search_index
>
>
> --
> Sincerely,
> Darshan
>



-- 
Sincerely,
Darshan

RE: newbie question

[Gonzalo Rodriguez]

Hi Darshan,

It looks like you are listing the instanceDir's name twice in the create 
collection command, it should be

$ solrctl --zk  host:2181/solr --solr host:8983/solr/ collection --create 
Catalog_search_index -s 10 -c Catalog_search_index

Without the extra ". Catalog_search_index" at the end. Also, because your new 
collection's name is the same as the instanceDir's, you could just omit that 
parameter and it should work ok.

Try that and see if it works.

Good luck,
Gonzalo

-Original Message-
From: Darshan Pandya [mailto:darshanpan...@gmail.com] 
Sent: Wednesday, September 7, 2016 12:02 PM
To: solr-user@lucene.apache.org
Subject: newbie question

hello,

I am using solr cloud with cloudera. When I try to create a collection, it 
fails with the following error.
Any hints / answers will be helpful.


$ solrctl --zk  host:2181/solr instancedir --list

Catalog_search_index

$ solrctl --zk  shot:2181/solr --solr host:8983/solr/ collection --create 
Catalog_search_index -s 10 -c Catalog_search_index.Catalog_search_index

Error: A call to SolrCloud WEB APIs failed: HTTP/1.1 200 OK

Server: Apache-Coyote/1.1

Content-Type: application/xml;charset=UTF-8

Transfer-Encoding: chunked

Date: Wed, 07 Sep 2016 17:58:13 GMT












0



1165







org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
CREATEing SolrCore 'Catalog_search_index_shard1_replica1': Unable to create 
core [Catalog_search_index_shard1_replica1] Caused by: Specified config does 
not exist in ZooKeeper:Catalog_search_index.dataCatalog_search_index


--
Sincerely,
Darshan

Re: [Newbie question] in SOLR 5, would I have a "master-to-slave" relationship for two servers?

[Erick Erickson]

To pile on to Chris' comment. In the M/S situation
you describe, all the query traffic goes to the slave.

True, this relieves the slave from doing the work of
indexing, but it _also_ prevents the master from
answering queries. So going to SolrCloud trades
off indexing on _both_ machines to also querying on
_both_ machines.

And this doesn't even take into account the issues
involved in recovering if one or the other (especially
the master) goes down, which is automatically
handled in SolrCloud.

Add to that the fact that memory management is
_very_ significantly improved starting with Solr
4x (see: 
https://lucidworks.com/blog/2012/04/06/memory-comparisons-between-solr-3x-and-trunk/)
and my claim is that you are _far_ better off
using SolrCloud than M/S in 5x.

As always, YMMV of course.

Best,
Erick


On Thu, Nov 5, 2015 at 1:12 PM, Chris Hostetter
 wrote:
>
> : The database of server 2 is considered the "master" and it is replicated
> : regularly to server 1, the "slave".
> :
> : The advantage is the responsiveness of server 1 is not impacted with server
> : 2 gets busy with lots of indexing.
> :
> : QUESTION: When deploying a SOLR 5 setup, do I set things up the same way?
> : Or do I cluster bother servers together into one "cloud"?   That is, in
> : SOLR 5, how do I ensure the indexing process will not impact the
> : performance of the web app?
>
> There is nothing preventing you from using a master slave setup with Solr
> 5...
>
> https://cwiki.apache.org/confluence/display/solr/Index+Replication
>
> ...however if you do so you have to take responsibility for the same
> risks/tradeoffs that existed with this type of setup in Solr 3...
>
> 1) if the "query slave" goes down, you can't serve quiers w/o manually
> redirecting traffic to your "indexing master"
>
> 2) if the "indexing master" goes down you can't accept index updates w/o
> manually redirecting update to your "query slave" -- and manually
> rectifying the descrepencies if/when your master comes back online.
>
>
> When using a cloud based setup these types of problems go away because
> there is no single "master", clients can send updates/queries to any node
> (and if you use SolrJ your clients will be "ZK aware" and know
> automatically if/when a node is down or new nodes are added) ...
> many people concerned about performance/reliability consider these
> benefits more important then the risks/tradeoffs of performance impacts of
> indexing directy to nodes that are serving queries -- especially with
> other NRT (Near Real Time) related improvements to Solr over the years
> (Soft Commits, DocValues instead of FieldCache, etc...)
>
>
> -Hoss
> http://www.lucidworks.com/

Re: [Newbie question] what is a "core" and are they different from 3.x to 5.x ?

[Chris Hostetter]

: I can see there is something called a "core" ... it appears there can be
: many cores for a single SOLR server.
: 
: Can someone "explain like I'm five" -- what is a core?

https://cwiki.apache.org/confluence/display/solr/Solr+Cores+and+solr.xml

"In Solr, the term core is used to refer to a single index and associated 
transaction log and configuration files (including schema.xml and 
solrconfig.xml, among others). Your Solr installation can have multiple 
cores if needed, which allows you to index data with different structures 
in the same server, and maintain more control over how your data is 
presented to different audiences."

: And how do "cores" differ from 3.x to 5.x.


The only fundemental differences between "cores" in Solr 3.x vs 5.x are:

1) in 3.x there was a concept known as the "default core" (if you didn't 
explicitly use multiple cores)  with 5.x every request (updates or 
queries) must be to an explicit core (or collection)

2) when using SolrCloud in 5.x, you should think (logically) in terms of 
the higher level concept of "collections" which (depending on the settings 
when the collection is created) may be *implemented* by multiple cores 
that are managed under the covers for you...

https://cwiki.apache.org/confluence/display/solr/SolrCloud
https://cwiki.apache.org/confluence/display/solr/Nodes%2C+Cores%2C+Clusters+and+Leaders


-Hoss
http://www.lucidworks.com/

Re: [Newbie question] in SOLR 5, would I have a "master-to-slave" relationship for two servers?

[Chris Hostetter]

: The database of server 2 is considered the "master" and it is replicated
: regularly to server 1, the "slave".
: 
: The advantage is the responsiveness of server 1 is not impacted with server
: 2 gets busy with lots of indexing.
: 
: QUESTION: When deploying a SOLR 5 setup, do I set things up the same way?
: Or do I cluster bother servers together into one "cloud"?   That is, in
: SOLR 5, how do I ensure the indexing process will not impact the
: performance of the web app?

There is nothing preventing you from using a master slave setup with Solr 
5...

https://cwiki.apache.org/confluence/display/solr/Index+Replication

...however if you do so you have to take responsibility for the same 
risks/tradeoffs that existed with this type of setup in Solr 3...

1) if the "query slave" goes down, you can't serve quiers w/o manually 
redirecting traffic to your "indexing master"

2) if the "indexing master" goes down you can't accept index updates w/o 
manually redirecting update to your "query slave" -- and manually 
rectifying the descrepencies if/when your master comes back online.


When using a cloud based setup these types of problems go away because 
there is no single "master", clients can send updates/queries to any node 
(and if you use SolrJ your clients will be "ZK aware" and know 
automatically if/when a node is down or new nodes are added) ... 
many people concerned about performance/reliability consider these 
benefits more important then the risks/tradeoffs of performance impacts of 
indexing directy to nodes that are serving queries -- especially with 
other NRT (Near Real Time) related improvements to Solr over the years 
(Soft Commits, DocValues instead of FieldCache, etc...)


-Hoss
http://www.lucidworks.com/

Re: Newbie Question: Master Index or 100s Small Index

[Toke Eskildsen]

On Wed, 2014-03-19 at 11:55 +0100, Colin R wrote:
 We run a central database of 14M (and growing) photos with dates, captions,
 keywords, etc. 
 
 We currently upgrading from old Lucene Servers to latest Solr running with a
 couple of dedicated  servers (6 core, 36GB, 500SSD). Planning on using Solr
 Cloud.

What hardware are your past experiences based on? If they have less
cores, lower memory and spinning drives, I foresee that your question
can be reduced to which architecture you prefer from a logistic point of
view, rather than performance.

 We take in thousands of changes each day (big and small) so indexing may be
 a bigger problem than searching.

Thousands of updates in a day is a very low number. Do you have hard
requirements for update time, perform heavy faceting or do anything
special for this to be a cause of concern?

 Is it quicker/better to just have one big 14M index and filter the
 complexities for each website or is it better to still maintain hundreds of
 indexes so we are searching smaller one.

All else being equal, a search in a specific small index will be faster
than filtering on the large one. But as we know, all else is never
equal. A 14M document index in itself is not really a challenge for
Lucene/Solr, but this depends a lot on your specific setup. How large is
the 14M index in terms of bytes?

 Bear in mind, we get thousands of changes a day PLUS very busy search servers.

How many queries/second are we talking about here? What is a typical
query (faceting, grouping, special processing...)?

Regards,
Toke Eskildsen, State and University Library, Denmark

Re: Newbie Question: Master Index or 100s Small Index

[Colin R]

Hi Toke

Thanks for replying.

My question is really regarding index architecture. One big or many small
(with merged big ones)

We probably get 5-10K photos added each day. Others are updated, some are
deleted.

Updates need to happen quite fast (e.g. within minutes of our Databases
receiving them).

In terms of bytes, each photo has a up to 1.5KB of data.

Special requirements are search by date range, text, date range and text.
Plus some boolean filtering. All results can be sorted by date or filename.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Newbie-Question-Master-Index-or-100s-Small-Index-tp4125407p4125429.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie Question: Master Index or 100s Small Index

[Toke Eskildsen]

On Wed, 2014-03-19 at 13:28 +0100, Colin R wrote:
 My question is really regarding index architecture. One big or many small
 (with merged big ones)

One difference is that having a single index/collection gives you better
ranked searches within each collection. If you only use date/filename
sorting, that is of course irrelevant.

 In terms of bytes, each photo has a up to 1.5KB of data.

So about 20GB for the full index?

 Special requirements are search by date range, text, date range and text.
 Plus some boolean filtering. All results can be sorted by date or filename.

With no faceting, grouping or similar aggregating processing,
(re)opening of an index searcher should be very fast. The only thing
that takes a moment is the initial date or filename sorting. Asking for
minute-level data updates is thus very modest. With the information you
have given, you could aim for a few seconds.

None of the things you have said gives any cause for concern about
performance and even though you have an existing system running and is
upgrading to a presumably faster one, you sound concerned. Do you
currently have performance problems, and if so, what is your current
hardware?

- Toke Eskildsen, State and University Library, Denmark

Re: Newbie Question: Master Index or 100s Small Index

[Colin R]

Hi Toke

Our current configuration Lucene 2.(something) with RAILO/CFML app server.

10K drives, Quad Core, 16GB, Two servers. But the indexing and searching are
starting to fail and our developer is no longer with us so it is quicker to
rebuild than fix all the code.

Our existing config is lots of indexes with merges into the larger ones.

They are still running very fast but indexing is causing us issues.

Thanks



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Newbie-Question-Master-Index-or-100s-Small-Index-tp4125407p4125447.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie Question: Master Index or 100s Small Index

[Erick Erickson]

Oh My. 2(something) is ancient, I second your move
to scrap the current situation and start over. I'm
really curious what the _reason_ for such a complex
setup are/were.

I second Toke's comments. This is actually
quite small by modern Solr/Lucene standards.

Personally I would index them all to a single index,
include something like a 'source' field that allowed
one to restrict the returned documents by a filter
query (fq) clause.

Toke makes the point that you will get subtly different
search results because the tf/idf calculations are
slightly different across your entire corpus than
within various sub-sections, but I suspect that you
won't notice it. Test and see, you can change later.

One thing to look at is the new hard/soft commit
distinction, see:
http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

The short form is you want to define your hard
autocommit to be fairly short (maybe 1 minute?)
with openSearcher=false for durability and your
soft commit whatever latency you need for being
able to search the newly-added docs.

I don't know how you're feeding docs to Solr, but
if you're using the ExtractingRequestHandler,
you are
1 transmitting the entire document over the wire,
only to throw most of it away. I'm guessing your 1.5K
of data is just a few percent of the total file size.
2 you're putting the extraction work on the same
box running Solr.

If that machine is overloaded, consider moving the Tika
processing over to one or more clients and only
sending the data you actually want to index over to Solr,
See:
http://searchhub.org/2012/02/14/indexing-with-solrj/

Best,
Erick

On Wed, Mar 19, 2014 at 7:02 AM, Colin R colin.russ...@dasmail.co.uk wrote:
 Hi Toke

 Our current configuration Lucene 2.(something) with RAILO/CFML app server.

 10K drives, Quad Core, 16GB, Two servers. But the indexing and searching are
 starting to fail and our developer is no longer with us so it is quicker to
 rebuild than fix all the code.

 Our existing config is lots of indexes with merges into the larger ones.

 They are still running very fast but indexing is causing us issues.

 Thanks



 --
 View this message in context: 
 http://lucene.472066.n3.nabble.com/Newbie-Question-Master-Index-or-100s-Small-Index-tp4125407p4125447.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie Question: Master Index or 100s Small Index

[Shawn Heisey]

On 3/19/2014 4:55 AM, Colin R wrote:

My question is an architecture one.

These photos are currently indexed and searched in three ways.

1: The 14M pictures from above are split into a few hundred indexes that
feed a single website. This means index sizes of between 100 and 500,000
entries each.

2: 95% of these same photos are also wanted for searching on a global site.
Index size of 12M plus.

3: 80% of these same photos are also required for smaller group sites. Index
sizes of between 400K and 4M.

We currently make changes the single indexes and then merge into groups and
global. Due to the size of the numbers, is it worth changing or not.

Is it quicker/better to just have one big 14M index and filter the
complexities for each website or is it better to still maintain hundreds of
indexes so we are searching smaller one. Bear in mind, we get thousands of
changes a day PLUS very busy search servers.


My primary use for Solr is an archive of 92 million documents, most of 
which are photos.  We have thousands of new photos every day.  I haven't 
been cleared to mention what company it's for.


This screenshot of my status servlet page answers tons of questions 
about my index, but if you have additional questions, ask:


https://www.dropbox.com/s/6p1puq1gq3j8nln/solr-status-servlet.png

Here are some details about each host that you cannot see in the 
screenshot: 6 SATA disks in RAID10 with 3TB of usable space.  64GB of 
RAM.  Dual quad-core Intel E54xx series CPUs.Chain A is running Solr 
4.2.1 on Java 6, chain B is running Solr 4.6.1 on Java 7, with some 
additional plugin software that increases the index size.  There is one 
Solr process per host, with a 6GB heap.


As long as you index fields that can be used to filter searches 
according to what a user is allowed to see, I don't see any problem with 
putting all of your data into one index.The main thing you'll want to be 
sure of is that you have enough RAM to effectively cache your index.  
Because you have SSD, you probably don't need to have enough RAM to 
cache ALL of the index data, but it wouldn't hurt.  With 36GB of RAM per 
machine, you will probably have enough.


Thanks,
Shawn

Re: Newbie question on Deduplication overWriteDupes flag

[Chris Hostetter]

: How do I achieve, add if not there, fail if duplicate is found. I though

You can use the optimistic concurrency features to do this, by including a 
_version_=-1 field value in the document.

this will instruct solr that the update should only be processed if the 
document does not already exist...

https://cwiki.apache.org/confluence/display/solr/Updating+Parts+of+Documents




-Hoss
http://www.lucidworks.com/

Re: Newbie question on Deduplication overWriteDupes flag

[Alexandre Rafalovitch]

A follow up question on this (as it is kind of new functionality).

What happens if several documents are submitted and one of them fails
due to that? Do they get rolled back or only one?

Regards,
   Alex.
Personal website: http://www.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all
at once. Lately, it doesn't seem to be working.  (Anonymous  - via GTD
book)


On Thu, Feb 6, 2014 at 11:17 PM, Chris Hostetter
hossman_luc...@fucit.org wrote:

 : How do I achieve, add if not there, fail if duplicate is found. I though

 You can use the optimistic concurrency features to do this, by including a
 _version_=-1 field value in the document.

 this will instruct solr that the update should only be processed if the
 document does not already exist...

 https://cwiki.apache.org/confluence/display/solr/Updating+Parts+of+Documents




 -Hoss
 http://www.lucidworks.com/

Re: Newbie question on recurring theme: Dynamic Fields

[Erik Hatcher]

You need to use dynamicField not field, that's all :)

 Erik

On Feb 20, 2013, at 4:06, Erik Dybdahl erik...@gmail.com wrote:

 Hi,
 I'm currently assessing lucene/solr as a search front end for documents
 currently stored in an rdbms.
 The data has been made searchable to clients, in a way so that each
 client/customer may define what elements of their documents are searchable,
 by defining a field name, and a reference from that name into the data
 which is subsequently used to extract the value from the client documents
 and for each document associate it with the customer chosen field name.
 The concept of dynamic fields seemed to be exactly what I was looking for.
 Under Solr Features, Detailed Features, lucene.apache.org/solr says
 Dynamic Fields enables on-the-fly addition of new fields.
 And, in more depth, at wiki.apache.org/solr/SchemaXml:
 One of the powerful features of Lucene is that you don't have to
 pre-define every field when you first create your index.
 :
 For example the following dynamic field declaration tells Solr that
 whenever it sees a field name ending in _i which is not an explicitly
 defined field, then it should dynamically create an integer field with that
 name...
dynamicField name=*_i  type=integer  indexed=true
 stored=true/
 Cool.
 However, after definining
   field name=customerField_* type=string indexed=true
 stored=true multiValued=true/
 then trying to add the dynamic fields from the values in the db using solrj
 thus
 
 solrInputDocument.addField(customerField_+searchFieldResultSet.getString(name),
 searchFieldResultSet.getString(value));
 (i.e. attempting to create a dynamic field for e.g. the customer field
 FirstName) yields
 
 org.apache.solr.common.SolrException: ERROR: [doc=62485318] unknown field
 'customerField_FirstName'
at
 org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:404)
 
 How come? The documentation states clearly that solr should dynamically
 create a field in such cases.
 I am aware that similar problems have been discussed in several threads
 before, but the appearant complexity of the issue confuses me.
 
 Is what is stated in the documentation correct?
 In that case, what am I doing wrong, and what is the correct way?
 If the Dynamic Field concept is just a way of specifying common
 characteristics of multiple defined fields, then I think the documentation
 ought to be changed.
 
 I was able to solve the problem by following a suggestion in one of the
 threads, namely to create just one generic field, and then concatenate name
 and value of the customer field into it, like this:
   field name=customerField_s type=string indexed=true stored=true
 multiValued=true/
 and
solrInputDocument.addField(customerField_s,
 searchFieldResultSet.getString(name)+$+searchFieldResultSet.getString(value));
 which enables queries like:
  customerField_s:LastName$PETTERSON
 but that's not a very elegant solution.
 Also, attempting to use customerField_* in the query does not work, and
 from reading the documentation I do not understand why.
 
 Anyway, the response time when searching for these values is superb, when
 compared to doing the search directly in the database :-)
 
 Regards
 Erik

Re: Newbie question on recurring theme: Dynamic Fields

[Toke Eskildsen]

On Wed, 2013-02-20 at 10:06 +0100, Erik Dybdahl wrote:
 However, after definining
field name=customerField_* type=string indexed=true
 stored=true multiValued=true/

Seems like a typo to me: You need to write dynamicField, not
field, when defining a dynamic field.

Regards,
Toke Eskildsen

Re: Newbie question on recurring theme: Dynamic Fields

[Erik Dybdahl]

Excellent, works like a charm!
Though embarassing, it's still a good thing the only problem was me being
blind :-)

Thank you, Toke and Erik.


On Wed, Feb 20, 2013 at 11:47 AM, Toke Eskildsen 
t...@statsbiblioteket.dkwrote:

 On Wed, 2013-02-20 at 10:06 +0100, Erik Dybdahl wrote:
  However, after definining
 field name=customerField_* type=string indexed=true
  stored=true multiValued=true/

 Seems like a typo to me: You need to write dynamicField, not
 field, when defining a dynamic field.

 Regards,
 Toke Eskildsen

Re: Newbie question on sorting

[Jacek]

Erick, I'll do that. Thank you very much.

Regards,
Jacek

On Tue, May 1, 2012 at 7:19 AM, Erick Erickson erickerick...@gmail.comwrote:

 The easiest way is to do that in the app. That is, return the top
 10 to the app (by score) then re-order them there. There's nothing
 in Solr that I know of that does what you want out of the box.

 Best
 Erick

 On Mon, Apr 30, 2012 at 11:10 AM, Jacek pjac...@gmail.com wrote:
  Hello all,
 
  I'm facing this simple problem, yet impossible to resolve for me (I'm a
  newbie in Solr).
  I need to sort the results by score (it is simple, of course), but then
  what I need is to take top 10 results, and re-order it (only those top 10
  results) by a date field.
  It's not the same as sort=score,creationdate
 
  Any suggestions will be greatly appreciated!

Re: Newbie question on sorting

[Erick Erickson]

The easiest way is to do that in the app. That is, return the top
10 to the app (by score) then re-order them there. There's nothing
in Solr that I know of that does what you want out of the box.

Best
Erick

On Mon, Apr 30, 2012 at 11:10 AM, Jacek pjac...@gmail.com wrote:
 Hello all,

 I'm facing this simple problem, yet impossible to resolve for me (I'm a
 newbie in Solr).
 I need to sort the results by score (it is simple, of course), but then
 what I need is to take top 10 results, and re-order it (only those top 10
 results) by a date field.
 It's not the same as sort=score,creationdate

 Any suggestions will be greatly appreciated!

Re: Newbie question

[Chris Hostetter]

: If using CommonsHttpSolrServer query() method with parameter wt=json, when
: retrieving QueryResponse, how to do to get JSON result output stream ?

when you are using the CommonsHttpSolrServer level of API, the client 
takes care of parsing the response (which is typically in an efficient 
binary representation) into the basic data structures.

if you just want the raw response (in json or xml, or whatever) as a java 
String, then it may not be the API you really want: just use a basic 
HttpClient.


Alternately: you could consider writing your own subclass of 
ResponseParser that just slurps the InputStream that CommonsHttpSolrServer 
fetches for you.


-Hoss

RE: Newbie question, ant target for packaging source files from local copy?

[syyang]

Hi Steve,

I've filed a new JIRA issue along with the patch, which can be found at
lt;https://issues.apache.org/jira/browse/LUCENE-3406gt;.

Please let me know if you see any problem.

Thanks!
-Sid

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Newbie-question-ant-target-for-packaging-source-files-from-local-copy-tp3282787p3294320.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie Question, can I store structured sub elements?

[Zac Tolley]

I know I can have multi value on them but that doesn't let me see that
a showing instance happens at a particular time on a particular
channel, just that it shows on a range of channels at a range of times

Starting to think I will have to either store a formatted string that
combines them or keep it flat just for indexing, retrieve ids and use
them to get data out of the RDBMS


On 24 Aug 2011, at 23:09, dan whelan d...@adicio.com wrote:

 You could change starttime and channelname to multiValued=true and use these 
 fields to store all the values for those fields.

 showing.movie_id and showing.id probably isn't needed in a solr record.



 On 8/24/11 7:53 AM, Zac Tolley wrote:
 I have a very scenario in which I have a film and showings, each film has
 multiple showings at set times on set channels, so I have:

 Movie
 -
 id
 title
 description
 duration


 Showing
 -
 id
 movie_id
 starttime
 channelname



 I want to know can I store this in solr so that I keep this stucture?

 I did try to do an initial import with the DIH using this config:

 entity name=movie query=SELECT * from movies
field column=ID name=id/
field column=TITLE name=title/
field name=DESCRIPTION name=description/

entity name=showing query=SELECT * from showings WHERE movie_id = ${
 movie.id}
  field column=ID name=id/
  field column=STARTTIME name=starttime/
  field column=CHANNELNAME name=channelname/
/entity
 /entity

 I was hoping, for each movie to get a sub entity with the showing like:

 doc
str name=title./str
showing
  str name=channelname.



 but instead all the fields are flattened down to the top level.

 I know this must be easy, what am I missing... ?

RE: Newbie question, ant target for packaging source files from local copy?

[Steven A Rowe]

Hi sid,

The current source packaging scheme aims to *avoid* including local changes :), 
so yes, there is no support currently for what you want to do.

Prior to https://issues.apache.org/jira/browse/LUCENE-2973, the source 
packaging scheme used the current sources rather than pulling from Subversion.  
If you check out trunk revision 1083212 or earlier, or branch_3x revision 
1083234 or earlier, you can see how it used to be done.

If you want to resurrect the previous source packaging scheme as a new Ant 
target (maybe named package-local-src-tgz?), and you make a new JIRA issue 
and post a patch, and I'll help you get it committed (assuming nobody objects). 
 If you haven't seen the Solr Wiki HowToContribute page 
http://wiki.apache.org/solr/HowToContribute, it may be of use to you for this.

Steve

 -Original Message-
 From: syyang [mailto:syyan...@gmail.com]
 Sent: Wednesday, August 24, 2011 10:07 PM
 To: solr-user@lucene.apache.org
 Subject: Newbie question, ant target for packaging source files from
 local copy?
 
 Hi all,
 
 I am trying to package source files containing local changes. While
 running
 ant dist creates a war file containing the local changes, running ant
 package-src-tgz exports files straight from svn repository, and does not
 pick up any of the local changes.
 
 Is there an ant target that I can use to package local copy of the source
 files? Or are are we expected to just write our own?
 
 Thanks,
 -Sid
 
 --
 View this message in context: http://lucene.472066.n3.nabble.com/Newbie-
 question-ant-target-for-packaging-source-files-from-local-copy-
 tp3282787p3282787.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie Question, can I store structured sub elements?

[Erick Erickson]

nope, it's not easy. Solr docs are flat, flat, flat with the tiny
exception that multiValued fields are returned as a lists.

However, you can count on multi-valued fields being returned
in the order they were added, so it might work out for you to
treat these as parallel arrays in Solr documents.

Best
Erick

On Thu, Aug 25, 2011 at 3:10 AM, Zac Tolley z...@thetolleys.com wrote:
 I know I can have multi value on them but that doesn't let me see that
 a showing instance happens at a particular time on a particular
 channel, just that it shows on a range of channels at a range of times

 Starting to think I will have to either store a formatted string that
 combines them or keep it flat just for indexing, retrieve ids and use
 them to get data out of the RDBMS


 On 24 Aug 2011, at 23:09, dan whelan d...@adicio.com wrote:

 You could change starttime and channelname to multiValued=true and use these 
 fields to store all the values for those fields.

 showing.movie_id and showing.id probably isn't needed in a solr record.



 On 8/24/11 7:53 AM, Zac Tolley wrote:
 I have a very scenario in which I have a film and showings, each film has
 multiple showings at set times on set channels, so I have:

 Movie
 -
 id
 title
 description
 duration


 Showing
 -
 id
 movie_id
 starttime
 channelname



 I want to know can I store this in solr so that I keep this stucture?

 I did try to do an initial import with the DIH using this config:

 entity name=movie query=SELECT * from movies
    field column=ID name=id/
    field column=TITLE name=title/
    field name=DESCRIPTION name=description/

    entity name=showing query=SELECT * from showings WHERE movie_id = ${
 movie.id}
      field column=ID name=id/
      field column=STARTTIME name=starttime/
      field column=CHANNELNAME name=channelname/
    /entity
 /entity

 I was hoping, for each movie to get a sub entity with the showing like:

 doc
    str name=title./str
    showing
      str name=channelname.



 but instead all the fields are flattened down to the top level.

 I know this must be easy, what am I missing... ?

Re: Newbie Question, can I store structured sub elements?

[Zac Tolley]

have come to that conclusion  so had to choose between multiple fields with
multiple vales or a field with delimited text, gone for the former

On Thu, Aug 25, 2011 at 7:58 PM, Erick Erickson erickerick...@gmail.comwrote:

 nope, it's not easy. Solr docs are flat, flat, flat with the tiny
 exception that multiValued fields are returned as a lists.

 However, you can count on multi-valued fields being returned
 in the order they were added, so it might work out for you to
 treat these as parallel arrays in Solr documents.

 Best
 Erick

 On Thu, Aug 25, 2011 at 3:10 AM, Zac Tolley z...@thetolleys.com wrote:
  I know I can have multi value on them but that doesn't let me see that
  a showing instance happens at a particular time on a particular
  channel, just that it shows on a range of channels at a range of times
 
  Starting to think I will have to either store a formatted string that
  combines them or keep it flat just for indexing, retrieve ids and use
  them to get data out of the RDBMS
 
 
  On 24 Aug 2011, at 23:09, dan whelan d...@adicio.com wrote:
 
  You could change starttime and channelname to multiValued=true and use
 these fields to store all the values for those fields.
 
  showing.movie_id and showing.id probably isn't needed in a solr record.
 
 
 
  On 8/24/11 7:53 AM, Zac Tolley wrote:
  I have a very scenario in which I have a film and showings, each film
 has
  multiple showings at set times on set channels, so I have:
 
  Movie
  -
  id
  title
  description
  duration
 
 
  Showing
  -
  id
  movie_id
  starttime
  channelname
 
 
 
  I want to know can I store this in solr so that I keep this stucture?
 
  I did try to do an initial import with the DIH using this config:
 
  entity name=movie query=SELECT * from movies
 field column=ID name=id/
 field column=TITLE name=title/
 field name=DESCRIPTION name=description/
 
 entity name=showing query=SELECT * from showings WHERE movie_id
 = ${
  movie.id}
   field column=ID name=id/
   field column=STARTTIME name=starttime/
   field column=CHANNELNAME name=channelname/
 /entity
  /entity
 
  I was hoping, for each movie to get a sub entity with the showing like:
 
  doc
 str name=title./str
 showing
   str name=channelname.
 
 
 
  but instead all the fields are flattened down to the top level.
 
  I know this must be easy, what am I missing... ?
 
 
 

Re: Newbie Question, can I store structured sub elements?

[Paul Libbrecht]

Delimited text is the baby form of lists.
Text can be made very very structured (think XML, ontologies...).
I think the crux is your search needs.

For example, with Lucene, I made a search for formulæ (including sub-terms) by 
converting the OpenMath-encoded terms into rows of tokens and querying with 
SpanQueries. Quite structured to my taste.

What you don't have is the freedom of joins which brings a very flexible query 
mechanism almost independent of the schema... but this often can be 
circumvented by the flat solr and lucene storage whose performance is really 
amazing.

paul


Le 25 août 2011 à 21:07, Zac Tolley a écrit :

 have come to that conclusion  so had to choose between multiple fields with
 multiple vales or a field with delimited text, gone for the former
 
 On Thu, Aug 25, 2011 at 7:58 PM, Erick Erickson 
 erickerick...@gmail.comwrote:
 
 nope, it's not easy. Solr docs are flat, flat, flat with the tiny
 exception that multiValued fields are returned as a lists.
 
 However, you can count on multi-valued fields being returned
 in the order they were added, so it might work out for you to
 treat these as parallel arrays in Solr documents.
 
 Best
 Erick
 
 On Thu, Aug 25, 2011 at 3:10 AM, Zac Tolley z...@thetolleys.com wrote:
 I know I can have multi value on them but that doesn't let me see that
 a showing instance happens at a particular time on a particular
 channel, just that it shows on a range of channels at a range of times
 
 Starting to think I will have to either store a formatted string that
 combines them or keep it flat just for indexing, retrieve ids and use
 them to get data out of the RDBMS
 
 
 On 24 Aug 2011, at 23:09, dan whelan d...@adicio.com wrote:
 
 You could change starttime and channelname to multiValued=true and use
 these fields to store all the values for those fields.
 
 showing.movie_id and showing.id probably isn't needed in a solr record.
 
 
 
 On 8/24/11 7:53 AM, Zac Tolley wrote:
 I have a very scenario in which I have a film and showings, each film
 has
 multiple showings at set times on set channels, so I have:
 
 Movie
 -
 id
 title
 description
 duration
 
 
 Showing
 -
 id
 movie_id
 starttime
 channelname
 
 
 
 I want to know can I store this in solr so that I keep this stucture?
 
 I did try to do an initial import with the DIH using this config:
 
 entity name=movie query=SELECT * from movies
   field column=ID name=id/
   field column=TITLE name=title/
   field name=DESCRIPTION name=description/
 
   entity name=showing query=SELECT * from showings WHERE movie_id
 = ${
 movie.id}
 field column=ID name=id/
 field column=STARTTIME name=starttime/
 field column=CHANNELNAME name=channelname/
   /entity
 /entity
 
 I was hoping, for each movie to get a sub entity with the showing like:
 
 doc
   str name=title./str
   showing
 str name=channelname.
 
 
 
 but instead all the fields are flattened down to the top level.
 
 I know this must be easy, what am I missing... ?
 
 
 
 

Re: Newbie Question, can I store structured sub elements?

[Zac Tolley]

My search is very simple, mainly on titles, actors, show times and channels.
Having multiple lists of values is probably better for that, and as the
order is kept the same its relatively simple to map the response back onto
pojos for my presentation layer.

On Thu, Aug 25, 2011 at 8:18 PM, Paul Libbrecht p...@hoplahup.net wrote:

 Delimited text is the baby form of lists.
 Text can be made very very structured (think XML, ontologies...).
 I think the crux is your search needs.

 For example, with Lucene, I made a search for formulæ (including sub-terms)
 by converting the OpenMath-encoded terms into rows of tokens and querying
 with SpanQueries. Quite structured to my taste.

 What you don't have is the freedom of joins which brings a very flexible
 query mechanism almost independent of the schema... but this often can be
 circumvented by the flat solr and lucene storage whose performance is really
 amazing.

 paul


 Le 25 août 2011 à 21:07, Zac Tolley a écrit :

  have come to that conclusion  so had to choose between multiple fields
 with
  multiple vales or a field with delimited text, gone for the former
 
  On Thu, Aug 25, 2011 at 7:58 PM, Erick Erickson erickerick...@gmail.com
 wrote:
 
  nope, it's not easy. Solr docs are flat, flat, flat with the tiny
  exception that multiValued fields are returned as a lists.
 
  However, you can count on multi-valued fields being returned
  in the order they were added, so it might work out for you to
  treat these as parallel arrays in Solr documents.
 
  Best
  Erick
 
  On Thu, Aug 25, 2011 at 3:10 AM, Zac Tolley z...@thetolleys.com wrote:
  I know I can have multi value on them but that doesn't let me see that
  a showing instance happens at a particular time on a particular
  channel, just that it shows on a range of channels at a range of times
 
  Starting to think I will have to either store a formatted string that
  combines them or keep it flat just for indexing, retrieve ids and use
  them to get data out of the RDBMS
 
 
  On 24 Aug 2011, at 23:09, dan whelan d...@adicio.com wrote:
 
  You could change starttime and channelname to multiValued=true and use
  these fields to store all the values for those fields.
 
  showing.movie_id and showing.id probably isn't needed in a solr
 record.
 
 
 
  On 8/24/11 7:53 AM, Zac Tolley wrote:
  I have a very scenario in which I have a film and showings, each film
  has
  multiple showings at set times on set channels, so I have:
 
  Movie
  -
  id
  title
  description
  duration
 
 
  Showing
  -
  id
  movie_id
  starttime
  channelname
 
 
 
  I want to know can I store this in solr so that I keep this stucture?
 
  I did try to do an initial import with the DIH using this config:
 
  entity name=movie query=SELECT * from movies
field column=ID name=id/
field column=TITLE name=title/
field name=DESCRIPTION name=description/
 
entity name=showing query=SELECT * from showings WHERE movie_id
  = ${
  movie.id}
  field column=ID name=id/
  field column=STARTTIME name=starttime/
  field column=CHANNELNAME name=channelname/
/entity
  /entity
 
  I was hoping, for each movie to get a sub entity with the showing
 like:
 
  doc
str name=title./str
showing
  str name=channelname.
 
 
 
  but instead all the fields are flattened down to the top level.
 
  I know this must be easy, what am I missing... ?
 
 
 
 

Re: Newbie Question, can I store structured sub elements?

[Paul Libbrecht]

Whether multi-valued or token-streams, the question is search, not 
(de)serialization: that's opaque to Solr which will take and give it to you as 
needed.

paul


Le 25 août 2011 à 21:24, Zac Tolley a écrit :

 My search is very simple, mainly on titles, actors, show times and channels.
 Having multiple lists of values is probably better for that, and as the
 order is kept the same its relatively simple to map the response back onto
 pojos for my presentation layer.

Re: Newbie Question, can I store structured sub elements?

[dan whelan]

You could change starttime and channelname to multiValued=true and use 
these fields to store all the values for those fields.


showing.movie_id and showing.id probably isn't needed in a solr record.



On 8/24/11 7:53 AM, Zac Tolley wrote:

I have a very scenario in which I have a film and showings, each film has
multiple showings at set times on set channels, so I have:

Movie
-
id
title
description
duration


Showing
-
id
movie_id
starttime
channelname



I want to know can I store this in solr so that I keep this stucture?

I did try to do an initial import with the DIH using this config:

entity name=movie query=SELECT * from movies
field column=ID name=id/
field column=TITLE name=title/
field name=DESCRIPTION name=description/

entity name=showing query=SELECT * from showings WHERE movie_id = ${
movie.id}
  field column=ID name=id/
  field column=STARTTIME name=starttime/
  field column=CHANNELNAME name=channelname/
/entity
/entity

I was hoping, for each movie to get a sub entity with the showing like:

doc
str name=title./str
showing
  str name=channelname.



but instead all the fields are flattened down to the top level.

I know this must be easy, what am I missing... ?

Re: Newbie question: how to deal with different # of search results per page due to pagination then grouping

[Michael Sokolov]

Just keep one extra facet value hidden; ie request one more than you 
need to show the current page.  If you get it, there are more (show the 
next button), otherwise there aren't.  You can't page arbitrarily deep 
like this, but you can have a next button reliably enabled or disabled.


On 6/1/2011 5:57 PM, Robert Petersen wrote:

Yes that is exactly the issue... we're thinking just maybe always have a
next button and if you go too far you just get zero results.  User gets
what the user asks for, and so user could simply back up if desired to
where the facet still has values.  Could also detect an empty facet
results on the front end.  You can also only expand one facet only to
allow paging only the facet pane and not the whole page using an ajax
call.



-Original Message-
From: Jonathan Rochkind [mailto:rochk...@jhu.edu]
Sent: Wednesday, June 01, 2011 2:30 PM
To: solr-user@lucene.apache.org
Cc: Robert Petersen
Subject: Re: Newbie question: how to deal with different # of search
results per page due to pagination then grouping

How do you know whether to provide a 'next' button, or whether you are
the end of your facet list?

On 6/1/2011 4:47 PM, Robert Petersen wrote:

I think facet.offset allows facet paging nicely by letting you index
into the list of facet values.  It is working for me...

http://wiki.apache.org/solr/SimpleFacetParameters#facet.offset


-Original Message-
From: Jonathan Rochkind [mailto:rochk...@jhu.edu]
Sent: Wednesday, June 01, 2011 12:41 PM
To: solr-user@lucene.apache.org
Subject: Re: Newbie question: how to deal with different # of search
results per page due to pagination then grouping

There's no great way to do that.

One approach would be using facets, but that will just get you the
author names (as stored in fields), and not the documents under it. If
you really only want to show the author names, facets could work. One
issue with facets though is Solr won't tell you the total number of
facet values for your query, so it's tricky to provide next/prev

paging

through them.

There is also a 'field collapsing' feature that I think is not in a
released Solr, but may be in the Solr repo. I'm not sure it will quite
do what you want either though, although it's related and worth a

look.

http://wiki.apache.org/solr/FieldCollapsing

Another vaguely related thing that is also not yet in a released Solr,
is a 'join' function. That could possibly be used to do what you want,
although it'd be tricky too.
https://issues.apache.org/jira/browse/SOLR-2272

Jonathan

On 6/1/2011 2:56 PM, beccax wrote:

Apologize if this question has already been raised.  I tried

searching

but

couldn't find the relevant posts.

We've indexed a bunch of documents by different authors.  Then for

search

results, we'd like to show the authors that have 1 or more documents
matching the search keywords.

The problem is right now our solr search method first paginates

results to

100 documents per page, then we take the results and group by

authors.

This

results in different number of authors per page.  (Some authors may

only

have one matching document and others 5 or 10.)

How do we change it to somehow show the same number of authors (say

25) per

page?

I mean alternatively we could just show all the documents themselves

ordered

by author, but it's not the user experience we're looking for.

Thanks so much.  And please let me know if you need more details not
provided here.
B

--
View this message in context:

http://lucene.472066.n3.nabble.com/Newbie-question-how-to-deal-with-diff
erent-of-search-results-per-page-due-to-pagination-then-grouping-tp30121

68p3012168.html

Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie question: how to deal with different # of search results per page due to pagination then grouping

[Jonathan Rochkind]

There's no great way to do that.

One approach would be using facets, but that will just get you the 
author names (as stored in fields), and not the documents under it. If 
you really only want to show the author names, facets could work. One 
issue with facets though is Solr won't tell you the total number of 
facet values for your query, so it's tricky to provide next/prev paging 
through them.


There is also a 'field collapsing' feature that I think is not in a 
released Solr, but may be in the Solr repo. I'm not sure it will quite 
do what you want either though, although it's related and worth a look. 
http://wiki.apache.org/solr/FieldCollapsing


Another vaguely related thing that is also not yet in a released Solr, 
is a 'join' function. That could possibly be used to do what you want, 
although it'd be tricky too. https://issues.apache.org/jira/browse/SOLR-2272


Jonathan

On 6/1/2011 2:56 PM, beccax wrote:

Apologize if this question has already been raised.  I tried searching but
couldn't find the relevant posts.

We've indexed a bunch of documents by different authors.  Then for search
results, we'd like to show the authors that have 1 or more documents
matching the search keywords.

The problem is right now our solr search method first paginates results to
100 documents per page, then we take the results and group by authors.  This
results in different number of authors per page.  (Some authors may only
have one matching document and others 5 or 10.)

How do we change it to somehow show the same number of authors (say 25) per
page?

I mean alternatively we could just show all the documents themselves ordered
by author, but it's not the user experience we're looking for.

Thanks so much.  And please let me know if you need more details not
provided here.
B

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Newbie-question-how-to-deal-with-different-of-search-results-per-page-due-to-pagination-then-grouping-tp3012168p3012168.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Newbie question: how to deal with different # of search results per page due to pagination then grouping

[Robert Petersen]

Don't manually group by author from your results, the list will always
be incomplete...  use faceting instead to show the authors of the books
you have found in your search.

http://wiki.apache.org/solr/SolrFacetingOverview

-Original Message-
From: beccax [mailto:bec...@gmail.com] 
Sent: Wednesday, June 01, 2011 11:56 AM
To: solr-user@lucene.apache.org
Subject: Newbie question: how to deal with different # of search results
per page due to pagination then grouping

Apologize if this question has already been raised.  I tried searching
but
couldn't find the relevant posts.

We've indexed a bunch of documents by different authors.  Then for
search
results, we'd like to show the authors that have 1 or more documents
matching the search keywords.  

The problem is right now our solr search method first paginates results
to
100 documents per page, then we take the results and group by authors.
This
results in different number of authors per page.  (Some authors may only
have one matching document and others 5 or 10.)

How do we change it to somehow show the same number of authors (say 25)
per
page?

I mean alternatively we could just show all the documents themselves
ordered
by author, but it's not the user experience we're looking for.

Thanks so much.  And please let me know if you need more details not
provided here.
B

--
View this message in context:
http://lucene.472066.n3.nabble.com/Newbie-question-how-to-deal-with-diff
erent-of-search-results-per-page-due-to-pagination-then-grouping-tp30121
68p3012168.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Newbie question: how to deal with different # of search results per page due to pagination then grouping

[Robert Petersen]

I think facet.offset allows facet paging nicely by letting you index
into the list of facet values.  It is working for me...

http://wiki.apache.org/solr/SimpleFacetParameters#facet.offset


-Original Message-
From: Jonathan Rochkind [mailto:rochk...@jhu.edu] 
Sent: Wednesday, June 01, 2011 12:41 PM
To: solr-user@lucene.apache.org
Subject: Re: Newbie question: how to deal with different # of search
results per page due to pagination then grouping

There's no great way to do that.

One approach would be using facets, but that will just get you the 
author names (as stored in fields), and not the documents under it. If 
you really only want to show the author names, facets could work. One 
issue with facets though is Solr won't tell you the total number of 
facet values for your query, so it's tricky to provide next/prev paging 
through them.

There is also a 'field collapsing' feature that I think is not in a 
released Solr, but may be in the Solr repo. I'm not sure it will quite 
do what you want either though, although it's related and worth a look. 
http://wiki.apache.org/solr/FieldCollapsing

Another vaguely related thing that is also not yet in a released Solr, 
is a 'join' function. That could possibly be used to do what you want, 
although it'd be tricky too.
https://issues.apache.org/jira/browse/SOLR-2272

Jonathan

On 6/1/2011 2:56 PM, beccax wrote:
 Apologize if this question has already been raised.  I tried searching
but
 couldn't find the relevant posts.

 We've indexed a bunch of documents by different authors.  Then for
search
 results, we'd like to show the authors that have 1 or more documents
 matching the search keywords.

 The problem is right now our solr search method first paginates
results to
 100 documents per page, then we take the results and group by authors.
This
 results in different number of authors per page.  (Some authors may
only
 have one matching document and others 5 or 10.)

 How do we change it to somehow show the same number of authors (say
25) per
 page?

 I mean alternatively we could just show all the documents themselves
ordered
 by author, but it's not the user experience we're looking for.

 Thanks so much.  And please let me know if you need more details not
 provided here.
 B

 --
 View this message in context:
http://lucene.472066.n3.nabble.com/Newbie-question-how-to-deal-with-diff
erent-of-search-results-per-page-due-to-pagination-then-grouping-tp30121
68p3012168.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie question: how to deal with different # of search results per page due to pagination then grouping

[Jonathan Rochkind]

How do you know whether to provide a 'next' button, or whether you are 
the end of your facet list?


On 6/1/2011 4:47 PM, Robert Petersen wrote:

I think facet.offset allows facet paging nicely by letting you index
into the list of facet values.  It is working for me...

http://wiki.apache.org/solr/SimpleFacetParameters#facet.offset


-Original Message-
From: Jonathan Rochkind [mailto:rochk...@jhu.edu]
Sent: Wednesday, June 01, 2011 12:41 PM
To: solr-user@lucene.apache.org
Subject: Re: Newbie question: how to deal with different # of search
results per page due to pagination then grouping

There's no great way to do that.

One approach would be using facets, but that will just get you the
author names (as stored in fields), and not the documents under it. If
you really only want to show the author names, facets could work. One
issue with facets though is Solr won't tell you the total number of
facet values for your query, so it's tricky to provide next/prev paging
through them.

There is also a 'field collapsing' feature that I think is not in a
released Solr, but may be in the Solr repo. I'm not sure it will quite
do what you want either though, although it's related and worth a look.
http://wiki.apache.org/solr/FieldCollapsing

Another vaguely related thing that is also not yet in a released Solr,
is a 'join' function. That could possibly be used to do what you want,
although it'd be tricky too.
https://issues.apache.org/jira/browse/SOLR-2272

Jonathan

On 6/1/2011 2:56 PM, beccax wrote:

Apologize if this question has already been raised.  I tried searching

but

couldn't find the relevant posts.

We've indexed a bunch of documents by different authors.  Then for

search

results, we'd like to show the authors that have 1 or more documents
matching the search keywords.

The problem is right now our solr search method first paginates

results to

100 documents per page, then we take the results and group by authors.

This

results in different number of authors per page.  (Some authors may

only

have one matching document and others 5 or 10.)

How do we change it to somehow show the same number of authors (say

25) per

page?

I mean alternatively we could just show all the documents themselves

ordered

by author, but it's not the user experience we're looking for.

Thanks so much.  And please let me know if you need more details not
provided here.
B

--
View this message in context:

http://lucene.472066.n3.nabble.com/Newbie-question-how-to-deal-with-diff
erent-of-search-results-per-page-due-to-pagination-then-grouping-tp30121
68p3012168.html

Sent from the Solr - User mailing list archive at Nabble.com.

RE: Newbie question: how to deal with different # of search results per page due to pagination then grouping

[Robert Petersen]

Yes that is exactly the issue... we're thinking just maybe always have a
next button and if you go too far you just get zero results.  User gets
what the user asks for, and so user could simply back up if desired to
where the facet still has values.  Could also detect an empty facet
results on the front end.  You can also only expand one facet only to
allow paging only the facet pane and not the whole page using an ajax
call.



-Original Message-
From: Jonathan Rochkind [mailto:rochk...@jhu.edu] 
Sent: Wednesday, June 01, 2011 2:30 PM
To: solr-user@lucene.apache.org
Cc: Robert Petersen
Subject: Re: Newbie question: how to deal with different # of search
results per page due to pagination then grouping

How do you know whether to provide a 'next' button, or whether you are 
the end of your facet list?

On 6/1/2011 4:47 PM, Robert Petersen wrote:
 I think facet.offset allows facet paging nicely by letting you index
 into the list of facet values.  It is working for me...

 http://wiki.apache.org/solr/SimpleFacetParameters#facet.offset


 -Original Message-
 From: Jonathan Rochkind [mailto:rochk...@jhu.edu]
 Sent: Wednesday, June 01, 2011 12:41 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Newbie question: how to deal with different # of search
 results per page due to pagination then grouping

 There's no great way to do that.

 One approach would be using facets, but that will just get you the
 author names (as stored in fields), and not the documents under it. If
 you really only want to show the author names, facets could work. One
 issue with facets though is Solr won't tell you the total number of
 facet values for your query, so it's tricky to provide next/prev
paging
 through them.

 There is also a 'field collapsing' feature that I think is not in a
 released Solr, but may be in the Solr repo. I'm not sure it will quite
 do what you want either though, although it's related and worth a
look.
 http://wiki.apache.org/solr/FieldCollapsing

 Another vaguely related thing that is also not yet in a released Solr,
 is a 'join' function. That could possibly be used to do what you want,
 although it'd be tricky too.
 https://issues.apache.org/jira/browse/SOLR-2272

 Jonathan

 On 6/1/2011 2:56 PM, beccax wrote:
 Apologize if this question has already been raised.  I tried
searching
 but
 couldn't find the relevant posts.

 We've indexed a bunch of documents by different authors.  Then for
 search
 results, we'd like to show the authors that have 1 or more documents
 matching the search keywords.

 The problem is right now our solr search method first paginates
 results to
 100 documents per page, then we take the results and group by
authors.
 This
 results in different number of authors per page.  (Some authors may
 only
 have one matching document and others 5 or 10.)

 How do we change it to somehow show the same number of authors (say
 25) per
 page?

 I mean alternatively we could just show all the documents themselves
 ordered
 by author, but it's not the user experience we're looking for.

 Thanks so much.  And please let me know if you need more details not
 provided here.
 B

 --
 View this message in context:

http://lucene.472066.n3.nabble.com/Newbie-question-how-to-deal-with-diff

erent-of-search-results-per-page-due-to-pagination-then-grouping-tp30121
 68p3012168.html
 Sent from the Solr - User mailing list archive at Nabble.com.

RE: newbie question for DataImportHandler

[Kevin Bootz]

In the op it's stated that the index was deleted. I'm guessing that means the 
physical files, /data/  
quote
populate the table 
 with another million rows of data.
 I remove the index that solr previously create. I restart solr and go 
 to
the
 data import handler development console and do the full import again.
endquote

Is there a separate cache that could be causing the issue? I'm a newbie as well 
and it seems that if I delete the index there shouldn't be any vestige info 
left anywhere

Thanks

-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com] 
Sent: Sunday, May 29, 2011 9:00 PM
To: solr-user@lucene.apache.org
Subject: Re: newbie question for DataImportHandler

This trips up a lot of folks. Sold just marks docs as deleted, the terms etc 
are left in the index until an optimize is performed, or the segments are 
merged. This latter isn't very predictable, so just do an optimize.

The docs aren't returned as results though.

Best
Erick
On May 24, 2011 10:22 PM, antoniosi antonio...@gmail.com wrote:
 Hi,

 I am new to Solr; apologize in advance if this is a stupid question.

 I have created a simple database, with only 1 table with 3 columns, 
 id, name, and last_update fields.

 I populate the database with 1 million test rows.
 I run solr, go to the data import handler development console and do a
full
 import. I use the Luke tool to look at the content of the lucene index.

 This all works fine so far.

 I remove all the 1 million rows from my table and populate the table 
 with another million rows of data.
 I remove the index that solr previously create. I restart solr and go 
 to
the
 data import handler development console and do the full import again.

 I use the Luke tool to look at the content of the lucene index. 
 However,
I
 am seeing the old data in my new index.

 Doe Solr keeps a cached copy of the index somewhere?

 I hope I have described my problem clearly.

 Thanks in advance.

 --
 View this message in context:
http://lucene.472066.n3.nabble.com/newbie-question-for-DataImportHandler-tp2982277p2982277.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: newbie question for DataImportHandler

[Erick Erickson]

This trips up a lot of folks. Sold just marks docs as deleted, the terms etc
are left in the index until an optimize is performed, or the segments are
merged. This latter isn't very predictable, so just do an optimize.

The docs aren't returned as results though.

Best
Erick
On May 24, 2011 10:22 PM, antoniosi antonio...@gmail.com wrote:
 Hi,

 I am new to Solr; apologize in advance if this is a stupid question.

 I have created a simple database, with only 1 table with 3 columns, id,
 name, and last_update fields.

 I populate the database with 1 million test rows.
 I run solr, go to the data import handler development console and do a
full
 import. I use the Luke tool to look at the content of the lucene index.

 This all works fine so far.

 I remove all the 1 million rows from my table and populate the table with
 another million rows of data.
 I remove the index that solr previously create. I restart solr and go to
the
 data import handler development console and do the full import again.

 I use the Luke tool to look at the content of the lucene index. However,
I
 am seeing the old data in my new index.

 Doe Solr keeps a cached copy of the index somewhere?

 I hope I have described my problem clearly.

 Thanks in advance.

 --
 View this message in context:
http://lucene.472066.n3.nabble.com/newbie-question-for-DataImportHandler-tp2982277p2982277.html
 Sent from the Solr - User mailing list archive at Nabble.com.

RE: newbie question for DataImportHandler

[Zac Smith]

Sounds like you might not be committing the delete. How are you deleting it?
If you run the data import handler with clean=true (which is the default) it 
will delete the data for you anyway so you don't need to delete it yourself.

Hope that helps.

-Original Message-
From: antoniosi [mailto:antonio...@gmail.com] 
Sent: Tuesday, May 24, 2011 4:43 PM
To: solr-user@lucene.apache.org
Subject: newbie question for DataImportHandler

Hi,

I am new to Solr; apologize in advance if this is a stupid question.

I have created a simple database, with only 1 table with 3 columns, id, name, 
and last_update fields.

I populate the database with 1 million test rows.
I run solr, go to the data import handler development console and do a full 
import. I use the Luke tool to look at the content of the lucene index.

This all works fine so far.

I remove all the 1 million rows from my table and populate the table with 
another million rows of data.
I remove the index that solr previously create. I restart solr and go to the 
data import handler development console and do the full import again.

I use the Luke tool to look at the content of the lucene index. However, I am 
seeing the old data in my new index.

Doe Solr keeps a cached copy of the index somewhere?

I hope I have described my problem clearly.

Thanks in advance.

--
View this message in context: 
http://lucene.472066.n3.nabble.com/newbie-question-for-DataImportHandler-tp2982277p2982277.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie question: no search results

[BobG]

Hi Lance and Gora,

Thanks for your support!

I have changed 
 fields
field name=Shop_artikel_rg type=string  indexed=true 
stored=true
/
field name=Artikel type=string indexed=true stored=true /
field name=Omschrijving type=string indexed=true stored=true /
 /fields
Into
 fields
field name=Shop_artikel_rg type=string  indexed=true 
stored=true
/
field name=Artikel type=text indexed=true stored=true /
field name=Omschrijving type=text indexed=true stored=true /
 /fields

In the schema.xml, then restarted Tomcat. 
Also I used quotation marks for my search string and it works fine now!

Problem solved!

Best regards,
Bob

-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Newbie-question-no-search-results-tp1416482p1422211.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie question: no search results

[Gora Mohanty]

On Sat, 4 Sep 2010 01:15:11 -0700 (PDT)
BobG b...@bitwise-bncc.nl wrote:

 
 Hi,
 I am trying to set up a new SOLR search engine on a windows
 platform. It seems like I managed to fill an index with the
 contents of my SQL server table.
 
 When I use the default *.* query I get a nice result:
[...]
 However when I try to query the table with a search term I get no
 results.
[...]

 What am I doing wrong here?
[...]

Could you post your schema.xml. In particular, how is the Artikel
field being processed at index/query time?

Regards,
Gora

Re: Newbie question: no search results

[Lance Norskog]

More directly: if the 'Artikel' field is a string, only the whole 
string will match:

Artikel:Kerstman baardstel.
Or you can use a wildcard:  Kerstmann*  or just Kerst*

If it is a text field, it is chopped into words and 
q=Artikel:Kerstmann would work.


Gora Mohanty wrote:

On Sat, 4 Sep 2010 01:15:11 -0700 (PDT)
BobGb...@bitwise-bncc.nl  wrote:

   

Hi,
I am trying to set up a new SOLR search engine on a windows
platform. It seems like I managed to fill an index with the
contents of my SQL server table.

When I use the default *.* query I get a nice result:
 

[...]
   

However when I try to query the table with a search term I get no
results.
 

[...]

   

What am I doing wrong here?
 

[...]

Could you post your schema.xml. In particular, how is the Artikel
field being processed at index/query time?

Regards,
Gora
   

RE: Newbie question about search behavior

[Markus Jelsma]

You can append it in your middleware, or try the EdgeNGramTokenizer [1]. If 
you're going for the latter, don't forget to reindex and expect a larger index.

 

[1]: 
http://lucene.apache.org/java/2_9_0/api/all/org/apache/lucene/analysis/ngram/EdgeNGramTokenizer.html
-Original message-
From: Mike Thomsen mikerthom...@gmail.com
Sent: Mon 16-08-2010 19:09
To: solr-user@lucene.apache.org; 
Subject: Newbie question about search behavior

Is it possible to set up Lucene to treat a keyword search such as

title:News

implicitly like

title:News*

so that any title that begins with News will be returned without the
user having to throw in a wildcard?

Also, are there any common filters and such that are generally
considered a good practice to throw into the schema for an
English-language website?

Thanks,

Mike

 

Re: newbie question on how to batch commit documents

[findbestopensource]

Add commit after the loop. I would advise to use commit in a separate
thread. I do keep separate timer thread, where every minute I will do
commit and at the end of every day I will optimize the index.

Regards
Aditya
www.findbestopensource.com


On Tue, Jun 1, 2010 at 2:57 AM, Steve Kuo kuosen...@gmail.com wrote:

 I have a newbie question on what is the best way to batch add/commit a
 large
 collection of document data via solrj.  My first attempt  was to write a
 multi-threaded application that did following.

 CollectionSolrInputDocument docs = new ArrayListSolrInputDocument();
 for (Widget w : widges) {
doc.addField(id, w.getId());
doc.addField(name, w.getName());
   doc.addField(price, w.getPrice());
doc.addField(category, w.getCat());
doc.addField(srcType, w.getSrcType());
docs.add(doc);

// commit docs to solr server
server.add(docs);
server.commit();
 }

 And I got this exception.

 rg.apache.solr.common.SolrException:

 Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers2_try_again_later


 Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers2_try_again_later

at
 org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:424)
at
 org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:243)
at
 org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
at
 org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:86)

 The solrj wiki/documents seemed to indicate that because multiple threads
 were calling SolrServer.commit() which in term called
 CommonsHttpSolrServer.request() resulting in multiple searchers.  My first
 thought was to change the configs for autowarming.  But after looking at
 the
 autowarm params, I am not sure what can be changed or perhaps a different
 approach is recommened.

filterCache
  class=solr.FastLRUCache
  size=512
  initialSize=512
  autowarmCount=0/

queryResultCache
  class=solr.LRUCache
  size=512
  initialSize=512
  autowarmCount=0/

documentCache
  class=solr.LRUCache
  size=512
  initialSize=512
  autowarmCount=0/

 Your help is much appreciated.

Re: newbie question on how to batch commit documents

[olivier sallou]

I would additionally suggest to use embeddedSolrServer for large uploads if
possible, performance are better.

2010/5/31 Steve Kuo kuosen...@gmail.com

 I have a newbie question on what is the best way to batch add/commit a
 large
 collection of document data via solrj.  My first attempt  was to write a
 multi-threaded application that did following.

 CollectionSolrInputDocument docs = new ArrayListSolrInputDocument();
 for (Widget w : widges) {
doc.addField(id, w.getId());
doc.addField(name, w.getName());
   doc.addField(price, w.getPrice());
doc.addField(category, w.getCat());
doc.addField(srcType, w.getSrcType());
docs.add(doc);

// commit docs to solr server
server.add(docs);
server.commit();
 }

 And I got this exception.

 rg.apache.solr.common.SolrException:

 Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers2_try_again_later


 Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers2_try_again_later

at
 org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:424)
at
 org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:243)
at
 org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
at
 org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:86)

 The solrj wiki/documents seemed to indicate that because multiple threads
 were calling SolrServer.commit() which in term called
 CommonsHttpSolrServer.request() resulting in multiple searchers.  My first
 thought was to change the configs for autowarming.  But after looking at
 the
 autowarm params, I am not sure what can be changed or perhaps a different
 approach is recommened.

filterCache
  class=solr.FastLRUCache
  size=512
  initialSize=512
  autowarmCount=0/

queryResultCache
  class=solr.LRUCache
  size=512
  initialSize=512
  autowarmCount=0/

documentCache
  class=solr.LRUCache
  size=512
  initialSize=512
  autowarmCount=0/

 Your help is much appreciated.

Re: newbie question on how to batch commit documents

[Chris Hostetter]

: CommonsHttpSolrServer.request() resulting in multiple searchers.  My first
: thought was to change the configs for autowarming.  But after looking at the
: autowarm params, I am not sure what can be changed or perhaps a different
: approach is recommened.

even with 0 autowarming (which is what you have) it can still take time to 
close/open a searcher on every commit -- which is why a commit per doc is 
not usually a good idea (and is *definitely* not a good idea when doing 
batch indexing.

most people can get away with just doing one commit after all their docs 
have been added (ie: at the end of the batch) but if you've got a ot of 
distinct clients, doing a lot of parllel indexing and you don't want to 
coordinate who is responsible for sending the commit, you can configure 
autocommit to happen on the solr server...

http://wiki.apache.org/solr/SolrConfigXml#Update_Handler_Section

...but in general you should make sure that your clients sending docs can 
deal with the occasional long delays (or possibly even needing to retry) 
when an occasional commit might block add/delete operations because of an 
expensive segment merge.

-Hoss

Re: newbie question on how to batch commit documents

[Erik Hatcher]

Move the commit outside your loop and you'll be in better shape.   
Better yet, enable autocommit in solrconfig.xml and don't commit from  
your multithreaded client, otherwise you still run the risk of too  
many commits happening concurrently.


Erik

On May 31, 2010, at 5:27 PM, Steve Kuo wrote:

I have a newbie question on what is the best way to batch add/commit  
a large
collection of document data via solrj.  My first attempt  was to  
write a

multi-threaded application that did following.

CollectionSolrInputDocument docs = new  
ArrayListSolrInputDocument();

for (Widget w : widges) {
   doc.addField(id, w.getId());
   doc.addField(name, w.getName());
  doc.addField(price, w.getPrice());
   doc.addField(category, w.getCat());
   doc.addField(srcType, w.getSrcType());
   docs.add(doc);

   // commit docs to solr server
   server.add(docs);
   server.commit();
}

And I got this exception.

rg.apache.solr.common.SolrException:
Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers2_try_again_later

Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers2_try_again_later

	at  
org 
.apache 
.solr 
.client 
.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java: 
424)
	at  
org 
.apache 
.solr 
.client 
.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java: 
243)
	at  
org 
.apache 
.solr 
.client 
.solrj 
.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)

at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:86)

The solrj wiki/documents seemed to indicate that because multiple  
threads

were calling SolrServer.commit() which in term called
CommonsHttpSolrServer.request() resulting in multiple searchers.  My  
first
thought was to change the configs for autowarming.  But after  
looking at the
autowarm params, I am not sure what can be changed or perhaps a  
different

approach is recommened.

   filterCache
 class=solr.FastLRUCache
 size=512
 initialSize=512
 autowarmCount=0/

   queryResultCache
 class=solr.LRUCache
 size=512
 initialSize=512
 autowarmCount=0/

   documentCache
 class=solr.LRUCache
 size=512
 initialSize=512
 autowarmCount=0/

Your help is much appreciated.

Re: Newbie Question on Custom Query Generation

[Erik Hatcher]

dismax won't quite give you the same query result.  What you can do  
pretty easily, though, is create a QParser and QParserPlugin pair,  
register it solrconfig.xml and then use defType=name registered.   
Pretty straightforward.  Have a look at Solr's various QParserPlugin  
implementations for details.


Erik

On Jan 29, 2010, at 12:30 AM, Abin Mathew wrote:

Hi I want to generate my own customized query from the input string  
entered

by the user. It should look something like this

*Search field : Microsoft*
*
Generated Query*  :
description:microsoft +((tags:microsoft^1.5 title:microsoft^3.0
role:microsoft requi
rement:microsoft company:microsoft city:microsoft)^5.0)  
tags:microsoft^2.0

title:microsoft^3.5 functionalArea:microsoft

*The lucene code we used is like this*
BooleanQuery must = new BooleanQuery();

addToBooleanQuery(must, tags, inputData, synonymAnalyzer, 1.5f);
addToBooleanQuery(must, title, inputData, synonymAnalyzer);
addToBooleanQuery(must, role, inputData, synonymAnalyzer);
addToBooleanQuery(query, description, inputData, synonymAnalyzer);
addToBooleanQuery(must, requirement, inputData, synonymAnalyzer);
addToBooleanQuery(must, company, inputData, standardAnalyzer);
addToBooleanQuery(must, city, inputData, standardAnalyzer);
must.setBoost(5.0f);
query.add(must, Occur.MUST);
addToBooleanQuery(query, tags, includeAll, synonymAnalyzer, 2.0f);
addToBooleanQuery(query, title, includeAll, synonymAnalyzer, 3.5f);
addToBooleanQuery(query, functionalArea, inputData,  
synonymAnalyzer,);

*
In Simple english*
addToBooleanQuery will add the particular field to the query after  
analysing

using the analyser mentioned and setting a boost as specified
So there MUST be a keyword match with any of the fields
tags,title,role,description,requirement,company,city and it SHOULD  
occur

in the fields tags,title and functionalArea.

Hope you have got an idea of my requirement. I am not asking anyone  
to do it
for me. Please let me know where can i start and give me some useful  
tips to
move ahead with this. I believe that it has to do with modifying the  
XML
configuration file and setting the parameters in Dismax handler. But  
I am

still not sure. Please help

Thanks  Regards
Abin Mathew

Re: Newbie Question on Custom Query Generation

[Wangsheng Mei]

What's the point of generating your own query?
Are you sure that solr query syntax cannot satisfy your need?

2010/1/29 Abin Mathew abin.mat...@toostep.com

 Hi I want to generate my own customized query from the input string entered
 by the user. It should look something like this

 *Search field : Microsoft*
 *
 Generated Query*  :
 description:microsoft +((tags:microsoft^1.5 title:microsoft^3.0
 role:microsoft requi
 rement:microsoft company:microsoft city:microsoft)^5.0) tags:microsoft^2.0
 title:microsoft^3.5 functionalArea:microsoft

 *The lucene code we used is like this*
 BooleanQuery must = new BooleanQuery();

 addToBooleanQuery(must, tags, inputData, synonymAnalyzer, 1.5f);
 addToBooleanQuery(must, title, inputData, synonymAnalyzer);
 addToBooleanQuery(must, role, inputData, synonymAnalyzer);
 addToBooleanQuery(query, description, inputData, synonymAnalyzer);
 addToBooleanQuery(must, requirement, inputData, synonymAnalyzer);
 addToBooleanQuery(must, company, inputData, standardAnalyzer);
 addToBooleanQuery(must, city, inputData, standardAnalyzer);
 must.setBoost(5.0f);
 query.add(must, Occur.MUST);
 addToBooleanQuery(query, tags, includeAll, synonymAnalyzer, 2.0f);
 addToBooleanQuery(query, title, includeAll, synonymAnalyzer, 3.5f);
 addToBooleanQuery(query, functionalArea, inputData, synonymAnalyzer,);
 *
 In Simple english*
 addToBooleanQuery will add the particular field to the query after
 analysing
 using the analyser mentioned and setting a boost as specified
 So there MUST be a keyword match with any of the fields
 tags,title,role,description,requirement,company,city and it SHOULD occur
 in the fields tags,title and functionalArea.

 Hope you have got an idea of my requirement. I am not asking anyone to do
 it
 for me. Please let me know where can i start and give me some useful tips
 to
 move ahead with this. I believe that it has to do with modifying the XML
 configuration file and setting the parameters in Dismax handler. But I am
 still not sure. Please help

 Thanks  Regards
 Abin Mathew




-- 
梅旺生

Re: Newbie Question on Custom Query Generation

[Abin Mathew]

Hi, I realized the power of Dismax Query Handler recently and now I
dont need to generate my own query since Dismax is giving better
results.Thanks a lot

2010/1/29 Wangsheng Mei hairr...@gmail.com:
 What's the point of generating your own query?
 Are you sure that solr query syntax cannot satisfy your need?

 2010/1/29 Abin Mathew abin.mat...@toostep.com

 Hi I want to generate my own customized query from the input string entered
 by the user. It should look something like this

 *Search field : Microsoft*
 *
 Generated Query*  :
 description:microsoft +((tags:microsoft^1.5 title:microsoft^3.0
 role:microsoft requi
 rement:microsoft company:microsoft city:microsoft)^5.0) tags:microsoft^2.0
 title:microsoft^3.5 functionalArea:microsoft

 *The lucene code we used is like this*
 BooleanQuery must = new BooleanQuery();

 addToBooleanQuery(must, tags, inputData, synonymAnalyzer, 1.5f);
 addToBooleanQuery(must, title, inputData, synonymAnalyzer);
 addToBooleanQuery(must, role, inputData, synonymAnalyzer);
 addToBooleanQuery(query, description, inputData, synonymAnalyzer);
 addToBooleanQuery(must, requirement, inputData, synonymAnalyzer);
 addToBooleanQuery(must, company, inputData, standardAnalyzer);
 addToBooleanQuery(must, city, inputData, standardAnalyzer);
 must.setBoost(5.0f);
 query.add(must, Occur.MUST);
 addToBooleanQuery(query, tags, includeAll, synonymAnalyzer, 2.0f);
 addToBooleanQuery(query, title, includeAll, synonymAnalyzer, 3.5f);
 addToBooleanQuery(query, functionalArea, inputData, synonymAnalyzer,);
 *
 In Simple english*
 addToBooleanQuery will add the particular field to the query after
 analysing
 using the analyser mentioned and setting a boost as specified
 So there MUST be a keyword match with any of the fields
 tags,title,role,description,requirement,company,city and it SHOULD occur
 in the fields tags,title and functionalArea.

 Hope you have got an idea of my requirement. I am not asking anyone to do
 it
 for me. Please let me know where can i start and give me some useful tips
 to
 move ahead with this. I believe that it has to do with modifying the XML
 configuration file and setting the parameters in Dismax handler. But I am
 still not sure. Please help

 Thanks  Regards
 Abin Mathew




 --
 梅旺生

Re: Newbie question

[Wayne Pope]

Hello Shalin,

thaks you for your help. yes it answers my question.

Much appreciated



Shalin Shekhar Mangar wrote:
 
 On Tue, May 12, 2009 at 9:48 PM, Wayne Pope
 waynemailingli...@gmail.comwrote:
 

 I have this request:


 http://localhost:8983/solr/select?start=0rows=20qt=dismaxq=copyhl=truehl.snippets=4hl.fragsize=50facet=truefacet.mincount=1facet.limit=8facet.field=typefq=company-id%3A1wt=javabinversion=2.2

 (I've been using this to see it rendered in the browser:

 http://localhost:8983/solr/select?indent=onversion=2.2q=copystart=0rows=10fl=*%2Cscoreqt=standardwt=standardexplainOther=hl=onhl.fl=featureshl=truehl.fragsize=50
 )


 that I've been trying out. I get a good responce - however the
 hl.fragsize
 is ignored and the hl.fragsize in the solrconfig.xml is ignored. Instead
 I
 get back the whole document (10,000 chars!) in the doc txt field. And
 bizarely the response header is this:

 
 hl.fragsize is relevant only for the snippets created by the highlighter.
 The returned fields will always have the complete data for a document.
 Does
 that answer your question?
 
 -- 
 Regards,
 Shalin Shekhar Mangar.
 
 

-- 
View this message in context: 
http://www.nabble.com/Newbie-question-tp23505802p23518485.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie question

[Shalin Shekhar Mangar]

On Tue, May 12, 2009 at 9:48 PM, Wayne Pope waynemailingli...@gmail.comwrote:


 I have this request:


 http://localhost:8983/solr/select?start=0rows=20qt=dismaxq=copyhl=truehl.snippets=4hl.fragsize=50facet=truefacet.mincount=1facet.limit=8facet.field=typefq=company-id%3A1wt=javabinversion=2.2

 (I've been using this to see it rendered in the browser:

 http://localhost:8983/solr/select?indent=onversion=2.2q=copystart=0rows=10fl=*%2Cscoreqt=standardwt=standardexplainOther=hl=onhl.fl=featureshl=truehl.fragsize=50
 )


 that I've been trying out. I get a good responce - however the hl.fragsize
 is ignored and the hl.fragsize in the solrconfig.xml is ignored. Instead I
 get back the whole document (10,000 chars!) in the doc txt field. And
 bizarely the response header is this:


hl.fragsize is relevant only for the snippets created by the highlighter.
The returned fields will always have the complete data for a document. Does
that answer your question?

-- 
Regards,
Shalin Shekhar Mangar.

Re: newbie question about indexing RSS feeds with SOLR

[Koji Sekiguchi]

Just an FYI: I've never tried, but there seems to be RSS feed sample in DIH:

http://wiki.apache.org/solr/DataImportHandler#head-e68aa93c9ca7b8d261cede2bf1d6110ab1725476

Koji

Tom H wrote:
 Hi,

 I've just downloaded solr and got it working, it seems pretty cool.

 I have a project which needs to maintain an index of articles that were
 published on the web via rss feed.

 Basically I need to watch some rss feeds, and search and index the items
 to be searched.

 Additionally, I need to run jobs based on particular keywords or events
 during parsing.

 is this something that I can do with SOLR? are their any related
 projects using SOLR that are better suited to indexing specific xml
 types like RSS?

 I had a look at the project enormo which appears to be a property
 lettings and sales listing aggregator. But I can see that they must have
 solved some of the problems I am thinking of such as scheduled indexing
 of remote resources, and writing a parser to get data fields from some
 other sites templates.

 Any advice would be welcome...

 Many Thanks,

 Tom




   

Re: newbie question --- multiple schemas

[Chris Hostetter]

: Is it possible to define more than one schema? I'm reading the example 
: schema.xml. It seems that we can only define one schema? What about if I 
: want to define one schema for document type A and another schema for 
: document type B?

there are lots of ways to tackle a problem like this, depending on your 
specific needs, some starting points are...

http://wiki.apache.org/solr/MultipleIndexes



-Hoss

Re: newbie question --- multiple schemas

[Noble Paul നോബിള്‍ नोब्ळ्]

have two different cores and you can have separate schema for each.

On Thu, Jan 29, 2009 at 1:20 PM, Cheng Zhang zhangyongji...@yahoo.com wrote:
 Hello,

 Is it possible to define more than one schema? I'm reading the example 
 schema.xml. It seems that we can only define one schema? What about if I want 
 to define one schema for document type A and another schema for document type 
 B?

 Thanks a lot,
 Kevin





-- 
--Noble Paul

Re: Newbie Question boosting

[Erick Erickson]

I can help a bit with 2...

First, keep in mind the difference between index and query
time boosting:

From Hossman:
..Index time field bosts are a way to express things like
this documents title is worth twice as much as the title of most documents
query time boosts are a way to express i care about matches on this clause
of my query twice as much as i do about matches to other clauses of my
query.

Boosting doesn't do anything if your search doesn't use the field you've
boosted
on, but that's only relevant for index-time boosts. You can't have a
query-time
boost that doesn't reference the field since that's how you boost at query
time by definition.

So, say you have documents that you think are more important than
run-of-the-mill
documents. An example could be an animals database. You might want
documents from scholarly sources to have a greater weight, thus be more
likely
to appear at the top of your search results. These would be fine documents
to
boost the fields for at index time. That would mean that the scholarly
documents
would tend to be higher up in your search results.

For query time, say you have a genealogy database. Books in this category
often have
titles like The Erickson family history in New England, 1700-1930. Now say
you
have separate fields for title and text. A user searches for erickson. You
want the
title to weigh much more than the text since it's much more likely that the
user
would find that document more valuable than some document about the Swanson
family that mentions somewhere in the text that Olaf Swanson married Edna
Erickson. So you'd form some query like: title:erickson^10 text:erickson


HTH
Erick

On Thu, Dec 11, 2008 at 1:47 AM, ayyanar
[EMAIL PROTECTED]wrote:


 I read many articles on boosting still iam not so clear on boosting. Can
 anyone explain the following questions with examples?

 1) Can you given an example for field level boosting and document level
 boosting and the difference between two?

 2) If we set the boost at field level (index time), should the query
 contains the that particular field?
 For example, if we set the boost for title field, should we create the
 termquery for title field?

 Also, based on your experience, can you explain why you need the boosting.

 Thanks,
 Ayyanar. A
 --
 View this message in context:
 http://www.nabble.com/Newbie-Question-boosting-tp20950268p20950268.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Newbie question - using existing Lucene Index

[Grant Ingersoll]

On Dec 3, 2008, at 11:53 AM, Sudarsan, Sithu D. wrote:


Hi All,

Using Lucene, index has been created. It has five different fields.

How to just use those index from SOLR for searching? I tried changing
the schema as in tutorial, and copied the index to the data directory,
but all searches return empty and no error message!


You also need to make sure the analyzers in the schema.xml are setup  
the same way as the one you indexed with to create the index originally.


You might also try going to wherever you are running, i.e. http://localhost:8983/solr/admin 
 (or whatever you're URL is) and entering *:* into the query box  
there.  This should return all documents in the index and doesn't  
require any analysis.


Also, check your logs to see if there were any exceptions on startup.




Is there a sample project available which shows using tomcat as the  
web

engine rather than jetty?


Instructions should be on the Wiki: http://wiki.apache.org/solr





Your help is appreciated,
Sincerely,
Sithu D Sudarsan

ORISE Fellow, DESE/OSEL/CDRH
WO62 - 3209

GRA, UALR

[EMAIL PROTECTED]
[EMAIL PROTECTED]



--
Grant Ingersoll

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ

RE: newbie question on SOLR distributed searches with many shards

[Gerald De Conto]

Thx Paul

How would I do so? i.e. would you be able to provide a simple example of 
putting shard info into the SearchHandlers defaults or invariants section?

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Noble Paul 
??? ??
Sent: Tuesday, November 25, 2008 8:29 PM
To: solr-user@lucene.apache.org
Subject: Re: newbie question on SOLR distributed searches with many shards

anything that is passed as a request parameter can be put into the
SearchHandlers defaults or invariants section .


This is equivalent to passing the shard url in the request

However this expects that you may need to setup a loadbalancer if a
shard hhos more than one host


On Wed, Nov 26, 2008 at 12:25 AM, Gerald De Conto
[EMAIL PROTECTED] wrote:
 I wasn't able to find examples/anything via google so thought I'd ask:



 Say I want to implement a solution using distributed searches with many
 shards in SOLR 1.3.0. Also, say there are too many shards to pass in
 via the URL (dozens, hundreds, whatever)



 Is there a way to specify in solrconfig.xml (or elsewhere) a list of the
 shard URLs to use?



 I saw references to a shards.txt but no info on it.  I also saw bits of
 info that suggested that there MIGHT be another way to do this.



 Any info appreciated on doing this sort of distributed search.



 thx





-- 
--Noble Paul

Re: newbie question on SOLR distributed searches with many shards

[Noble Paul നോബിള്‍ नोब्ळ्]

Look at the file
http://svn.apache.org/viewvc/lucene/solr/trunk/example/solr/conf/solrconfig.xml?revision=720502view=markup

and take a look at the

line
requestHandler name=standard class=solr.SearchHandler default=true

you may see the defaults there.
you can add your param just the way the echoParams is configured

On Wed, Nov 26, 2008 at 9:32 PM, Gerald De Conto
[EMAIL PROTECTED] wrote:
 Thx Paul

 How would I do so? i.e. would you be able to provide a simple example of 
 putting shard info into the SearchHandlers defaults or invariants section?

 -Original Message-
 From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Noble Paul 
 ??? ??
 Sent: Tuesday, November 25, 2008 8:29 PM
 To: solr-user@lucene.apache.org
 Subject: Re: newbie question on SOLR distributed searches with many shards

 anything that is passed as a request parameter can be put into the
 SearchHandlers defaults or invariants section .


 This is equivalent to passing the shard url in the request

 However this expects that you may need to setup a loadbalancer if a
 shard hhos more than one host


 On Wed, Nov 26, 2008 at 12:25 AM, Gerald De Conto
 [EMAIL PROTECTED] wrote:
 I wasn't able to find examples/anything via google so thought I'd ask:



 Say I want to implement a solution using distributed searches with many
 shards in SOLR 1.3.0. Also, say there are too many shards to pass in
 via the URL (dozens, hundreds, whatever)



 Is there a way to specify in solrconfig.xml (or elsewhere) a list of the
 shard URLs to use?



 I saw references to a shards.txt but no info on it.  I also saw bits of
 info that suggested that there MIGHT be another way to do this.



 Any info appreciated on doing this sort of distributed search.



 thx





 --
 --Noble Paul




-- 
--Noble Paul

Re: newbie question on SOLR distributed searches with many shards

[Noble Paul നോബിള്‍ नोब्ळ्]

anything that is passed as a request parameter can be put into the
SearchHandlers defaults or invariants section .


This is equivalent to passing the shard url in the request

However this expects that you may need to setup a loadbalancer if a
shard hhos more than one host


On Wed, Nov 26, 2008 at 12:25 AM, Gerald De Conto
[EMAIL PROTECTED] wrote:
 I wasn't able to find examples/anything via google so thought I'd ask:



 Say I want to implement a solution using distributed searches with many
 shards in SOLR 1.3.0. Also, say there are too many shards to pass in
 via the URL (dozens, hundreds, whatever)



 Is there a way to specify in solrconfig.xml (or elsewhere) a list of the
 shard URLs to use?



 I saw references to a shards.txt but no info on it.  I also saw bits of
 info that suggested that there MIGHT be another way to do this.



 Any info appreciated on doing this sort of distributed search.



 thx





-- 
--Noble Paul

Re: Newbie Question - getting search results from dataimport request handler

[Chris Hostetter]

:  Logging an error and returning successfully (without adding any docs) is
:  still inconsistent with the way all other RequestHandlers work: fail the
:  request.
: 
:  I know DIH isn't a typical RequestHandler, but some things (like failing
:  on failure) seem like they should be a given.
: SOLR-842 .
: DIH is an ETL tool pretending to be a RequestHandler. Originally it
: was built to run outside of Solr using SolrJ. For better integration
: and ease of use we changed it later.
: 
: SOLR-853 aims to achieve the oroginal goal
: 
: The goal of DIH is to become a full featured ETL tool.

Understood ... but shouldn't ETL Tools fail on failure ?

I mean forget Solr for a minute:   If i've got a standalone ETL Tool that 
runs as a daemon, and on startup it logs some error messages because i've 
got bad configs (and it can tell the fields i've listed for my 
'target' system don't exist there) should it report success everytime i 
push data to it?

Based on this thread, that's what it sounds like DIH is doing right now in 
situations like this.

If nothing else, we could give DIH a way to check the global
abortOnConfigurationError value from solrconfig.xml and make it's 
decisison that way.



-Hoss

Re: Newbie Question - getting search results from dataimport request handler

[Noble Paul നോബിള്‍ नोब्ळ्]

On Mon, Nov 24, 2008 at 7:25 AM, Chris Hostetter
[EMAIL PROTECTED] wrote:

 :  Logging an error and returning successfully (without adding any docs) is
 :  still inconsistent with the way all other RequestHandlers work: fail the
 :  request.
 : 
 :  I know DIH isn't a typical RequestHandler, but some things (like failing
 :  on failure) seem like they should be a given.
 : SOLR-842 .
 : DIH is an ETL tool pretending to be a RequestHandler. Originally it
 : was built to run outside of Solr using SolrJ. For better integration
 : and ease of use we changed it later.
 :
 : SOLR-853 aims to achieve the oroginal goal
 :
 : The goal of DIH is to become a full featured ETL tool.

 Understood ... but shouldn't ETL Tools fail on failure ?

 I mean forget Solr for a minute:   If i've got a standalone ETL Tool that
 runs as a daemon, and on startup it logs some error messages because i've
 got bad configs (and it can tell the fields i've listed for my
 'target' system don't exist there) should it report success everytime i
 push data to it?

 Based on this thread, that's what it sounds like DIH is doing right now in
 situations like this.

 If nothing else, we could give DIH a way to check the global
 abortOnConfigurationError value from solrconfig.xml and make it's
 decisison that way
We considered these. The severity of errors are very much specific to
the source of data. It is very unlikely that a DB source throws up
errors. In xml data sources say out of x urls 1 or two are wrong,
would the user wish to ignore or want to abort the entire import.


So we decided to give more options and the implementations are left to
the EntityProcessor. Moreover the default is set to onError=abort





 -Hoss





-- 
--Noble Paul

RE: Newbie Question - getting search results from dataimport request handler

[Norskog, Lance]

As part of the ETL effort, please consider how to integrate with these two 
open-source ETL systems. I'm not asking for an implementation, just suggesting 
that having a concrete context will help you in the architecture phase.

http://kettle.pentaho.org/
http://www.talend.com/products-data-integration/talend-open-studio.php 

Thanks,

Lance

-Original Message-
From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED] 
Sent: Friday, November 21, 2008 8:12 PM
To: solr-user@lucene.apache.org
Subject: Re: Newbie Question - getting search results from dataimport request 
handler

On Sat, Nov 22, 2008 at 3:10 AM, Chris Hostetter [EMAIL PROTECTED] wrote:

 :  it might be worth considering a new @attribute for fields to 
 indicate
 :  that they are going to be used purely as component fields (ie: 
 your
 :  first-name/last-name example) and then have DIH pass all 
 non-component
 :  fields along and error if undefined in the schema just like other 
 updating
 :  RequestHandlers do.
 : 
 :  either that, or require that people declaure indexed=false
 :  stored=false fields in the schema for these intermediate 
 component
 :  fields so that we can properly warn then when DIH is getting data 
 it
 :  doesn't know what to do with -- protecting people from field name 
 typos
 :  and returning errors instead of silently ignoring unexpected input 
 is
 :  fairly important behavir -- especially for new users.

 : Actually it is done by DIH . When the dataconfig is loaded DIH 
 reports
 : these information on the console. though it is limited , it helps to 
 a
 : certain extent

 Hmmm.

 Logging an error and returning successfully (without adding any docs) 
 is still inconsistent with the way all other RequestHandlers work: 
 fail the request.

 I know DIH isn't a typical RequestHandler, but some things (like 
 failing on failure) seem like they should be a given.
SOLR-842 .
DIH is an ETL tool pretending to be a RequestHandler. Originally it was built 
to run outside of Solr using SolrJ. For better integration and ease of use we 
changed it later.

SOLR-853 aims to achieve the oroginal goal

The goal of DIH is to become a full featured ETL tool.






 -Hoss





--
--Noble Paul

Re: Newbie Question - getting search results from dataimport request handler

[Chris Hostetter]

:  it might be worth considering a new @attribute for fields to indicate
:  that they are going to be used purely as component fields (ie: your
:  first-name/last-name example) and then have DIH pass all non-component
:  fields along and error if undefined in the schema just like other updating
:  RequestHandlers do.
: 
:  either that, or require that people declaure indexed=false
:  stored=false fields in the schema for these intermediate component
:  fields so that we can properly warn then when DIH is getting data it
:  doesn't know what to do with -- protecting people from field name typos
:  and returning errors instead of silently ignoring unexpected input is
:  fairly important behavir -- especially for new users.

: Actually it is done by DIH . When the dataconfig is loaded DIH reports
: these information on the console. though it is limited , it helps to a
: certain extent

Hmmm. 

Logging an error and returning successfully (without adding any docs) is 
still inconsistent with the way all other RequestHandlers work: fail the 
request.

I know DIH isn't a typical RequestHandler, but some things (like failing 
on failure) seem like they should be a given.



-Hoss

Re: Newbie Question - getting search results from dataimport request handler

[Noble Paul നോബിള്‍ नोब्ळ्]

On Sat, Nov 22, 2008 at 3:10 AM, Chris Hostetter
[EMAIL PROTECTED] wrote:

 :  it might be worth considering a new @attribute for fields to indicate
 :  that they are going to be used purely as component fields (ie: your
 :  first-name/last-name example) and then have DIH pass all non-component
 :  fields along and error if undefined in the schema just like other updating
 :  RequestHandlers do.
 : 
 :  either that, or require that people declaure indexed=false
 :  stored=false fields in the schema for these intermediate component
 :  fields so that we can properly warn then when DIH is getting data it
 :  doesn't know what to do with -- protecting people from field name typos
 :  and returning errors instead of silently ignoring unexpected input is
 :  fairly important behavir -- especially for new users.

 : Actually it is done by DIH . When the dataconfig is loaded DIH reports
 : these information on the console. though it is limited , it helps to a
 : certain extent

 Hmmm.

 Logging an error and returning successfully (without adding any docs) is
 still inconsistent with the way all other RequestHandlers work: fail the
 request.

 I know DIH isn't a typical RequestHandler, but some things (like failing
 on failure) seem like they should be a given.
SOLR-842 .
DIH is an ETL tool pretending to be a RequestHandler. Originally it
was built to run outside of Solr using SolrJ. For better integration
and ease of use we changed it later.

SOLR-853 aims to achieve the oroginal goal

The goal of DIH is to become a full featured ETL tool.






 -Hoss





-- 
--Noble Paul

Re: Newbie Question - getting search results from dataimport request handler

[Noble Paul നോബിള്‍ नोब्ळ्]

On Sat, Nov 15, 2008 at 6:33 AM, Chris Hostetter
[EMAIL PROTECTED] wrote:

 :  Is here a bug in DIH that caused these unrecognized fields to be ignored,
 :  or is it possible the errors were logged (by DUH2 maybe? ... it's been a
 :  while since i looked at the update code) but DIH didn't notice them and
 :  reported success anyway?
 :
 :
 : If the data contains a field name which is not defined in the schema.xml,
 : then DIH ignores it. This is a very common use-case where you may want to
 : process intermediate data and add it to a completely new field. For example,
 : if you have first-name and last-name coming in from DB and you want to
 : combine them into a new field name with TemplateTransformer.

 A i see.  so it's a feature that sometimes acts like a bug :)

 it might be worth considering a new @attribute for fields to indicate
 that they are going to be used purely as component fields (ie: your
 first-name/last-name example) and then have DIH pass all non-component
 fields along and error if undefined in the schema just like other updating
 RequestHandlers do.

 either that, or require that people declaure indexed=false
 stored=false fields in the schema for these intermediate component
 fields so that we can properly warn then when DIH is getting data it
 doesn't know what to do with -- protecting people from field name typos
 and returning errors instead of silently ignoring unexpected input is
 fairly important behavir -- especially for new users.

Actually it is done by DIH . When the dataconfig is loaded DIH reports
these information on the console. though it is limited , it helps to a
certain extent
 -Hoss





-- 
--Noble Paul

Re: Newbie Question - getting search results from dataimport request handler

[Chris Hostetter]

: You need to modify the schema which came with Solr to suit your data. There

If i'm understanding this thread correctly, DIH ran successfully, docs 
were created, some fields were stored and indexed (because they did exist 
in the schema) but other fields the user was attempting to create didn't 
exist...

:  So there is data there, however I have not defined any 'Fields' in my
:  data-config and am hoping my column names are the 'fields', yet I'm not
:  seeing any of them being returned in the 'doc' node below 

...presumably the example schema.xml was being used, where there is no 
dynamicField name=* ... / ... so shouldn't the unrecognized field 
names have generated an error during indexing?

Is here a bug in DIH that caused these unrecognized fields to be ignored, 
or is it possible the errors were logged (by DUH2 maybe? ... it's been a 
while since i looked at the update code) but DIH didn't notice them and 
reported success anyway?

I smell a bug somewhere, i just don't know enough about the code to know 
where hte smell is coming from.

-Hoss

Re: Newbie Question - getting search results from dataimport request handler

[Shalin Shekhar Mangar]

On Thu, Nov 13, 2008 at 3:52 AM, Chris Hostetter
[EMAIL PROTECTED]wrote:


 : You need to modify the schema which came with Solr to suit your data.
 There

 If i'm understanding this thread correctly, DIH ran successfully, docs
 were created, some fields were stored and indexed (because they did exist
 in the schema) but other fields the user was attempting to create didn't
 exist...

 :  So there is data there, however I have not defined any 'Fields' in my
 :  data-config and am hoping my column names are the 'fields', yet I'm not
 :  seeing any of them being returned in the 'doc' node below

 ...presumably the example schema.xml was being used, where there is no
 dynamicField name=* ... / ... so shouldn't the unrecognized field
 names have generated an error during indexing?

 Is here a bug in DIH that caused these unrecognized fields to be ignored,
 or is it possible the errors were logged (by DUH2 maybe? ... it's been a
 while since i looked at the update code) but DIH didn't notice them and
 reported success anyway?


If the data contains a field name which is not defined in the schema.xml,
then DIH ignores it. This is a very common use-case where you may want to
process intermediate data and add it to a completely new field. For example,
if you have first-name and last-name coming in from DB and you want to
combine them into a new field name with TemplateTransformer.

-- 
Regards,
Shalin Shekhar Mangar.

Re: Newbie Question - getting search results from dataimport request handler

[Noble Paul നോബിള്‍ नोब्ळ्]

On Thu, Nov 13, 2008 at 3:52 AM, Chris Hostetter
[EMAIL PROTECTED] wrote:

 : You need to modify the schema which came with Solr to suit your data. There

 If i'm understanding this thread correctly, DIH ran successfully, docs
 were created, some fields were stored and indexed (because they did exist
 in the schema) but other fields the user was attempting to create didn't
 exist...

 :  So there is data there, however I have not defined any 'Fields' in my
 :  data-config and am hoping my column names are the 'fields', yet I'm not
 :  seeing any of them being returned in the 'doc' node below

 ...presumably the example schema.xml was being used, where there is no
 dynamicField name=* ... / ... so shouldn't the unrecognized field
 names have generated an error during indexing?

 Is here a bug in DIH that caused these unrecognized fields to be ignored,
 or is it possible the errors were logged (by DUH2 maybe? ... it's been a
 while since i looked at the update code) but DIH didn't notice them and
 reported success anyway?

DIH in the 1.3 release had a bug with handling dynamic fields (SOLR-742)

 I smell a bug somewhere, i just don't know enough about the code to know
 where hte smell is coming from.

 -Hoss





-- 
--Noble Paul

Re: Newbie Question - getting search results from dataimport request handler

[Noble Paul നോബിള്‍ नोब्ळ्]

you cannot query the DIH. It can only do indexing
after indexing you must do the indexing on the regular query interface

On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED] wrote:
 My Question is: what is the format of a search that will return data?
 i.e. /solr/select?q=developerqt=dataimport (won't work) nor will 
 /solr/dataimport?q=developer (won't work)
 HTTP ERROR: 404
 NOT_FOUND
 RequestURI=/solr/dataimport

 I have created a 'dataimport' set that contains data from a sql db.

 I can view meta data from this url: /solr/dataimport
 response
 −
 lst name=responseHeader
 int name=status0/int
 int name=QTime0/int
 /lst
 −
 lst name=initArgs
 −
 lst name=defaults
 str name=configdata-config.xml/str
 /lst
 /lst
 str name=statusidle/str
 str name=importResponse/
 −
 lst name=statusMessages
 str name=Total Requests made to DataSource1/str
 str name=Total Rows Fetched10/str
 str name=Total Documents Skipped0/str
 str name=Full Dump Started2008-11-10 21:51:40/str
 str name=Time taken 0:0:4.594/str
 /lst
 −
 str name=WARNING
 This response format is experimental.  It is likely to change in the future.
 /str
 /response

 I can verify that the data is there by going through 
 /solr/admin/dataimport.jsp and doing 'verbose' true and debug now.
 It shows me the xml data set on the right as such:

 response
 −
 lst name=responseHeader
 int name=status0/int
 int name=QTime4594/int
 /lst
 −
 lst name=initArgs
 −
 lst name=defaults
 str name=configdata-config.xml/str
 /lst
 /lst
 str name=commandfull-import/str
 str name=modedebug/str
 −
 arr name=documents
 −
 arr
 −
 arr
 int87133/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87134/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87135/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87136/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87137/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87138/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87139/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87140/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87141/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87142/int
 /arr
 /arr
 /arr
 −
 lst name=verbose-output
 −
 lst name=entity:item
 −
 lst name=document#1
 −
 str name=query
 SELECT  j.id  , j.title  ,  FROM  dbo.jobs j WITH (NOLOCK)  
 LEFT  WHERE j.siteid = 46 and j.active = 1
 /str
 str name=time-taken0:0:4.578/str
 str--- row #1-/str
 str name=zip/
 str name=urltitleOperations Software Developer Job/str
 str name=altlocationSAN ANTONIO, TX, 78229/str
 str name=alttitleOpe…


 Here is my solconfig.xml
 …
 requestHandler name=dataimport 
 class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
  str name=configdata-config.xml/str
/lst
  /requestHandler
 …
 Data-config.xml is in the same dir as solconfig.xml

 My data-config.xml is like any other:
 dataConfig
dataSource driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
 url=jdbc:sqlserver://:1433;databaseName=x user=x 
 password=x /
document name=jobs
entity name=item pk=id query=SELECT  j.id
, j.title
…
FROM
dbo.jobs …
WHERE j.siteid = 46 and 
 j.active = 1
deltaQuery=select id from dbo.jobs where lastmodified  
 '${dataimporter.last_index_time}'

/entity
/document
 /dataConfig

 I'm using win xp with apache – and jetty + solr 1.3.0

 Thanks






-- 
--Noble Paul

RE: Newbie Question - getting search results from dataimport request handler

[Kevin Penny]

Ok - and what would that be? (query interface)

I need the URL format that would work in this situation to return data from my 
setup.

I've gone through the tutorial and used execution strings like:
http://localhost:8983/solr/select/?indent=onq=videosort=price+desc
etc however I'm working with sql data and not xml data.

Thanks

-Original Message-
From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED]
Sent: Monday, November 10, 2008 10:18 PM
To: solr-user@lucene.apache.org
Subject: Re: Newbie Question - getting search results from dataimport request 
handler

you cannot query the DIH. It can only do indexing
after indexing you must do the indexing on the regular query interface

On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED] wrote:
 My Question is: what is the format of a search that will return data?
 i.e. /solr/select?q=developerqt=dataimport (won't work) nor will 
 /solr/dataimport?q=developer (won't work)
 HTTP ERROR: 404
 NOT_FOUND
 RequestURI=/solr/dataimport

 I have created a 'dataimport' set that contains data from a sql db.

 I can view meta data from this url: /solr/dataimport
 response
 −
 lst name=responseHeader
 int name=status0/int
 int name=QTime0/int
 /lst
 −
 lst name=initArgs
 −
 lst name=defaults
 str name=configdata-config.xml/str
 /lst
 /lst
 str name=statusidle/str
 str name=importResponse/
 −
 lst name=statusMessages
 str name=Total Requests made to DataSource1/str
 str name=Total Rows Fetched10/str
 str name=Total Documents Skipped0/str
 str name=Full Dump Started2008-11-10 21:51:40/str
 str name=Time taken 0:0:4.594/str
 /lst
 −
 str name=WARNING
 This response format is experimental.  It is likely to change in the future.
 /str
 /response

 I can verify that the data is there by going through 
 /solr/admin/dataimport.jsp and doing 'verbose' true and debug now.
 It shows me the xml data set on the right as such:

 response
 −
 lst name=responseHeader
 int name=status0/int
 int name=QTime4594/int
 /lst
 −
 lst name=initArgs
 −
 lst name=defaults
 str name=configdata-config.xml/str
 /lst
 /lst
 str name=commandfull-import/str
 str name=modedebug/str
 −
 arr name=documents
 −
 arr
 −
 arr
 int87133/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87134/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87135/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87136/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87137/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87138/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87139/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87140/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87141/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87142/int
 /arr
 /arr
 /arr
 −
 lst name=verbose-output
 −
 lst name=entity:item
 −
 lst name=document#1
 −
 str name=query
 SELECT  j.id  , j.title  ,  FROM  dbo.jobs j WITH (NOLOCK)  
 LEFT  WHERE j.siteid = 46 and j.active = 1
 /str
 str name=time-taken0:0:4.578/str
 str--- row #1-/str
 str name=zip/
 str name=urltitleOperations Software Developer Job/str
 str name=altlocationSAN ANTONIO, TX, 78229/str
 str name=alttitleOpe…


 Here is my solconfig.xml
 …
 requestHandler name=dataimport 
 class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
  str name=configdata-config.xml/str
/lst
  /requestHandler
 …
 Data-config.xml is in the same dir as solconfig.xml

 My data-config.xml is like any other:
 dataConfig
dataSource driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
 url=jdbc:sqlserver://:1433;databaseName=x user=x 
 password=x /
document name=jobs
entity name=item pk=id query=SELECT  j.id
, j.title
…
FROM
dbo.jobs …
WHERE j.siteid = 46 and 
 j.active = 1
deltaQuery=select id from dbo.jobs where lastmodified  
 '${dataimporter.last_index_time}'

/entity
/document
 /dataConfig

 I'm using win xp with apache – and jetty + solr 1.3.0

 Thanks






--
--Noble Paul

Re: Newbie Question - getting search results from dataimport request handler

[Noble Paul നോബിള്‍ नोब्ळ्]

XML is just an intermediete data format Solr internally has no XML
data. When the data comes out XML is just another representation of
the same data.

Whether you put in data using XML or DB (SQL) it all goes into the
same index . Query must be done on that index using the syntax
http://localhost:8983/solr/select/?q=your-query-goes-here

On Tue, Nov 11, 2008 at 9:55 AM, Kevin Penny [EMAIL PROTECTED] wrote:
 Ok - and what would that be? (query interface)

 I need the URL format that would work in this situation to return data from 
 my setup.

 I've gone through the tutorial and used execution strings like:
 http://localhost:8983/solr/select/?indent=onq=videosort=price+desc
 etc however I'm working with sql data and not xml data.

 Thanks

 -Original Message-
 From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED]
 Sent: Monday, November 10, 2008 10:18 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Newbie Question - getting search results from dataimport request 
 handler

 you cannot query the DIH. It can only do indexing
 after indexing you must do the indexing on the regular query interface

 On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED] wrote:
 My Question is: what is the format of a search that will return data?
 i.e. /solr/select?q=developerqt=dataimport (won't work) nor will 
 /solr/dataimport?q=developer (won't work)
 HTTP ERROR: 404
 NOT_FOUND
 RequestURI=/solr/dataimport

 I have created a 'dataimport' set that contains data from a sql db.

 I can view meta data from this url: /solr/dataimport
 response
 −
 lst name=responseHeader
 int name=status0/int
 int name=QTime0/int
 /lst
 −
 lst name=initArgs
 −
 lst name=defaults
 str name=configdata-config.xml/str
 /lst
 /lst
 str name=statusidle/str
 str name=importResponse/
 −
 lst name=statusMessages
 str name=Total Requests made to DataSource1/str
 str name=Total Rows Fetched10/str
 str name=Total Documents Skipped0/str
 str name=Full Dump Started2008-11-10 21:51:40/str
 str name=Time taken 0:0:4.594/str
 /lst
 −
 str name=WARNING
 This response format is experimental.  It is likely to change in the future.
 /str
 /response

 I can verify that the data is there by going through 
 /solr/admin/dataimport.jsp and doing 'verbose' true and debug now.
 It shows me the xml data set on the right as such:

 response
 −
 lst name=responseHeader
 int name=status0/int
 int name=QTime4594/int
 /lst
 −
 lst name=initArgs
 −
 lst name=defaults
 str name=configdata-config.xml/str
 /lst
 /lst
 str name=commandfull-import/str
 str name=modedebug/str
 −
 arr name=documents
 −
 arr
 −
 arr
 int87133/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87134/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87135/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87136/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87137/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87138/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87139/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87140/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87141/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87142/int
 /arr
 /arr
 /arr
 −
 lst name=verbose-output
 −
 lst name=entity:item
 −
 lst name=document#1
 −
 str name=query
 SELECT  j.id  , j.title  ,  FROM  dbo.jobs j WITH (NOLOCK)  
 LEFT  WHERE j.siteid = 46 and j.active = 1
 /str
 str name=time-taken0:0:4.578/str
 str--- row #1-/str
 str name=zip/
 str name=urltitleOperations Software Developer Job/str
 str name=altlocationSAN ANTONIO, TX, 78229/str
 str name=alttitleOpe…


 Here is my solconfig.xml
 …
 requestHandler name=dataimport 
 class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
  str name=configdata-config.xml/str
/lst
  /requestHandler
 …
 Data-config.xml is in the same dir as solconfig.xml

 My data-config.xml is like any other:
 dataConfig
dataSource driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
 url=jdbc:sqlserver://:1433;databaseName=x user=x 
 password=x /
document name=jobs
entity name=item pk=id query=SELECT  j.id
, j.title
…
FROM
dbo.jobs …
WHERE j.siteid = 46 and 
 j.active = 1
deltaQuery=select id from dbo.jobs where lastmodified  
 '${dataimporter.last_index_time}'

/entity
/document
 /dataConfig

 I'm using win xp with apache – and jetty + solr 1.3.0

 Thanks






 --
 --Noble Paul




-- 
--Noble Paul

Re: Newbie Question - getting search results from dataimport request handler

[Noble Paul നോബിള്‍ नोब्ळ्]

search for *:* and see if the index indeed has the documents .
Once you ensure the docs are there go through the lucene query syntax
and check your query

On Tue, Nov 11, 2008 at 10:07 AM, Kevin Penny [EMAIL PROTECTED] wrote:
 Ok so I executed a:
 solr/dataimport?command=full-import
 then I checked here:
 solr/dataimport

 I get a good xml message (figure 1.1) showing me that 125 records have been 
 indexed (good) and I know one of them contains the word 'job'.

 I sould get results from this query string then right (figure 1.0 is my 
 result - 0 records found)?
 solr/select?q=job


 figure 1.0
 response
 −
 lst name=responseHeader
 int name=status0/int
 int name=QTime0/int
 −
 lst name=params
 str name=qjob/str
 /lst
 /lst
 result name=response numFound=0 start=0/
 /response

 figure 1.1
 response
 −
 lst name=responseHeader
 int name=status0/int
 int name=QTime0/int
 /lst
 −
 lst name=initArgs
 −
 lst name=defaults
 str name=configdata-config.xml/str
 /lst
 /lst
 str name=statusidle/str
 str name=importResponse/
 −
 lst name=statusMessages
 str name=Total Requests made to DataSource1/str
 str name=Total Rows Fetched125/str
 str name=Total Documents Skipped0/str
 str name=Full Dump Started2008-11-10 22:33:55/str
 −
 str name=
 Indexing completed. Added/Updated: 125 documents. Deleted 0 documents.
 /str
 str name=Committed2008-11-10 22:34:00/str
 str name=Optimized2008-11-10 22:34:00/str
 str name=Time taken 0:0:5.79/str
 /lst
 −
 str name=WARNING
 This response format is experimental.  It is likely to change in the future.
 /str
 /response

 Kevin

 -Original Message-
 From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED]
 Sent: Monday, November 10, 2008 10:30 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Newbie Question - getting search results from dataimport request 
 handler

 XML is just an intermediete data format Solr internally has no XML
 data. When the data comes out XML is just another representation of
 the same data.

 Whether you put in data using XML or DB (SQL) it all goes into the
 same index . Query must be done on that index using the syntax
 http://localhost:8983/solr/select/?q=your-query-goes-here

 On Tue, Nov 11, 2008 at 9:55 AM, Kevin Penny [EMAIL PROTECTED] wrote:
 Ok - and what would that be? (query interface)

 I need the URL format that would work in this situation to return data from 
 my setup.

 I've gone through the tutorial and used execution strings like:
 http://localhost:8983/solr/select/?indent=onq=videosort=price+desc
 etc however I'm working with sql data and not xml data.

 Thanks

 -Original Message-
 From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED]
 Sent: Monday, November 10, 2008 10:18 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Newbie Question - getting search results from dataimport 
 request handler

 you cannot query the DIH. It can only do indexing
 after indexing you must do the indexing on the regular query interface

 On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED] wrote:
 My Question is: what is the format of a search that will return data?
 i.e. /solr/select?q=developerqt=dataimport (won't work) nor will 
 /solr/dataimport?q=developer (won't work)
 HTTP ERROR: 404
 NOT_FOUND
 RequestURI=/solr/dataimport

 I have created a 'dataimport' set that contains data from a sql db.

 I can view meta data from this url: /solr/dataimport
 response
 −
 lst name=responseHeader
 int name=status0/int
 int name=QTime0/int
 /lst
 −
 lst name=initArgs
 −
 lst name=defaults
 str name=configdata-config.xml/str
 /lst
 /lst
 str name=statusidle/str
 str name=importResponse/
 −
 lst name=statusMessages
 str name=Total Requests made to DataSource1/str
 str name=Total Rows Fetched10/str
 str name=Total Documents Skipped0/str
 str name=Full Dump Started2008-11-10 21:51:40/str
 str name=Time taken 0:0:4.594/str
 /lst
 −
 str name=WARNING
 This response format is experimental.  It is likely to change in the future.
 /str
 /response

 I can verify that the data is there by going through 
 /solr/admin/dataimport.jsp and doing 'verbose' true and debug now.
 It shows me the xml data set on the right as such:

 response
 −
 lst name=responseHeader
 int name=status0/int
 int name=QTime4594/int
 /lst
 −
 lst name=initArgs
 −
 lst name=defaults
 str name=configdata-config.xml/str
 /lst
 /lst
 str name=commandfull-import/str
 str name=modedebug/str
 −
 arr name=documents
 −
 arr
 −
 arr
 int87133/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87134/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87135/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87136/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87137/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87138/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87139/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87140/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87141/int
 /arr
 /arr
 −
 arr
 −
 arr
 int87142/int
 /arr
 /arr
 /arr
 −
 lst name=verbose-output
 −
 lst name=entity:item
 −
 lst name=document#1
 −
 str name=query
 SELECT  j.id  , j.title  ,  FROM

Re: Newbie Question - getting search results from dataimport request handler

[Shalin Shekhar Mangar]

Hi Kevin,

You need to modify the schema which came with Solr to suit your data. There
should be a schema.xml inside example/solr/conf directory. Once you do that,
re-import your data.

Take a look at http://wiki.apache.org/solr/SchemaXml

On Tue, Nov 11, 2008 at 10:59 AM, Kevin Penny [EMAIL PROTECTED] wrote:

 I can execute: /solr/select?q=id:87133

 So there is data there, however I have not defined any 'Fields' in my
 data-config and am hoping my column names are the 'fields', yet I'm not
 seeing any of them being returned in the 'doc' node below :


 response
 -
 lst name=responseHeader
 int name=status0/int
 int name=QTime0/int
 -
 lst name=params
 str name=qid:87133/str
 /lst
 /lst
 -
 result name=response numFound=1 start=0
 -
 doc
 str name=id87133/str
 int name=popularity0/int
 str name=sku87133/str
 date name=timestamp2008-11-11T05:25:29Z/date
 /doc
 /result
 /response

 Kevin

 -Original Message-
 From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED]
 Sent: Monday, November 10, 2008 11:23 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Newbie Question - getting search results from dataimport
 request handler

 search for *:* and see if the index indeed has the documents .
 Once you ensure the docs are there go through the lucene query syntax
 and check your query

 On Tue, Nov 11, 2008 at 10:07 AM, Kevin Penny [EMAIL PROTECTED] wrote:
  Ok so I executed a:
  solr/dataimport?command=full-import
  then I checked here:
  solr/dataimport
 
  I get a good xml message (figure 1.1) showing me that 125 records have
 been indexed (good) and I know one of them contains the word 'job'.
 
  I sould get results from this query string then right (figure 1.0 is my
 result - 0 records found)?
  solr/select?q=job
 
 
  figure 1.0
  response
  −
  lst name=responseHeader
  int name=status0/int
  int name=QTime0/int
  −
  lst name=params
  str name=qjob/str
  /lst
  /lst
  result name=response numFound=0 start=0/
  /response
 
  figure 1.1
  response
  −
  lst name=responseHeader
  int name=status0/int
  int name=QTime0/int
  /lst
  −
  lst name=initArgs
  −
  lst name=defaults
  str name=configdata-config.xml/str
  /lst
  /lst
  str name=statusidle/str
  str name=importResponse/
  −
  lst name=statusMessages
  str name=Total Requests made to DataSource1/str
  str name=Total Rows Fetched125/str
  str name=Total Documents Skipped0/str
  str name=Full Dump Started2008-11-10 22:33:55/str
  −
  str name=
  Indexing completed. Added/Updated: 125 documents. Deleted 0 documents.
  /str
  str name=Committed2008-11-10 22:34:00/str
  str name=Optimized2008-11-10 22:34:00/str
  str name=Time taken 0:0:5.79/str
  /lst
  −
  str name=WARNING
  This response format is experimental.  It is likely to change in the
 future.
  /str
  /response
 
  Kevin
 
  -Original Message-
  From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED]
  Sent: Monday, November 10, 2008 10:30 PM
  To: solr-user@lucene.apache.org
  Subject: Re: Newbie Question - getting search results from dataimport
 request handler
 
  XML is just an intermediete data format Solr internally has no XML
  data. When the data comes out XML is just another representation of
  the same data.
 
  Whether you put in data using XML or DB (SQL) it all goes into the
  same index . Query must be done on that index using the syntax
  http://localhost:8983/solr/select/?q=your-query-goes-here
 
  On Tue, Nov 11, 2008 at 9:55 AM, Kevin Penny [EMAIL PROTECTED]
 wrote:
  Ok - and what would that be? (query interface)
 
  I need the URL format that would work in this situation to return data
 from my setup.
 
  I've gone through the tutorial and used execution strings like:
  http://localhost:8983/solr/select/?indent=onq=videosort=price+desc
  etc however I'm working with sql data and not xml data.
 
  Thanks
 
  -Original Message-
  From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED]
  Sent: Monday, November 10, 2008 10:18 PM
  To: solr-user@lucene.apache.org
  Subject: Re: Newbie Question - getting search results from dataimport
 request handler
 
  you cannot query the DIH. It can only do indexing
  after indexing you must do the indexing on the regular query interface
 
  On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED]
 wrote:
  My Question is: what is the format of a search that will return data?
  i.e. /solr/select?q=developerqt=dataimport (won't work) nor will
 /solr/dataimport?q=developer (won't work)
  HTTP ERROR: 404
  NOT_FOUND
  RequestURI=/solr/dataimport
 
  I have created a 'dataimport' set that contains data from a sql db.
 
  I can view meta data from this url: /solr/dataimport
  response
  −
  lst name=responseHeader
  int name=status0/int
  int name=QTime0/int
  /lst
  −
  lst name=initArgs
  −
  lst name=defaults
  str name=configdata-config.xml/str
  /lst
  /lst
  str name=statusidle/str
  str name=importResponse/
  −
  lst name=statusMessages
  str name=Total Requests made to DataSource1/str
  str name=Total

RE: Newbie Question - getting search results from dataimport request handler

[Kevin Penny]

Excellent!

Thanks a bunch - that did the trick - all defined and my terms are being 
returned nicely - schema.xml was the ticket - not sure how I missed that in the 
docs.

Kevin

-Original Message-
From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED]
Sent: Monday, November 10, 2008 11:35 PM
To: solr-user@lucene.apache.org
Subject: Re: Newbie Question - getting search results from dataimport request 
handler

Hi Kevin,

You need to modify the schema which came with Solr to suit your data. There
should be a schema.xml inside example/solr/conf directory. Once you do that,
re-import your data.

Take a look at http://wiki.apache.org/solr/SchemaXml

On Tue, Nov 11, 2008 at 10:59 AM, Kevin Penny [EMAIL PROTECTED] wrote:

 I can execute: /solr/select?q=id:87133

 So there is data there, however I have not defined any 'Fields' in my
 data-config and am hoping my column names are the 'fields', yet I'm not
 seeing any of them being returned in the 'doc' node below :


 response
 -
 lst name=responseHeader
 int name=status0/int
 int name=QTime0/int
 -
 lst name=params
 str name=qid:87133/str
 /lst
 /lst
 -
 result name=response numFound=1 start=0
 -
 doc
 str name=id87133/str
 int name=popularity0/int
 str name=sku87133/str
 date name=timestamp2008-11-11T05:25:29Z/date
 /doc
 /result
 /response

 Kevin

 -Original Message-
 From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED]
 Sent: Monday, November 10, 2008 11:23 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Newbie Question - getting search results from dataimport
 request handler

 search for *:* and see if the index indeed has the documents .
 Once you ensure the docs are there go through the lucene query syntax
 and check your query

 On Tue, Nov 11, 2008 at 10:07 AM, Kevin Penny [EMAIL PROTECTED] wrote:
  Ok so I executed a:
  solr/dataimport?command=full-import
  then I checked here:
  solr/dataimport
 
  I get a good xml message (figure 1.1) showing me that 125 records have
 been indexed (good) and I know one of them contains the word 'job'.
 
  I sould get results from this query string then right (figure 1.0 is my
 result - 0 records found)?
  solr/select?q=job
 
 
  figure 1.0
  response
  −
  lst name=responseHeader
  int name=status0/int
  int name=QTime0/int
  −
  lst name=params
  str name=qjob/str
  /lst
  /lst
  result name=response numFound=0 start=0/
  /response
 
  figure 1.1
  response
  −
  lst name=responseHeader
  int name=status0/int
  int name=QTime0/int
  /lst
  −
  lst name=initArgs
  −
  lst name=defaults
  str name=configdata-config.xml/str
  /lst
  /lst
  str name=statusidle/str
  str name=importResponse/
  −
  lst name=statusMessages
  str name=Total Requests made to DataSource1/str
  str name=Total Rows Fetched125/str
  str name=Total Documents Skipped0/str
  str name=Full Dump Started2008-11-10 22:33:55/str
  −
  str name=
  Indexing completed. Added/Updated: 125 documents. Deleted 0 documents.
  /str
  str name=Committed2008-11-10 22:34:00/str
  str name=Optimized2008-11-10 22:34:00/str
  str name=Time taken 0:0:5.79/str
  /lst
  −
  str name=WARNING
  This response format is experimental.  It is likely to change in the
 future.
  /str
  /response
 
  Kevin
 
  -Original Message-
  From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED]
  Sent: Monday, November 10, 2008 10:30 PM
  To: solr-user@lucene.apache.org
  Subject: Re: Newbie Question - getting search results from dataimport
 request handler
 
  XML is just an intermediete data format Solr internally has no XML
  data. When the data comes out XML is just another representation of
  the same data.
 
  Whether you put in data using XML or DB (SQL) it all goes into the
  same index . Query must be done on that index using the syntax
  http://localhost:8983/solr/select/?q=your-query-goes-here
 
  On Tue, Nov 11, 2008 at 9:55 AM, Kevin Penny [EMAIL PROTECTED]
 wrote:
  Ok - and what would that be? (query interface)
 
  I need the URL format that would work in this situation to return data
 from my setup.
 
  I've gone through the tutorial and used execution strings like:
  http://localhost:8983/solr/select/?indent=onq=videosort=price+desc
  etc however I'm working with sql data and not xml data.
 
  Thanks
 
  -Original Message-
  From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED]
  Sent: Monday, November 10, 2008 10:18 PM
  To: solr-user@lucene.apache.org
  Subject: Re: Newbie Question - getting search results from dataimport
 request handler
 
  you cannot query the DIH. It can only do indexing
  after indexing you must do the indexing on the regular query interface
 
  On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED]
 wrote:
  My Question is: what is the format of a search that will return data?
  i.e. /solr/select?q=developerqt=dataimport (won't work) nor will
 /solr/dataimport?q=developer (won't work)
  HTTP ERROR: 404
  NOT_FOUND
  RequestURI=/solr/dataimport
 
  I have created a 'dataimport' set