Re: Mongo DB Users

[Joan]

Remove please

2014-09-16 6:59 GMT+02:00 Patti Kelroe-Cooke :

> Remove
>
> Kind regards
> Patti
>
> On Mon, Sep 15, 2014 at 5:35 PM, Aaron Susan 
> wrote:
>
> > Hi,
> >
> > I am here to inform you that we are having a contact list of *Mongo DB
> > Users *would you be interested in it?
> >
> > Data Field’s Consist Of: Name, Job Title, Verified Phone Number, Verified
> > Email Address, Company Name & Address Employee Size, Revenue size, SIC
> > Code, Industry Type etc.,
> >
> > We also provide other technology users as well depends on your
> requirement.
> >
> > For Example:
> >
> >
> > *Red Hat *
> >
> > *Terra data *
> >
> > *Net-app *
> >
> > *NuoDB*
> >
> > *MongoHQ ** and many more*
> >
> >
> > We also provide IT Decision Makers, Sales and Marketing Decision Makers,
> > C-level Titles and other titles as per your requirement.
> >
> > Please review and let me know your interest if you are looking for above
> > mentioned users list or other contacts list for your campaigns.
> >
> > Waiting for a positive response!
> >
> > Thanks
> >
> > *Aaron Susan*
> > Data Specialist
> >
> > If you are not the right person, feel free to forward this email to the
> > right person in your organization. To opt out response Remove
> >
>

How to write core's name in log

[Joan]

Hi,

I have multiple core in Solr and I want to write core name in log through to
lo4j.

I've found in SolrException a method called log(Logger log, Throwable e) but
when It try to build a Exception it haven't core's name.

The Exception is built in toStr() method in SolrException class, so I want
to write core's name in the message of Exception.

I'm thinking to add MDC variable, this will be name of core. Finally I'll
use it in log4j configuration like this in ConversionPattern %X{core}

The idea is that when Solr received a request I'll add this new variable
"name of core".

But I don't know if it's a good idea or not.

or Do you already exists any solution for add name of core in log?

Thanks

Joan

Date faceting per last hour, three days and last week

[Joan]

Hi,

I'm trying date faceting per last 24 hours, three days and last week, but I
don't know how to do it.

I have a DateField and I want to set different ranges, it is posible?

I understand the example from solr
wiki<http://wiki.apache.org/solr/SimpleFacetParameters#Date_Faceting:_per_day_for_the_past_5_days>but
I want to do more "gaps" with the same field_date.

How I do this?

Thanks,

Joan

Solr + Hadoop

[Joan]

Hi,

I'm trying build solr index with MapReduce (Hadoop) and I'm using
https://issues.apache.org/jira/browse/SOLR-1301 but I've a problem with
hadoop version and this patch.

When I compile this patch, I use 0.21.0 hadoop version, I don't have any
problem but when I'm trying to run my job in Hadoop (0.0.21) I get some
error like this:

*Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
interface org.apache.hadoop.mapreduce.JobContext, but class was expected*
 at
org.apache.solr.hadoop.SolrOutputFormat.checkOutputSpecs(SolrOutputFormat.java:147)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:373)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:334)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:960)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:976)



I try to override the next method:

  *public void checkOutputSpecs(JobContext job) throws IOException {
super.checkOutputSpecs(job);
if (job.getConfiguration().get(SETUP_OK) == null) {
  throw new IOException("Solr home cache not set up!");
}
  }*

by

 * public void checkOutputSpecs(Job job) throws IOException {
super.checkOutputSpecs(job);
if (job.getConfiguration().get(SETUP_OK) == null) {
  throw new IOException("Solr home cache not set up!");
}
  }*

but I continue receive some error:
*
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.solr.hadoop.SolrOutputFormat
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1128)
at
org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:203)
at org.apache.hadoop.mapred.Task.initialize(Task.java:487)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:311)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)*



Please, Is someone using this patch with 0.21.0 Version Hadoop?. Can someone
help me?

Thanks,

Joan

Re: Creating Solr index from map/reduce

[Joan]

Thanks Alexander

2011/1/3 Alexander Kanarsky 

> Joan,
>
> current version of the patch assumes the location and names for the
> schema and solrconfig files ($SOLR_HOME/conf), it is hardcoded (see
> the SolrRecordWriter's constructor). Multi-core configuration with
> separate configuration locations via solr.xml is not supported as for
> now.  As a workaround, you could link or copy the schema and
> solrconfig files to follow the hardcoded assumption.
>
> Thanks,
> -Alexander
>
> On Wed, Dec 29, 2010 at 2:50 AM, Joan  wrote:
> > If I rename my custom schema file (schema-xx.xml), whitch is located in
> > SOLR_HOME/schema/, and then I copy it to "conf" folder and finally I try
> to
> > run CSVIndexer, it shows me an other error:
> >
> > Caused by: java.lang.RuntimeException: Can't find resource
> 'solrconfig.xml'
> > in classpath or
> >
> '/tmp/hadoop-root/mapred/local/taskTracker/archive/localhost/tmp/b7611d6d-9cc7-4237-a240-96ecaab9f21a.solr.zip/conf/'
> >
> > I dont't understand because I've a solr configuration file (solr.xml)
> where
> > I define all core:
> >
> >   >instanceDir="solr-data/index"
> >config="solr/conf/solrconfig_xx.xml"
> >schema="solr/schema/schema_xx.xml"
> >properties="solr/conf/solrcore.properties"/ >
> >
> > But I think that when I run CSVIndexer, it doesn't know that solr.xml
> exist,
> > and it try to looking for schema.xml and solrconfig.xml by default in
> > default folder (conf)
> >
> >
> >
> > 2010/12/29 Joan 
> >
> >> Hi,
> >>
> >> I'm trying generate Solr index from hadoop (map/reduce) so I'm using
> this
> >> patch SOLR-301 <https://issues.apache.org/jira/browse/SOLR-1301>,
> however
> >> I don't get it.
> >>
> >> When I try to run CSVIndexer with some arguments: 
> >> -solr  
> >>
> >> I'm runnig CSVIndexer:
> >>
> >> /bin/hadoop jar my.jar CSVIndexer  -solr
> >> / 
> >>
> >> Before that I run CSVIndexer, I've put csv file into HDFS.
> >>
> >> My Solr home hasn't default files configurations, but which is divided
> >> into multiple folders
> >>
> >> /conf
> >> /schema
> >>
> >> I have custom solr file configurations so CSVIndexer can't find
> schema.xml,
> >> obviously It won't be able to find it because this file doesn't exist,
> in my
> >> case, this file is named "schema-xx.xml" and CSVIndexer is looking for
> it
> >> inside "conf" folder and It don't know that schema folder exist. And I
> have
> >> solr configuration file (solr.xml) where I configure multiple cores.
> >>
> >> I tried to modify solr's paths but It still not working .
> >>
> >> I understand that CSVIndexer copy Solr Home specified into HDFS
> >> (/tmp/hadoop-user/mapred/local/taskTracker/archive/...) and when It try
> to
> >> find "schema.xml" it doesn't exit:
> >>
> >> 10/12/29 10:18:11 INFO mapred.JobClient: Task Id :
> >> attempt_201012291016_0002_r_00_1, Status : FAILED
> >> java.lang.IllegalStateException: Failed to initialize record writer for
> >> my.jar, attempt_201012291016_0002_r_00_1
> >> at
> >>
> org.apache.solr.hadoop.SolrRecordWriter.(SolrRecordWriter.java:253)
> >> at
> >>
> org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:152)
> >> at
> >> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> >> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> >> Caused by: java.io.FileNotFoundException: Source
> >>
> '/tmp/hadoop-guest/mapred/local/taskTracker/archive/localhost/tmp/e8be5bb1-e910-47a1-b5a7-1352dfec2b1f.solr.zip/conf/schema.xml'
> >> does not exist
> >> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:636)
> >> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:606)
> >> at
> >>
> org.apache.solr.hadoop.SolrRecordWriter.(SolrRecordWriter.java:222)
> >> ... 4 more
> >
>

Re: Creating Solr index from map/reduce

[Joan]

If I rename my custom schema file (schema-xx.xml), whitch is located in
SOLR_HOME/schema/, and then I copy it to "conf" folder and finally I try to
run CSVIndexer, it shows me an other error:

Caused by: java.lang.RuntimeException: Can't find resource 'solrconfig.xml'
in classpath or
'/tmp/hadoop-root/mapred/local/taskTracker/archive/localhost/tmp/b7611d6d-9cc7-4237-a240-96ecaab9f21a.solr.zip/conf/'

I dont't understand because I've a solr configuration file (solr.xml) where
I define all core:

  

But I think that when I run CSVIndexer, it doesn't know that solr.xml exist,
and it try to looking for schema.xml and solrconfig.xml by default in
default folder (conf)



2010/12/29 Joan 

> Hi,
>
> I'm trying generate Solr index from hadoop (map/reduce) so I'm using this
> patch SOLR-301 <https://issues.apache.org/jira/browse/SOLR-1301>, however
> I don't get it.
>
> When I try to run CSVIndexer with some arguments: 
> -solr  
>
> I'm runnig CSVIndexer:
>
> /bin/hadoop jar my.jar CSVIndexer  -solr
> / 
>
> Before that I run CSVIndexer, I've put csv file into HDFS.
>
> My Solr home hasn't default files configurations, but which is divided
> into multiple folders
>
> /conf
> /schema
>
> I have custom solr file configurations so CSVIndexer can't find schema.xml,
> obviously It won't be able to find it because this file doesn't exist, in my
> case, this file is named "schema-xx.xml" and CSVIndexer is looking for it
> inside "conf" folder and It don't know that schema folder exist. And I have
> solr configuration file (solr.xml) where I configure multiple cores.
>
> I tried to modify solr's paths but It still not working .
>
> I understand that CSVIndexer copy Solr Home specified into HDFS
> (/tmp/hadoop-user/mapred/local/taskTracker/archive/...) and when It try to
> find "schema.xml" it doesn't exit:
>
> 10/12/29 10:18:11 INFO mapred.JobClient: Task Id :
> attempt_201012291016_0002_r_00_1, Status : FAILED
> java.lang.IllegalStateException: Failed to initialize record writer for
> my.jar, attempt_201012291016_0002_r_00_1
> at
> org.apache.solr.hadoop.SolrRecordWriter.(SolrRecordWriter.java:253)
> at
> org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:152)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.io.FileNotFoundException: Source
> '/tmp/hadoop-guest/mapred/local/taskTracker/archive/localhost/tmp/e8be5bb1-e910-47a1-b5a7-1352dfec2b1f.solr.zip/conf/schema.xml'
> does not exist
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:636)
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:606)
> at
> org.apache.solr.hadoop.SolrRecordWriter.(SolrRecordWriter.java:222)
> ... 4 more

Creating Solr index from map/reduce

[Joan]

Hi,

I'm trying generate Solr index from hadoop (map/reduce) so I'm using this
patch SOLR-301 , however I
don't get it.

When I try to run CSVIndexer with some arguments: 
-solr  

I'm runnig CSVIndexer:

/bin/hadoop jar my.jar CSVIndexer  -solr
/ 

Before that I run CSVIndexer, I've put csv file into HDFS.

My Solr home hasn't default files configurations, but which is divided  into
multiple folders

/conf
/schema

I have custom solr file configurations so CSVIndexer can't find schema.xml,
obviously It won't be able to find it because this file doesn't exist, in my
case, this file is named "schema-xx.xml" and CSVIndexer is looking for it
inside "conf" folder and It don't know that schema folder exist. And I have
solr configuration file (solr.xml) where I configure multiple cores.

I tried to modify solr's paths but It still not working .

I understand that CSVIndexer copy Solr Home specified into HDFS
(/tmp/hadoop-user/mapred/local/taskTracker/archive/...) and when It try to
find "schema.xml" it doesn't exit:

10/12/29 10:18:11 INFO mapred.JobClient: Task Id :
attempt_201012291016_0002_r_00_1, Status : FAILED
java.lang.IllegalStateException: Failed to initialize record writer for
my.jar, attempt_201012291016_0002_r_00_1
at
org.apache.solr.hadoop.SolrRecordWriter.(SolrRecordWriter.java:253)
at
org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:152)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.io.FileNotFoundException: Source
'/tmp/hadoop-guest/mapred/local/taskTracker/archive/localhost/tmp/e8be5bb1-e910-47a1-b5a7-1352dfec2b1f.solr.zip/conf/schema.xml'
does not exist
at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:636)
at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:606)
at
org.apache.solr.hadoop.SolrRecordWriter.(SolrRecordWriter.java:222)
... 4 more

Re: storing results

[Joan Codina]

Thanks for your answers,

Yes, its true that with boolen queries things are much easier...
+(query1) +(query2) should do an and
or 
(query1) (query2) should do an or

and this does not need a special ability to parse the queries
I like the dismax approach, I think is interesting but then to
merge queries is a bit difficult, and I think for this option is better
to try the filter, but then there is no OR option, only , AND. And bq
(from what I understand from de docs) seems that also performs and AND

It's a pity I have to do it for a demo in a few days (and nights) and 
I don't think the resulting code will be nice enough to be shown.

Joan

Chris Hostetter wrote:

  : To do so I need to  store the results as a filter, with a given name, so
: the user can use it later on. But I need to store this in disk, as I can
: not trust on the cache or the web session.
: The user should  then indicate that the query that is doing now has a
: filter (a previous query) and this filter should be added to the query
: (this is allowed in solr,  i think) but as filter_ID, to be loaded to
: solve the query.

if you really want to be ableto refer to these previous searches using
some sort of identifier, and have them persist for an indefinite amount of
time, it's really out of Solr's hands -- if someone were to try and add a
fewture like this to Solr, how would it know which queries to remember and
generate names for? how long would it store each name? ... these are the
kidns of questsions that your app can understand more easily ... you could
concievable use Solr to store the name=>querystring mappings in little
custom solr docs if you wanted, but you have to decide when to create
those mappings and when to expire them.

in general though, all you really need to remember is hte query string,
remembering all of the results really isn't neccessary.  The next time
your user wants to "refine" his search -- wether it's 10 seconds latter or
10 days later -- just take the old query string and combine it with the
new query string.  how you combine it depends on how you want the scoring
to work, use an "fq" param if the old query shouldn't affect the score,
just define the super set of docs, use a BooleanQuery if you want all the
clauses from both searches to impact the score.

it's important to understand that trying to keep track of the actual
results would be very, very, very bad  (especially if you want remember
them for a really long time) because when the user comes back, the index
may have chagned, docs may have shifted ids, or been edited so they no
longer match the criteria, or have been deleted completely.



-Hoss

  


-- 

signUPF
Joan Codina
Filbà
Departament de Tecnologia
Universitat Pompeu Fabra
___

Abans d'imprimir aquest e-mail, pensa si realment
és necessari, i en
cas de que ho sigui, pensa que si ho fas a doble cara estalvies un 25%
del paper, els arbres t'ho agrairan.
___

La informació
d'aquest missatge electrònic és confidencial,
personal i intransferible
i només està dirigida a la/les
adreça/ces indicades a dalt. Si vostè
llegeix aquest missatge per equivocació, l'informem que
queda prohibida
la seva divulgació, ús o distribució,
completa o en part, i li preguem esborri el missatge original juntament
amb els seus fitxers annexos
sense llegir-lo ni gravar-lo. 
Gràcies. 

Re: storing results

[Joan Codina]

I would like to store the results of a query someway,. Then after the 
user analyzes some of  the documents (and he/she can take some days to 
do it),  the user can try to make a query refinement over the previous 
result, getting a subset of it.
To do so I need to  store the results as a filter, with a given name, so 
the user can use it later on. But I need to store this in disk, as I can 
not trust on the cache or the web session.
The user should  then indicate that the query that is doing now has a 
filter (a previous query) and this filter should be added to the query 
(this is allowed in solr,  i think) but as filter_ID, to be loaded to 
solve the query.


I hope now its clearer what I'm trying to do

Joan


Mike Klaas wrote:

On 3/27/07, Joan Codina <[EMAIL PROTECTED]> wrote:

 I'm using solr, to build a search engine, and it works great!!
 Thanks for the job,guys!


Glad it is working for you.


 but...
 I need to build a searcher that must allow to perform a "search 
process"

for a collection of documents. And this search process may last several
days,  and be composed of several queries that I must be able to 
store and

reuse.
 My doubts are:
 How can I store the search results?
 How can I add previously stored search results to a new search?


I'm not sure what you mean by "search process", nor really what you
mean by the functionality requirements you list.  You can certainly
store search results and combine them together however you wish, but
it seems that that is more on the post-query data-analysis side of
things rather than having anything to do with Solr.

Could you clarify what you're looking for Solr to do for you?

-Mike

storing results

[Joan Codina]

I'm using solr, to build a search
engine, and it works great!! 
Thanks for the job,guys!

but...
I need to build a searcher that must allow to perform a "search
process" for a collection of documents. And this search process
may last several days,  and be composed of several queries that I must
be able to store and reuse.
My doubts are: 
How can I store the search results? 
How can I add previously stored search results to a new search?

Thanks!!
-- 

signUPF
Joan Codina
Filbà
Departament de Tecnologia
Universitat Pompeu Fabra