Determining replication status
I am trying to copy contents of one core to other core by using fetchIndex command (/solr/CORE_NAME/replication?command=fetchindex&masterUrl=MASTER_URL). How can I determine if the copy is actually complete? I am using replication details command (/solr/CORE_NAME/replication?command=details) to check. I use isReplicating field & if its false, I consider the operation to be complete. But this approach is not giving me accurate results some time. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: How do I create a schema file for FIX data in Solr
Raymond Will you be streaming the FIX data, perhaps with aggregation? Just a thought, I have no experience with FIX. Streaming opens up lots of questions. Cheers -- Rick On March 31, 2018 2:33:25 PM EDT, Walter Underwood wrote: >Looks like Financial Information Exchange data, but, as Shawn says, the >real problem is what you want to do with it. > >* What fields will be searched? Those are indexed. >* What fields will be returned in the result? Those are stored. >* What is the data type for each field? > >I often store the data for most of the fields because it makes >debugging search problems so much easier. > >wunder >Walter Underwood >wun...@wunderwood.org >http://observer.wunderwood.org/ (my blog) > >> On Mar 31, 2018, at 11:29 AM, Shawn Heisey >wrote: >> >> On 3/31/2018 12:21 PM, Raymond Xie wrote: >>> I just started using Solr to create a Searching function on our >existing >>> data. >>> >>> The existing data is in FIX format sample as below: >> >>> all the red tags (I didn't mark all of them) are fields with >definition >>> from FIX standard, I need to create index on all the tags, how do I >start? >> >> I do not know what FIX means, and there are no colors in your email. >> >> Can you elaborate? >> >> Fine-tuning the schema can be one of the most time-consuming parts of >setting up a Solr installation, and there are usually no easy quick >answers. Exactly what to do will depend not only on the data that >you're indexing, but also what you want to do with it. >> >> Thanks, >> Shawn >> -- Sorry for being brief. Alternate email is rickleir at yahoo dot com
Re: How do I create a schema file for FIX data in Solr
Looks like Financial Information Exchange data, but, as Shawn says, the real problem is what you want to do with it. * What fields will be searched? Those are indexed. * What fields will be returned in the result? Those are stored. * What is the data type for each field? I often store the data for most of the fields because it makes debugging search problems so much easier. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Mar 31, 2018, at 11:29 AM, Shawn Heisey wrote: > > On 3/31/2018 12:21 PM, Raymond Xie wrote: >> I just started using Solr to create a Searching function on our existing >> data. >> >> The existing data is in FIX format sample as below: > >> all the red tags (I didn't mark all of them) are fields with definition >> from FIX standard, I need to create index on all the tags, how do I start? > > I do not know what FIX means, and there are no colors in your email. > > Can you elaborate? > > Fine-tuning the schema can be one of the most time-consuming parts of setting > up a Solr installation, and there are usually no easy quick answers. Exactly > what to do will depend not only on the data that you're indexing, but also > what you want to do with it. > > Thanks, > Shawn >
Re: How do I create a schema file for FIX data in Solr
On 3/31/2018 12:21 PM, Raymond Xie wrote: I just started using Solr to create a Searching function on our existing data. The existing data is in FIX format sample as below: all the red tags (I didn't mark all of them) are fields with definition from FIX standard, I need to create index on all the tags, how do I start? I do not know what FIX means, and there are no colors in your email. Can you elaborate? Fine-tuning the schema can be one of the most time-consuming parts of setting up a Solr installation, and there are usually no easy quick answers. Exactly what to do will depend not only on the data that you're indexing, but also what you want to do with it. Thanks, Shawn
How do I create a schema file for FIX data in Solr
Hello,
I just started using Solr to create a Searching function on our existing
data.
The existing data is in FIX format sample as below:
8=FIX.4.4 9=653 35=RIO 1=TEST 11=3379122 38=1 44=2.0 39=A 40=2
49=VIPER 50=JPNIK01 54=1 55=JNI253D8.OS 56=XSVC 59=0 75=20180350
100=XOSE 10039=viperooe 10241=viperooe 150=A 372=D
122=20180320-08:08:35.038 10066=20180320-08:08:35.038
10436=20180320-08:08:35.038 202=25375.0 52=20180320-08:08:35.088
60=20180320-08:08:35.088 10071=20180320-08:08:35.088
11210=3379122 37=3379122 10184=3379122 201=1
29=4 10438=RIO.4.5 10005=178 10515=178 10518=178 581=13 660=102 1133=G
528=P 10104=Y 10202=APMKTMAKING 10208=APAC.VIPER.OOE 10217=Y 10292=115
11032=-1 382=0 10537=XOSE 15=JPY 167=OPT 48=179492540 455=179492540
22=101 456=101 151=1.0 421=JPN 10=200
all the red tags (I didn't mark all of them) are fields with definition
from FIX standard, I need to create index on all the tags, how do I start?
Thank you very much.
**
*Sincerely yours,*
*Raymond*
On Sat, Mar 31, 2018 at 12:24 AM, Randy Fradin
wrote:
> I have a SolrCloud cluster (version 6.5.1) with around 3300 cores per
> instance. I've been investigating what is driving heap utilization since it
> is higher than I expected. I took a heap dump and found the largest driver
> of heap utilization is the array of VersionBucket objects in the
> org.apache.solr.update.VersionInfo class. The array is size 65536 and
> there
> is one per SolrCore instance. Each instance of the array is 1.8MB so the
> aggregate size is 6GB in heap.
>
> I understand from reading the discussion in SOLR-6820 that 65536 is the
> recommended default for this setting now because it results in higher
> document write rates than the old default of 256. I would like to reduce my
> heap utilization and I'm OK with somewhat slower document writing
> throughput. My question is, it is safe to reduce the value
> of numVersionBuckets on all of my existing cores without reindexing my
> data?
>
> My solrconfig.xml contains this for all of my collections:
>
>
>
> ${solr.ulog.dir:}
> ${solr.ulog.numVersionBuckets:
> 65536}
>
>
>
> Assuming it is safe to change, can I just add a vm arg to the Solr process
> like "-Dsolr.ulog.numVersionBuckets=256" to override the value for all
> cores at once? Or do I have to change and re-upload the solrconfig.xml
> files and reload the cores?
>
> Thanks
>
Re: Bad request from solr by just changing the order in the filter query
On 3/31/2018 10:22 AM, Elier Delgado wrote:
Hi, I'm working with facets in solr 7.2.1. Basically I'm following this
tutorial:
http://yonik.com/multi-select-faceting/
The following solr request works just fine:
&json.facet={domains:{type:terms,field:domains,domain:{excludeTags:DOMAIN}},specialties:{type:terms,field:specialties,domain:{excludeTags:SPECIALTY}}}
&fq={!tag=SPECIALTY}specialties:(1043 1023) AND {!tag=DOMAIN}domains:100
If I just change the order in the fq parameter, solr response with 400 Bad
Request.
&json.facet={domains:{type:terms,field:domains,domain:{excludeTags:DOMAIN}},specialties:{type:terms,field:specialties,domain:{excludeTags:SPECIALTY}}}
&fq={!tag=DOMAIN}domains:100 AND {!tag=SPECIALTY}specialties:(1043 1023)
I would like to know if I'm doing something wrong or it if could be a
problem in solr.
You've got {!tag} placed twice in one fq. I don't think you can do that
-- localparams (which is what that syntax is using) generally must be at
the very beginning of query text, and don't work when they're placed in
the middle.
Try this:
&fq={!tag=DOMAIN}domains:100&fq={!tag=SPECIALTY}specialties:(1043 1023)
The first request probably should have also thrown an error, and the
fact that it didn't MIGHT be a problem.
What is the full error, which should be many lines long? If you don't
see something that's lots of lines, look in solr.log.
Thanks,
Shawn
Bad request from solr by just changing the order in the filter query
Hi, I'm working with facets in solr 7.2.1. Basically I'm following this
tutorial:
http://yonik.com/multi-select-faceting/
The following solr request works just fine:
&json.facet={domains:{type:terms,field:domains,domain:{excludeTags:DOMAIN}},specialties:{type:terms,field:specialties,domain:{excludeTags:SPECIALTY}}}
&fq={!tag=SPECIALTY}specialties:(1043 1023) AND {!tag=DOMAIN}domains:100
If I just change the order in the fq parameter, solr response with 400 Bad
Request.
&json.facet={domains:{type:terms,field:domains,domain:{excludeTags:DOMAIN}},specialties:{type:terms,field:specialties,domain:{excludeTags:SPECIALTY}}}
&fq={!tag=DOMAIN}domains:100 AND {!tag=SPECIALTY}specialties:(1043 1023)
I would like to know if I'm doing something wrong or it if could be a
problem in solr.
Thanks!
Virus-free.
www.avg.com
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
