hanks!
Tim
On 5 February 2014 13:04, Tim Vaillancourt wrote:
> Hey guys,
>
> I am troubleshooting an issue on a 4.3.1 SolrCloud: 1 collection and 2
> shards over 4 Solr instances, (which results in 1 core per Solr instance).
>
> After some time in Production without issues,
Hey guys,
I am troubleshooting an issue on a 4.3.1 SolrCloud: 1 collection and 2
shards over 4 Solr instances, (which results in 1 core per Solr instance).
After some time in Production without issues, we are seeing errors related
to the IndexWriter all over our logs and an infinite loop of faili
I'm pretty interested in taking a stab at a Perl CPAN for SolrCloud that
is Zookeeper-aware; it's the least I can do for Solr as a non-Java
developer. :)
A quick question though: how would I write the shard logic to behave
similar to Java's Zookeeper-aware client? I'm able to get the hash/hex
This is a neat idea, but could be too close to lucene/etc.
You could jump up one level in the stack and use Redis/memcache as a
distributed HTTP cache in conjunction with Solr's HTTP caching and a proxy.
I tried doing this myself with Nginx, but I forgot what issue I hit - I
think "misses" needed
Hey guys,
I've been having an issue with 1 of my 4 replicas having an inconsistent
replica, and have been trying to fix it. At the core of this issue, I've
noticed /clusterstate.json doesn't seem to be receiving updates when cores
get unhealthy, or even added/removed.
Today I decided I would remo
I spoke too soon, my plan for fixing this didn't quite work.
I've moved this issue into a new thread/topic: "No /clusterstate.json
updates on Solrcloud 4.3.1 Cores API UNLOAD/CREATE".
Thanks all for the help on this one!
Tim
On 5 December 2013 11:37, Tim Vaillancourt
e/destroy of the
bad replica.
Thanks all!
Tim
On 4 December 2013 16:50, Mark Miller wrote:
> Keep in mind, there have been a *lot* of bug fixes since 4.3.1.
>
> - Mark
>
> On Dec 4, 2013, at 7:07 PM, Tim Vaillancourt wrote:
>
> > Hey all,
> >
> > Now that
st chain without recovering/re-replicating from leader.
I imagine my Zookeeper ensemble is having some problems unrelated to
Solr that is the real root cause.
Thanks!
Tim
On 04/12/13 03:00 PM, Tim Vaillancourt wrote:
Chris, this is extremely helpful and it's silly I didn't think of th
Chris, this is extremely helpful and it's silly I didn't think of this
sooner! Thanks a lot, this makes the situation make much more sense.
I will gather some proper data with your suggestion and get back to the
thread shortly.
Thanks!!
Tim
On 04/12/13 02:57 PM, Chris Hostetter wrote:
:
:
he SolrCloud down it is
remaining "state: active" in my /clusterstate.json - something is really
wrong with this cloud! Would a Zookeeper issue explain my varied results
when querying a core directly?
Thanks again!
Tim
On 04/12/13 02:17 PM, Tim Vaillancourt wrote:
Hey guys,
I'm lo
y /clusterstate.json - something is really
wrong with this cloud! Would a Zookeeper issue explain my varied results
when querying a core directly?
Thanks again!
Tim
On 04/12/13 02:17 PM, Tim Vaillancourt wrote:
Hey guys,
I'm looking into a strange issue on an unhealthy 4.3.1 SolrClo
Hey guys,
I'm looking into a strange issue on an unhealthy 4.3.1 SolrCloud with
3-node external Zookeeper and 1 collection (2 shards, 2 replicas).
Currently we are noticing inconsistent results from the SolrCloud when
performing the same simple /select query many times to our collection.
Alm
/issues.apache.org/jira/browse/SOLR-4792
P.S: There are no timelines for 5.0 for now, but it's the future
nevertheless.
On Fri, Oct 25, 2013 at 3:39 AM, Tim Vaillancourt
wrote:
I agree with Jonathan (and Shawn on the Jetty explanation), I think the
docs should make this a bit more clear - I no
t; P.S: There are no timelines for 5.0 for now, but it's the future
> nevertheless.
>
>
>
> On Fri, Oct 25, 2013 at 3:39 AM, Tim Vaillancourt >wrote:
>
> > I agree with Jonathan (and Shawn on the Jetty explanation), I think the
> > docs should make this a bit more
I agree with Jonathan (and Shawn on the Jetty explanation), I think the
docs should make this a bit more clear - I notice many people choosing
Tomcat and then learning these details after, possibly regretting it.
I'd be glad to modify the docs but I want to be careful how it is worded.
Is it fair
Awesome, this make a lot of sense now. Thanks a lot guys.
Currently the only mention of this setting in the docs is under
filterQuery on the "SolrCaching" page as:
" Solr3.4 Adding the
localParam flag of {!cache=false} to a query will prevent
the filterCach
, my guess is in the
queryResultCache. At first the query takes 500ms+, then all subsequent
requests take 0-1ms. I'll confirm this queryResultCache assumption today.
Cheers,
Tim
On 16/10/13 06:33 PM, Yonik Seeley wrote:
On Wed, Oct 16, 2013 at 6:18 PM, Tim Vaillancourt wrote:
I am debugging s
Not important, but I'm also curious why you would want SSL on Solr (adds
overhead, complexity, harder-to-troubleshoot, etc)?
To avoid the overhead, could you put Solr on a separate VLAN (with ACLs to
client servers)?
Cheers,
Tim
On 12 October 2013 17:30, Shawn Heisey wrote:
> On 10/11/2013 9
way to do this (without disabling my caches in solrconfig.xml),
or is this feature request?
Thanks!
Tim Vaillancourt
Thanks Koji!
Cheers,
Tim
On 14/10/13 03:56 PM, Koji Sekiguchi wrote:
Hi Tim,
(13/10/15 5:22), Tim Vaillancourt wrote:
Hey guys,
Sorry for such a simple question, but I am curious as to the
differences in caching between a
"combined" filter query, and many separate filter quer
sed 2
separate filter queries than 1 of 2 is still cached)?
Cheers,
Tim Vaillancourt
what different
> question that they might have information about. It's not about whether
> you're
> doing anything particularly wrong with the question. It's about making
> it easy for
> people to help.
>
> See http://people.apache.org/~hossman/#threadhijack
e to get noticed.
> Dmitry
>
>
> On Mon, Oct 7, 2013 at 9:44 PM, Tim Vaillancourt >wrote:
>
> > Is there a way to make autoCommit only commit if there are pending
> changes,
> > ie: if there are 0 adds pending commit, don't autoCommit (open-a-searcher
> > and
o hide
> behind when I didn't have a good answer to the hardware sizing question
> :).
>
> On Mon, Oct 7, 2013 at 2:48 PM, Tim Vaillancourt
> wrote:
> > Fantastic article!
> >
> > Tim
> >
> >
> > On 5 October 2013 18:14, Erick Erickson wrote:
&
Fantastic article!
Tim
On 5 October 2013 18:14, Erick Erickson wrote:
> From my perspective, your question is almost impossible to
> answer, there are too many variables. See:
>
> http://searchhub.org/dev/2012/07/23/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/
>
> Best
Is there a way to make autoCommit only commit if there are pending changes,
ie: if there are 0 adds pending commit, don't autoCommit (open-a-searcher
and wipe the caches)?
Cheers,
Tim
On 2 October 2013 00:52, Dmitry Kan wrote:
> right. We've got the autoHard commit configured only atm. The so
Jetty should be sufficient, and is the more-common container for Solr.
Also, Solr tests are written for Jetty.
Lastly, I'd argue Jetty is just-as "enterprise" as Tomcat. Google App
Engine (running lots of enterprise), is Jetty-based, for example.
Cheers,
Tim
On 2 October 2013 15:44, Mark wrot
se this patch and CloudSolrServer from SolrJ in the
> > meantime.
> >
> > I don't quite know whether SOLR-5232 will make it in to 4.5 or not, it
> > hasn't been committed anywhere yet. The Solr 4.5 release is imminent, RC0
> > is looking like it'll be ready to c
changes to this code (looks like
> Shikhar already raised an issue for instance) before 4.5 is finally cut...
>
> FWIW,
> Erick
>
>
>
> On Thu, Sep 12, 2013 at 2:13 AM, Tim Vaillancourt >wrote:
>
> > Thanks Erick!
> >
> > Yeah, I think the next step will
urrently we have a plain-old least-connection
HTTP VIP. If we go "direct" to what we need to update, this will reduce
concurrency in SolrCloud a bit. Thoughts?
Thanks all!
Cheers,
Tim
On 6 September 2013 14:47, Tim Vaillancourt wrote:
Enjoy your trip, Mark! Thanks again for the
Tim
On 6 September 2013 14:47, Tim Vaillancourt wrote:
> Enjoy your trip, Mark! Thanks again for the help!
>
> Tim
>
>
> On 6 September 2013 14:18, Mark Miller wrote:
>
>> Okay, thanks, useful info. Getting on a plane, but ill look more at this
>> soon. That
I wouldn't say I love this idea, but wouldn't it be safe to LVM snapshot
the Solr index? I think this may even work on a live server, depending on
some file I/O details. Has anyone tried this?
An in-Solr solution sounds more elegant, but considering the tlog concern
Shalin mentioned, I think this
em. We want to keep that from happening.
>
> Mark
>
> Sent from my iPhone
>
> On Sep 6, 2013, at 2:05 PM, Tim Vaillancourt wrote:
>
> > Hey Mark,
> >
> > The farthest we've made it at the same batch size/volume was 12 hours
> > without this patch, b
s JVM specifically (instead of system-wide).
4) Solr's JVM thread count (already done)
5) ?
Cheers,
Tim Vaillancourt
On 6 September 2013 13:11, Mark Miller wrote:
> Did you ever get to index that long before without hitting the deadlock?
>
> There really isn't anything neg
; the ERROR-severity stack traces and returns them in
order of occurrence.
Summary of my solr.log: http://pastebin.com/pBdMAWeb
Thanks!
Tim Vaillancourt
On 6 September 2013 07:27, Markus Jelsma wrote:
> Thanks!
>
> -Original message-
> > From:Erick Erickson
> &
September 2013, Tim Vaillancourt wrote:
> Thanks so much for the explanation Mark, I owe you one (many)!
>
> We have this on our high TPS cluster and will run it through it's paces
> tomorrow. I'll provide any feedback I can, more soon! :D
>
> Cheers,
>
> Tim
>
Thanks so much for the explanation Mark, I owe you one (many)!
We have this on our high TPS cluster and will run it through it's paces
tomorrow. I'll provide any feedback I can, more soon! :D
Cheers,
Tim
olume
> > >
> > > I'm going to try and fix the root cause for 4.5 - I've suspected what
> it
> > is since early this year, but it's never personally been an issue, so
> it's
> > rolled along for a long time.
> > >
> > > Mark
Hey Alejandro,
I guess it means what you call "more than one instance".
The request handlers are at the core-level, and not the Solr
instance/global level, and within each of those cores you could have one
or more data import handlers.
Most setups have 1 DIH per core at the handler location
oud?
5) Wild-ass-theory: would more shards provide more locks (whatever they
are) on update, and thus more update throughput?
To those interested, I've provided a stacktrace of 1 of 3 nodes at this URL
in gzipped form:
https://s3.amazonaws.com/timvaillancourt.com/tmp/solr-jstack-2013-08-23.gz
Any help/suggestions/ideas on this issue, big or small, would be much
appreciated.
Thanks so much all!
Tim Vaillancourt
Here you go Erick, feel free to update this.
I am unable to assign to you, but asked for someone to do so:
https://issues.apache.org/jira/browse/SOLR-5208
Cheers,
Tim
On 21/08/13 10:40 AM, Tim Vaillancourt wrote:
Well, the mention of DIH is a bit off-topic. I'll simplify and say all
I
Well, the mention of DIH is a bit off-topic. I'll simplify and say all I
need is the ability to set ANY variables in solrconfig.xml without having
to make N number of copies of the same configuration to achieve that.
Essentially I need 10+ collections to use the exact same config dir in
Zookeeper w
Hey guys,
I have a situation where I have a lot of collections that share the same
core config in Zookeeper. For each of my SolrCloud collections, 99.9% of
the config (schema.xml, solrcloud.xml) are the same, only the
DataImportHandler parameters are different for different database
names/credenti
Try adding 'ext' to your OPTIONS= line for Jetty.
Tim
On 16/08/13 05:04 AM, Dmitry Kan wrote:
Hi,
I have the following jar in jetty/lib/ext:
log4j-1.2.16.jar
slf4j-api-1.6.6.jar
slf4j-log4j12-1.6.6.jar
jcl-over-slf4j-1.6.6.jar
jul-to-slf4j-1.6.6.jar
do you?
Dmitry
On Thu, Aug 8, 2013 at 1
From: Mark Miller [mailto:markrmil...@gmail.com]
Sent: Monday, June 03, 2013 5:07 PM
To: solr-user@lucene.apache.org
Subject: Re: SolrCloud Load Balancer "weight"
On Jun 3, 2013, at 3:33 PM, Tim Vaillancourt wrote:
Should I JIRA this? Thoughts?
Yeah - it's always been in the ba
For me the biggest deal with increased chatter between SolrCloud is
object creation and GCs.
The resulting CPU load from the increase GCing seems to affect
performance for me in some load tests, but I'm still trying to gather
hard numbers on it.
Cheers,
Tim
On 07/08/13 04:05 PM, Shawn Heis
Another option is defining the location of these jars in your
solrconfig.xml and storing the libraries external to jetty, which has
some advantages.
Eg: MySQL connector is located at '/opt/mysql_connector' and adding this
to your solrconfig.xml alongside the other lib entities:
Cheers,
{BUILD_DIR};
then
rm -rf ${BUILD_DIR}
fi
# Wget solr:
SOLR_TAR=solr-${VERSION}.tgz
if test ! -e ${SOLR_TAR};
then
wget -N ${MIRROR_BASE}/lucene/solr/${VERSION}/${SOLR_TAR}
fi
# Debian metadata:
mkdir -p ${BUILD_DIR} ${BUILD_DIR}/DEBIAN
cat <>${BUILD_DIR}/DEBIAN/control
Package: sol
" and need to expect to need to do some heavy duty
trial and error tuning on your own.
-- Jack Krupansky
-Original Message- From: Tim Vaillancourt
Sent: Saturday, July 27, 2013 4:21 PM
To: solr-user@lucene.apache.org
Subject: Re: SolrCloud 4.3.1 - "Failure to open existing log file (non
sible that your transaction
logs are simply getting too large? tlogs are truncated whenever
you do a hard commit (autocommit) with openSearcher either
true for false it doesn't matter.
FWIW,
Erick
On Fri, Jul 26, 2013 at 12:56 AM, Tim Vaillancourt
wrote:
Thanks Shawn and Yonik!
Yonik:
05:55 PM, Yonik Seeley wrote:
On Thu, Jul 25, 2013 at 7:44 PM, Tim Vaillancourt wrote:
"ERROR [2013-07-25 19:34:24.264] [org.apache.solr.common.SolrException]
Failure to open existing log file (non fatal)
That itself isn't necessarily a problem (and why it says "non fatal"
On 25 July 2013 17:13, Shawn Heisey wrote:
> On 7/25/2013 5:44 PM, Tim Vaillancourt wrote:
>
>> The transaction log error I receive after about 10-30 minutes of load
>> testing is:
>>
>> "ERROR [2013-07-25 19:34:24.264] [org.apache.solr.common.**SolrException]
&
Stack trace:
http://timvaillancourt.com.s3.amazonaws.com/tmp/solrcloud.nodeC.2013-07-25-16.jstack.gz
Cheers!
Tim
On 25 July 2013 16:44, Tim Vaillancourt wrote:
> Hey guys,
>
> I am reaching out to the Solr list with a very vague issue: under high
> load against a SolrCloud 4.3.
Hey guys,
I am reaching out to the Solr list with a very vague issue: under high load
against a SolrCloud 4.3.1 cluster of 3 instances, 3 shards, 2 replicas (2
cores per instance), I eventually see failure messages related to
transaction logs, and shortly after these stacktraces occur the cluster
re:
http://www.openlogic.com/wazi/bid/257366/Power-Java-based-web-apps-with-Jetty-application-server
and here:
http://www.infoq.com/news/2009/08/google-chose-jetty
2013/7/13 Tim Vaillancourt
We run Jetty 8 and 9 with Solr. No issues I can think of.
We use Jetty interally anyways, and it seemed to be the mo
We run Jetty 8 and 9 with Solr. No issues I can think of.
We use Jetty interally anyways, and it seemed to be the most common
container out there for Solr (from reading this mailinglist, articles,
etc), so that made me feel a bit better if I needed advice or help from
the community - not to sa
ing in the cache. Are you perhaps soft committing frequently? Soft
commits throw away all the top-level caches including documentCache I
think....
Erick
On Fri, Jun 28, 2013 at 7:23 PM, Tim Vaillancourt
wrote:
Thanks Otis,
Yeah I realized after sending my e-mail that doc cache does not warm,
howev
numbers
> are non-0, though I haven't had the chance to confirm with Solr 4.3.1.
>
> Note that you can't really autowarm document cache...
>
> Otis
> --
> Solr & ElasticSearch Support -- http://sematext.com/
> Performance Monitoring -- http://sematext.
To answer some of my own question, Shawn H's great reply on this thread
explains why I see no autoWarming on doc cache:
http://www.marshut.com/iznwr/soft-commit-and-document-cache.html
It is still unclear to me why I see no other metrics, however.
Thanks Shawn,
Tim
On 28 June 2013 16:14
Hey guys,
This has to be a stupid question/I must be doing something wrong, but after
frequent load testing with documentCache enabled under Solr 4.3.1 with
autoWarmCount=150, I'm noticing that my documentCache metrics are always
zero for non-cumlative.
At first I thought my commit rate is fast e
To answer Otis' question of whether or not this would be useful, the
trouble is, I don't know! :) It very well could be useful for my use case.
Is there any way to determine the impact of result merging (time spent?
Etc?) aside from just 'trying it'?
Cheers,
Tim
On 10 June 2013 14:48, Otis Gos
If it makes you feel better, I also considered this approach when I was in
the same situation with a separate indexer and searcher on one Physical
linux machine.
My main concern was "re-using" the FS cache between both instances - If I
replicated to myself there would be two independent copies of
e xfs. Want to
> run some benchmarks and publish the results? :)
>
> Otis
> --
> Solr & ElasticSearch Support
> http://sematext.com/
>
>
>
>
>
> On Tue, Jun 4, 2013 at 6:48 PM, Tim Vaillancourt
> wrote:
> > Hey all,
> >
> > Does anyon
ucing some consistency
things like filesystem journaling, etc (ext2?) due to SolrCloud's
additional HA with replicas? How about RAID 0 x 3 replicas or so?
Thanks!
Tim Vaillancourt
Hey guys,
I have recently looked into an issue with my Solrcloud related to very high
load when performing a full-import on DIH.
While some work could be done to improve my queries, etc in DIH, this lead
me to a new feature idea in Solr: weighted internal load balancing.
Basically, I can think o
Interesting.
In your scenario would you use commit=true, or commit=false, and do you use
auto soft/hard commits?
Secondly, if you did use auto-soft/hard commits, how would they affect this
scenario? I'm guessing even with commit=false, the autoCommits would be
triggered either by time or max
A lot of people (including me) are asking for this type of support in this
JIRA:
https://issues.apache.org/jira/browse/SOLR-4470
Although brought up frequently on the list, the effort doesn't seem to be
moving too much. I can confirm that the most recent patch on this JIRA will
work with the spec
If centralization of storage is your goal by choosing NFS, iSCSI works
reasonably well with SOLR indexes, although good local-storage will
always be the overall winner.
I noticed a near 5% degredation in overall search performance (casual
testing, nothing scientific) when moving a 40-50GB inde
liases, but I've yet to get back to
finishing it.
- Mark
On Apr 7, 2013, at 5:10 PM, Tim Vaillancourt wrote:
I aim to use this feature in more in testing soon. I'll be sure to doc what I
can.
Cheers,
Tim
On 07/04/13 12:28 PM, Mark Miller wrote:
On Apr 7, 2013, at 9:44 AM, brad
I've thought about this too, and have heard of some people running a
lightweight http proxy upstream of Solr.
With the right network restrictions (only way for a client to reach solr
is via a proxy + the nodes can still talk to each other), you could
achieve the same thing SOLR-4470 is doing,
This JIRA covers a lot of what you're asking:
https://issues.apache.org/jira/browse/SOLR-4470
I am also trying to get this sort of solution in place, but it seems to
be dying off a bit. Hopefully we can get some interest on this again,
this question comes up every few weeks, it seems.
I can
y, so there is
possibly some other junk in my logs aside from CSS.
I am running Jetty 8.1.10 and Solr 4.2.1 (stable build).
Cheers!
Tim Vaillancourt
and
> afaik as well in 3.x) you can find the stats here:
> http://host:port/solr/admin/mbeans?stats=true
> in xml or json (setting the responsewriter with wt=json) - as you like
>
> HTH
> Stefan
>
>
>
> On Wednesday, April 10, 2013 at 9:53 PM, Tim Vaillancourt wrote
Hey guys,
This feels like a silly question already, here goes:
In SolrCloud it doesn't seem obvious to me where one can grab stats
regarding caches for a given core using an http call (JSON/XML). Those
values are available in the web-based app, but I am looking for a http call
that would return t
There is also this path for the SVN guys out there:
https://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_4_2_1
Cheers,
Tim
On 05/04/13 05:53 PM, Jagdish Nomula wrote:
That works out. Thanks for shooting the link.
On Fri, Apr 5, 2013 at 5:51 PM, Jack Krupanskywrote:
You want the "ta
I aim to use this feature in more in testing soon. I'll be sure to doc
what I can.
Cheers,
Tim
On 07/04/13 12:28 PM, Mark Miller wrote:
On Apr 7, 2013, at 9:44 AM, bradhill99 wrote:
Thanks Mark for this great feature but I suggest you can update the wiki
too.
Yeah, I've stopped updating
It its in your SolrCloud-based collection's config, it won't be on disk
and only in Zookeeper.
What I did was use the XInclude feature to include a file with my
dataimport handler properties, so I'm assuming you're doing the same.
Use a relative path to the config dir in Zookeeper, ie: no path
77 matches
Mail list logo
