Re: How to Configure Solr For Test Purposes?
Hi All;
I have defined just that:
autoSoftCommit
maxTime1/maxTime
/autoSoftCommit
then turned of hard commit. I've run my tests time and time again and did
not get any error. Who wants to write a unit test that interacts with Solr
as like my situation can use it.
Thanks;
Furkan KAMACI
2014-05-26 23:37 GMT+03:00 Furkan KAMACI furkankam...@gmail.com:
Hi Shawn;
I know that it is a bad practise but I just commit up to 5 documents and
there will not be more than 5 documents at any time at any test method. It
is just for test purpose to see that my API works. I want to have
automatic tests.
What do you suggest for my purpose? If a test case fails re-running it for
some times maybe a solution? What kind of configuration do you suggest for
my Solr configuration?
Thanks;
Furkan KAMACI
26 May 2014 21:03 tarihinde Shawn Heisey s...@elyograg.org yazdı:
On 5/26/2014 10:57 AM, Furkan KAMACI wrote:
Hi;
I run Solr within my Test Suite. I delete documents or atomically update
them and check whether if it works or not. I know that I have to setup a
hard/soft commit timing for my test Solr. However even I have that
settings:
autoCommit
maxTime1/maxTime
openSearchertrue/openSearcher
/autoCommit
autoSoftCommit
maxTime1/maxTime
/autoSoftCommit
I hope you know that this is BAD configuration. Doing automatic commits
on an interval of 1 millisecond is asking for a whole host of problems.
In some cases, this could do a commit after every single document that
is indexed, which is NOT recommended at all. The openSearcher setting
of true on autoCommit makes it even worse. There's no reason to do
both autoSoftCommit and autoCommit with openSearcher=true. I don't know
which one wins between autoCommit and autoSoftCommit if they both have
the same config, but I would guess the hard commit does.
and even I wait (Thread.sleep()) for a time to wait Solr *sometimes* my
tests are failed. I get fail error even I increase wait time. Example
of a
sometimes failed code piece:
for (int i = 0; i dummyDocumentSize; i++) {
deleteById(id + i);
dummyDocumentSize--;
queryResponse = query(solrParams);
assertTrue(queryResponse.getResults().size() ==
dummyDocumentSize);
}
at debug mode if I wait for Solr to reflect changes I see that I do not
get
error. What do you think, what kind of configuration I should have for
such
kind of purposes?
Chances are that commits are going to take longer than 1 millisecond.
If you're actively indexing, the system is going to be trying to stack
up lots of commits at the same time. The maxWarmingSearchers value will
limit the number of new searchers that can be opened, but it will not
stop the commits themselves. When lots of commits are going on, each
one will take *even longer* to complete, which probably explains the
problem.
Thanks,
Shawn
Re: How to Configure Solr For Test Purposes?
What do you suggest for my purpose? If a test case fails re-running it for
some times maybe a solution? What kind of configuration do you suggest for
my Solr configuration?
From the snippet of test that you showed, it looks like it's testing only
Solr functionality. So, first make sure this is a test that you really
need. Solr has it's own tests, and it you feel it could use more (for some
specific case or context), I'd open a Jira and try to get the test inside
Solr.
If my impression is wrong and your test is actually testing your code, then
I'd suggest you to use a specific soft commit call with waitSearcher = true
on your test instead of relying on the autocommit (and remove the
autocommit completely from your solrconfig).
Tomás
Thanks;
Furkan KAMACI
26 May 2014 21:03 tarihinde Shawn Heisey s...@elyograg.org yazdı:
On 5/26/2014 10:57 AM, Furkan KAMACI wrote:
Hi;
I run Solr within my Test Suite. I delete documents or atomically
update
them and check whether if it works or not. I know that I have to setup
a
hard/soft commit timing for my test Solr. However even I have that
settings:
autoCommit
maxTime1/maxTime
openSearchertrue/openSearcher
/autoCommit
autoSoftCommit
maxTime1/maxTime
/autoSoftCommit
I hope you know that this is BAD configuration. Doing automatic commits
on an interval of 1 millisecond is asking for a whole host of problems.
In some cases, this could do a commit after every single document that
is indexed, which is NOT recommended at all. The openSearcher setting
of true on autoCommit makes it even worse. There's no reason to do
both autoSoftCommit and autoCommit with openSearcher=true. I don't know
which one wins between autoCommit and autoSoftCommit if they both have
the same config, but I would guess the hard commit does.
and even I wait (Thread.sleep()) for a time to wait Solr *sometimes* my
tests are failed. I get fail error even I increase wait time. Example
of a
sometimes failed code piece:
for (int i = 0; i dummyDocumentSize; i++) {
deleteById(id + i);
dummyDocumentSize--;
queryResponse = query(solrParams);
assertTrue(queryResponse.getResults().size() ==
dummyDocumentSize);
}
at debug mode if I wait for Solr to reflect changes I see that I do not
get
error. What do you think, what kind of configuration I should have for
such
kind of purposes?
Chances are that commits are going to take longer than 1 millisecond.
If you're actively indexing, the system is going to be trying to stack
up lots of commits at the same time. The maxWarmingSearchers value will
limit the number of new searchers that can be opened, but it will not
stop the commits themselves. When lots of commits are going on, each
one will take *even longer* to complete, which probably explains the
problem.
Thanks,
Shawn
Re: How to Configure Solr For Test Purposes?
Hi;
I've developed a Proxy application that takes request from clients and
sends them to Solr then gets response and sends them to client as response.
So, I am testing my application, my proxy for Solr.
Thanks;
Furkan KAMACI
2014-05-27 14:52 GMT+03:00 Tomás Fernández Löbbe tomasflo...@gmail.com:
What do you suggest for my purpose? If a test case fails re-running it
for
some times maybe a solution? What kind of configuration do you suggest
for
my Solr configuration?
From the snippet of test that you showed, it looks like it's testing only
Solr functionality. So, first make sure this is a test that you really
need. Solr has it's own tests, and it you feel it could use more (for some
specific case or context), I'd open a Jira and try to get the test inside
Solr.
If my impression is wrong and your test is actually testing your code, then
I'd suggest you to use a specific soft commit call with waitSearcher = true
on your test instead of relying on the autocommit (and remove the
autocommit completely from your solrconfig).
Tomás
Thanks;
Furkan KAMACI
26 May 2014 21:03 tarihinde Shawn Heisey s...@elyograg.org yazdı:
On 5/26/2014 10:57 AM, Furkan KAMACI wrote:
Hi;
I run Solr within my Test Suite. I delete documents or atomically
update
them and check whether if it works or not. I know that I have to
setup
a
hard/soft commit timing for my test Solr. However even I have that
settings:
autoCommit
maxTime1/maxTime
openSearchertrue/openSearcher
/autoCommit
autoSoftCommit
maxTime1/maxTime
/autoSoftCommit
I hope you know that this is BAD configuration. Doing automatic
commits
on an interval of 1 millisecond is asking for a whole host of problems.
In some cases, this could do a commit after every single document that
is indexed, which is NOT recommended at all. The openSearcher setting
of true on autoCommit makes it even worse. There's no reason to do
both autoSoftCommit and autoCommit with openSearcher=true. I don't
know
which one wins between autoCommit and autoSoftCommit if they both
have
the same config, but I would guess the hard commit does.
and even I wait (Thread.sleep()) for a time to wait Solr *sometimes*
my
tests are failed. I get fail error even I increase wait time.
Example
of a
sometimes failed code piece:
for (int i = 0; i dummyDocumentSize; i++) {
deleteById(id + i);
dummyDocumentSize--;
queryResponse = query(solrParams);
assertTrue(queryResponse.getResults().size() ==
dummyDocumentSize);
}
at debug mode if I wait for Solr to reflect changes I see that I do
not
get
error. What do you think, what kind of configuration I should have
for
such
kind of purposes?
Chances are that commits are going to take longer than 1 millisecond.
If you're actively indexing, the system is going to be trying to stack
up lots of commits at the same time. The maxWarmingSearchers value
will
limit the number of new searchers that can be opened, but it will not
stop the commits themselves. When lots of commits are going on, each
one will take *even longer* to complete, which probably explains the
problem.
Thanks,
Shawn
How to Configure Solr For Test Purposes?
Hi;
I run Solr within my Test Suite. I delete documents or atomically update
them and check whether if it works or not. I know that I have to setup a
hard/soft commit timing for my test Solr. However even I have that settings:
autoCommit
maxTime1/maxTime
openSearchertrue/openSearcher
/autoCommit
autoSoftCommit
maxTime1/maxTime
/autoSoftCommit
and even I wait (Thread.sleep()) for a time to wait Solr *sometimes* my
tests are failed. I get fail error even I increase wait time. Example of a
sometimes failed code piece:
for (int i = 0; i dummyDocumentSize; i++) {
deleteById(id + i);
dummyDocumentSize--;
queryResponse = query(solrParams);
assertTrue(queryResponse.getResults().size() == dummyDocumentSize);
}
at debug mode if I wait for Solr to reflect changes I see that I do not get
error. What do you think, what kind of configuration I should have for such
kind of purposes?
Thanks;
Furkan KAMACI
Re: How to Configure Solr For Test Purposes?
On 5/26/2014 10:57 AM, Furkan KAMACI wrote:
Hi;
I run Solr within my Test Suite. I delete documents or atomically update
them and check whether if it works or not. I know that I have to setup a
hard/soft commit timing for my test Solr. However even I have that settings:
autoCommit
maxTime1/maxTime
openSearchertrue/openSearcher
/autoCommit
autoSoftCommit
maxTime1/maxTime
/autoSoftCommit
I hope you know that this is BAD configuration. Doing automatic commits
on an interval of 1 millisecond is asking for a whole host of problems.
In some cases, this could do a commit after every single document that
is indexed, which is NOT recommended at all. The openSearcher setting
of true on autoCommit makes it even worse. There's no reason to do
both autoSoftCommit and autoCommit with openSearcher=true. I don't know
which one wins between autoCommit and autoSoftCommit if they both have
the same config, but I would guess the hard commit does.
and even I wait (Thread.sleep()) for a time to wait Solr *sometimes* my
tests are failed. I get fail error even I increase wait time. Example of a
sometimes failed code piece:
for (int i = 0; i dummyDocumentSize; i++) {
deleteById(id + i);
dummyDocumentSize--;
queryResponse = query(solrParams);
assertTrue(queryResponse.getResults().size() == dummyDocumentSize);
}
at debug mode if I wait for Solr to reflect changes I see that I do not get
error. What do you think, what kind of configuration I should have for such
kind of purposes?
Chances are that commits are going to take longer than 1 millisecond.
If you're actively indexing, the system is going to be trying to stack
up lots of commits at the same time. The maxWarmingSearchers value will
limit the number of new searchers that can be opened, but it will not
stop the commits themselves. When lots of commits are going on, each
one will take *even longer* to complete, which probably explains the
problem.
Thanks,
Shawn
Re: How to Configure Solr For Test Purposes?
Hi Shawn;
I know that it is a bad practise but I just commit up to 5 documents and
there will not be more than 5 documents at any time at any test method. It
is just for test purpose to see that my API works. I want to have
automatic tests.
What do you suggest for my purpose? If a test case fails re-running it for
some times maybe a solution? What kind of configuration do you suggest for
my Solr configuration?
Thanks;
Furkan KAMACI
26 May 2014 21:03 tarihinde Shawn Heisey s...@elyograg.org yazdı:
On 5/26/2014 10:57 AM, Furkan KAMACI wrote:
Hi;
I run Solr within my Test Suite. I delete documents or atomically update
them and check whether if it works or not. I know that I have to setup a
hard/soft commit timing for my test Solr. However even I have that
settings:
autoCommit
maxTime1/maxTime
openSearchertrue/openSearcher
/autoCommit
autoSoftCommit
maxTime1/maxTime
/autoSoftCommit
I hope you know that this is BAD configuration. Doing automatic commits
on an interval of 1 millisecond is asking for a whole host of problems.
In some cases, this could do a commit after every single document that
is indexed, which is NOT recommended at all. The openSearcher setting
of true on autoCommit makes it even worse. There's no reason to do
both autoSoftCommit and autoCommit with openSearcher=true. I don't know
which one wins between autoCommit and autoSoftCommit if they both have
the same config, but I would guess the hard commit does.
and even I wait (Thread.sleep()) for a time to wait Solr *sometimes* my
tests are failed. I get fail error even I increase wait time. Example
of a
sometimes failed code piece:
for (int i = 0; i dummyDocumentSize; i++) {
deleteById(id + i);
dummyDocumentSize--;
queryResponse = query(solrParams);
assertTrue(queryResponse.getResults().size() ==
dummyDocumentSize);
}
at debug mode if I wait for Solr to reflect changes I see that I do not
get
error. What do you think, what kind of configuration I should have for
such
kind of purposes?
Chances are that commits are going to take longer than 1 millisecond.
If you're actively indexing, the system is going to be trying to stack
up lots of commits at the same time. The maxWarmingSearchers value will
limit the number of new searchers that can be opened, but it will not
stop the commits themselves. When lots of commits are going on, each
one will take *even longer* to complete, which probably explains the
problem.
Thanks,
Shawn
