Solr docker container logging certain entries to syslog
Hello,
I have a solr (version 6.6.6) docker container running where a few logs are
being written to /var/log/messages. My log4j.properties has the rootLogger set
to file and console. I have attached it for reference. My question is why are
some logs still being written to /var/log/messages and what can I do to stop
that?
Example of logs in /var/log/messages:
Aug 21 03:49:45 devstack 8fa87b1b73a8: 2020-08-21 03:49:45.209 INFO
(qtp1282788025-249) [ x:ask_a_question] o.a.s.c.S.Request [ask_a_question]
webapp=/solr path=/dataimport
params={RequestTimeout=60&qt=/dataimport&clean=false&command=full-import}
status=0 QTime=207
Aug 21 03:49:45 devstack e0ee5f7c16d0: 2020-08-21 03:49:45.700 INFO
(qtp1282788025-76) [ x:ta_portal] o.a.s.u.p.LogUpdateProcessorFactory
[ta_portal] webapp=/solr path=/dataimport
params={RequestTimeout=60&qt=/dataimport&clean=false&command=full-import}{} 0
15698
Aug 21 03:49:47 devstack 8fa87b1b73a8: 2020-08-21 03:49:47.823 INFO
(qtp1282788025-220) [ x:ta_portal] o.a.s.u.p.LogUpdateProcessorFactory
[ta_portal] webapp=/solr path=/dataimport
params={RequestTimeout=60&qt=/dataimport&clean=false&command=full-import}{} 0
17822
Thanks,
Sowmya.
Solr docker container logging certain entries to syslog
Hello,
I have a solr (version 6.6.6) docker container running where a few logs are
being written to /var/log/messages. My log4j.properties has the rootLogger set
to file and console. I have attached it for reference. My question is why are
some logs still being written to /var/log/messages and what can I do to stop
that?
Example of logs in /var/log/messages:
Aug 21 03:49:45 devstack 8fa87b1b73a8: 2020-08-21 03:49:45.209 INFO
(qtp1282788025-249) [ x:ask_a_question] o.a.s.c.S.Request [ask_a_question]
webapp=/solr path=/dataimport
params={RequestTimeout=60&qt=/dataimport&clean=false&command=full-import}
status=0 QTime=207
Aug 21 03:49:45 devstack e0ee5f7c16d0: 2020-08-21 03:49:45.700 INFO
(qtp1282788025-76) [ x:ta_portal] o.a.s.u.p.LogUpdateProcessorFactory
[ta_portal] webapp=/solr path=/dataimport
params={RequestTimeout=60&qt=/dataimport&clean=false&command=full-import}{} 0
15698
Aug 21 03:49:47 devstack 8fa87b1b73a8: 2020-08-21 03:49:47.823 INFO
(qtp1282788025-220) [ x:ta_portal] o.a.s.u.p.LogUpdateProcessorFactory
[ta_portal] webapp=/solr path=/dataimport
params={RequestTimeout=60&qt=/dataimport&clean=false&command=full-import}{} 0
17822
Thanks,
Sowmya.
RE: CachedSqlEntityProcessor with delta-import
Thanks. We did implement the delete by query on another core and thought of
giving the delta import a try here. Looks like differential via full index and
deletes using delete by id/query is the way to go.
-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Tuesday, October 25, 2016 12:31 PM
To: solr-user
Subject: Re: CachedSqlEntityProcessor with delta-import
Why not use delete by id rather than query? It'll be more efficient
Probably not a big deal though.
On Tue, Oct 25, 2016 at 1:47 AM, Aniket Khare wrote:
> Hi Sowmya,
>
> I my case I have implemeneted the data indexing suggested by James and
> for deleting the reords I have created my own data indexing job which
> will call the delete API periodically by passing the list of unique Id.
> https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+I
> ndex+Handlers
>
> http://localhost:8983/solr/update?stream.body=
> id:1234&commit=true
>
> Thanks,
> Aniket S. Khare
>
> On Tue, Oct 25, 2016 at 1:32 AM, Mohan, Sowmya wrote:
>
>> Thanks James. That's what I was using before. But I also wanted to
>> perform deletes using deletedPkQuery and hence switched to delta
>> imports. The problem with using deletedPkQuery with the full import
>> is that dataimporter.last_index_time is no longer accurate.
>>
>> Below is an example of my deletedPkQuery. If run the full-import for
>> a differential index, that would update the last index time. Running
>> the delta import to remove the deleted records then wouldn't do
>> anything since nothing changed since the last index time.
>>
>>
>> deletedPkQuery="SELECT id
>> FROM content
>> WHERE active = 1 AND lastUpdate >
>> '${dataimporter.last_index_time}'"
>>
>>
>>
>>
>>
>>
>> -Original Message-
>> From: Dyer, James [mailto:james.d...@ingramcontent.com]
>> Sent: Friday, October 21, 2016 4:23 PM
>> To: solr-user@lucene.apache.org
>> Subject: RE: CachedSqlEntityProcessor with delta-import
>>
>> Sowmya,
>>
>> My memory is that the cache feature does not work with Delta Imports.
>> In fact, I believe that nearly all DIH features except straight JDBC
>> imports do not work with Delta Imports. My advice is to not use the
>> Delta Import feature at all as the same result can (often
>> more-efficiently) be accomplished following the approach outlined here:
>> https://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport
>>
>> James Dyer
>> Ingram Content Group
>>
>> -Original Message-
>> From: Mohan, Sowmya [mailto:sowmya.mo...@icf.com]
>> Sent: Tuesday, October 18, 2016 10:07 AM
>> To: solr-user@lucene.apache.org
>> Subject: CachedSqlEntityProcessor with delta-import
>>
>> Good morning,
>>
>> Can CachedSqlEntityProcessor be used with delta-import? In my setup
>> when running a delta-import with CachedSqlEntityProcessor, the child
>> entity values are not correctly updated for the parent record. I am on Solr
>> 4.3.
>> Has anyone experienced this and if so how to resolve it?
>>
>> Thanks,
>> Sowmya.
>>
>>
>
>
> --
> Regards,
>
> Aniket S. Khare
RE: CachedSqlEntityProcessor with delta-import
Thanks James. That's what I was using before. But I also wanted to perform
deletes using deletedPkQuery and hence switched to delta imports. The problem
with using deletedPkQuery with the full import is that
dataimporter.last_index_time is no longer accurate.
Below is an example of my deletedPkQuery. If run the full-import for a
differential index, that would update the last index time. Running the delta
import to remove the deleted records then wouldn't do anything since nothing
changed since the last index time.
deletedPkQuery="SELECT id
FROM content
WHERE active = 1 AND lastUpdate >
'${dataimporter.last_index_time}'"
-Original Message-
From: Dyer, James [mailto:james.d...@ingramcontent.com]
Sent: Friday, October 21, 2016 4:23 PM
To: solr-user@lucene.apache.org
Subject: RE: CachedSqlEntityProcessor with delta-import
Sowmya,
My memory is that the cache feature does not work with Delta Imports. In fact,
I believe that nearly all DIH features except straight JDBC imports do not work
with Delta Imports. My advice is to not use the Delta Import feature at all as
the same result can (often more-efficiently) be accomplished following the
approach outlined here:
https://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport
James Dyer
Ingram Content Group
-Original Message-
From: Mohan, Sowmya [mailto:sowmya.mo...@icf.com]
Sent: Tuesday, October 18, 2016 10:07 AM
To: solr-user@lucene.apache.org
Subject: CachedSqlEntityProcessor with delta-import
Good morning,
Can CachedSqlEntityProcessor be used with delta-import? In my setup when
running a delta-import with CachedSqlEntityProcessor, the child entity values
are not correctly updated for the parent record. I am on Solr 4.3. Has anyone
experienced this and if so how to resolve it?
Thanks,
Sowmya.
CachedSqlEntityProcessor with delta-import
Good morning, Can CachedSqlEntityProcessor be used with delta-import? In my setup when running a delta-import with CachedSqlEntityProcessor, the child entity values are not correctly updated for the parent record. I am on Solr 4.3. Has anyone experienced this and if so how to resolve it? Thanks, Sowmya.
