Re: Change way of logging OOM and InterruptedException in WorkerThread
Yes, you can do that. Thanks! Karl On Tue, Mar 15, 2022 at 2:19 PM Julien Massiera < julien.massi...@francelabs.com> wrote: > Then I'll create a ticket to just add a date to the system.err.println > output of the OOM, is it ok ? > > -Message d'origine- > De : Karl Wright > Envoyé : mardi 15 mars 2022 19:10 > À : dev > Objet : Re: Change way of logging OOM and InterruptedException in > WorkerThread > > (1) InterruptedExceptions are thrown during normal shutdown and should > generally not be logged. This is by design. > (2) OOM conditions can prevent logging from working. So if you want to > log those, fine, but please retain the System.err output. > > Karl > > > On Tue, Mar 15, 2022 at 1:51 PM Julien Massiera < > julien.massi...@francelabs.com> wrote: > > > Hi, > > > > > > > > In the WorkerThread class, OOM are logged using System.err (l849) and > > therefore cannot be formated, but more importantly, the timestamp is > > lost, which can be very problematic to investigate issues. Also, > > InterruptedExceptions are not logged at all (l844) ! > > > > Can we use Logging.threads.fatal for OOM and Logging.threads.error > > for InterruptedExceptions instead ? > > > > > > > > If it is ok, I can create a ticket and make the changes myself. > > > > > > > > Regards, > > > > Julien > > > > > > > > > > > > > >
RE: Change way of logging OOM and InterruptedException in WorkerThread
Then I'll create a ticket to just add a date to the system.err.println output of the OOM, is it ok ? -Message d'origine- De : Karl Wright Envoyé : mardi 15 mars 2022 19:10 À : dev Objet : Re: Change way of logging OOM and InterruptedException in WorkerThread (1) InterruptedExceptions are thrown during normal shutdown and should generally not be logged. This is by design. (2) OOM conditions can prevent logging from working. So if you want to log those, fine, but please retain the System.err output. Karl On Tue, Mar 15, 2022 at 1:51 PM Julien Massiera < julien.massi...@francelabs.com> wrote: > Hi, > > > > In the WorkerThread class, OOM are logged using System.err (l849) and > therefore cannot be formated, but more importantly, the timestamp is > lost, which can be very problematic to investigate issues. Also, > InterruptedExceptions are not logged at all (l844) ! > > Can we use Logging.threads.fatal for OOM and Logging.threads.error > for InterruptedExceptions instead ? > > > > If it is ok, I can create a ticket and make the changes myself. > > > > Regards, > > Julien > > > > > >
Re: Change way of logging OOM and InterruptedException in WorkerThread
(1) InterruptedExceptions are thrown during normal shutdown and should generally not be logged. This is by design. (2) OOM conditions can prevent logging from working. So if you want to log those, fine, but please retain the System.err output. Karl On Tue, Mar 15, 2022 at 1:51 PM Julien Massiera < julien.massi...@francelabs.com> wrote: > Hi, > > > > In the WorkerThread class, OOM are logged using System.err (l849) and > therefore cannot be formated, but more importantly, the timestamp is lost, > which can be very problematic to investigate issues. Also, > InterruptedExceptions are not logged at all (l844) ! > > Can we use Logging.threads.fatal for OOM and Logging.threads.error for > InterruptedExceptions instead ? > > > > If it is ok, I can create a ticket and make the changes myself. > > > > Regards, > > Julien > > > > > >
Change way of logging OOM and InterruptedException in WorkerThread
Hi, In the WorkerThread class, OOM are logged using System.err (l849) and therefore cannot be formated, but more importantly, the timestamp is lost, which can be very problematic to investigate issues. Also, InterruptedExceptions are not logged at all (l844) ! Can we use Logging.threads.fatal for OOM and Logging.threads.error for InterruptedExceptions instead ? If it is ok, I can create a ticket and make the changes myself. Regards, Julien
[jira] [Resolved] (CONNECTORS-1700) TikaServiceRmeta: Add options to filter out metadata based on size
[ https://issues.apache.org/jira/browse/CONNECTORS-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Massiera resolved CONNECTORS-1700. - Fix Version/s: ManifoldCF next Resolution: Fixed r1898949 > TikaServiceRmeta: Add options to filter out metadata based on size > -- > > Key: CONNECTORS-1700 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1700 > Project: ManifoldCF > Issue Type: Improvement > Components: Tika service connector >Affects Versions: ManifoldCF 2.21 >Reporter: Julien Massiera >Assignee: Julien Massiera >Priority: Major > Fix For: ManifoldCF next > > > Some files may contain abnormally big metadata (several MB, be it for the > metadata values, but also for the total amount of metadata) that can be > problematic concerning the memory consumption of the connector. > To avoid this, we can provide job configuration options for the > TikaServiceRmetaConnector to set limits on both metadata values and global > amount of metadata, and exclude metadata that exceed the limits -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (CONNECTORS-1700) TikaServiceRmeta: Add options to filter out metadata based on size
Julien Massiera created CONNECTORS-1700: --- Summary: TikaServiceRmeta: Add options to filter out metadata based on size Key: CONNECTORS-1700 URL: https://issues.apache.org/jira/browse/CONNECTORS-1700 Project: ManifoldCF Issue Type: Improvement Components: Tika service connector Affects Versions: ManifoldCF 2.21 Reporter: Julien Massiera Assignee: Julien Massiera Some files may contain abnormally big metadata (several MB, be it for the metadata values, but also for the total amount of metadata) that can be problematic concerning the memory consumption of the connector. To avoid this, we can provide job configuration options for the TikaServiceRmetaConnector to set limits on both metadata values and global amount of metadata, and exclude metadata that exceed the limits -- This message was sent by Atlassian Jira (v8.20.1#820001)
