[
https://issues.apache.org/jira/browse/LENS-602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14624281#comment-14624281
]
Jaideep Dhok commented on LENS-602:
-----------------------------------
I am not sure about grepping the main log file. I think Option 2: having a
different logger object for each log file is a cleaner solution. There the
tradeoff is only the number of simultaneous open files.
We can retain reference to the logger in the query context, and close it when
query is purged from lens memory.
Reasons I am not in favor of grepping the main log file
# Grepping is computationally expensive, we need to limit (or perhaps
synchronize) the number of threads simultaneously grepping the log file.
# If admin is splitting main log files into several based on class names or log
levels, how do we make sure we are grepping through all of these files?
# Similar to above, how will we grep across time rotated or archived log files?
what if admin changes logging format?
# Admin should get to decide what to log where. If we build the feature on the
assumption that there is always a single log file, we are restricting admin
control.
I agree that there should be a high level feature flag to enable/disable this
entire feature.
> Easy access to per-query lens server logs
> -----------------------------------------
>
> Key: LENS-602
> URL: https://issues.apache.org/jira/browse/LENS-602
> Project: Apache Lens
> Issue Type: New Feature
> Reporter: Angad Singh
> Assignee: Amareshwari Sriramadasu
> Labels: Hackathon-July
>
> Right now one has to have access to the lens server machine and find a lens
> query's job logs manually in lensserver.log file. This is neither scalable
> nor user-friendly for a shared multi-tenanted lens server.
> Just throwing server-exceptions to the client or showing the job ID is also
> often not enough. Even when the query succeeds, one needs to see how
> candidate fact tables and their columns, etc. were pruned and how join chains
> were resolved, for example. That is only possible by seeing the lens server
> logs.
> Instead of that, this ticket is to propose that lens store logs for each lens
> query in a different log file on the server and that there be a REST end
> point to access a query's log (by query ID). That URL can be pasted on the
> client shell when the query is launched and the user can see a tailed log of
> all that lens is doing behind the scenes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
