[
https://issues.apache.org/jira/browse/LENS-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642642#comment-14642642
]
Amareshwari Sriramadasu edited comment on LENS-426 at 7/27/15 12:15 PM:
------------------------------------------------------------------------
I'm looking at improving logs on estimate path, after looking at one of the
estimate calls, I'm posting the frequent log that comes in the log files. Does
anyone think following logs are too much information and we should reduce?
* Logs to do with denormalized column and its corresponding optional dimension :
{noformat}
27 Jul 2015 08:50:19,465 [estimate-96] INFO
org.apache.lens.cube.parse.DenormalizationResolver - Adding denormalized
column for column:xxxx for table:fact_table
27 Jul 2015 08:50:19,465 [estimate-95] INFO
org.apache.lens.cube.parse.CubeQueryContext - Adding optional dimension:dimx
optDim:CubeQueryContext.OptionalDimCtx(colQueried=[xxxx],
requiredForCandidates=[....], isRequiredInJoinChain=false) for column:xxxx
isRef:true
27 Jul 2015 08:50:32,883 [estimate-95] INFO
org.apache.lens.cube.parse.DenormalizationResolver - Picked reference for xxxx
ref:[DenormalizationResolver.PickedReference(ref=ReferencedDimAtrribute(super=BaseDimAttribute(super=xxxx:Creative
Name#start:2011-04-01-00:0.0, type=string,
numOfDistinctValues=Optional.absent()), references=[], isJoinKey=false,
chainName=chainx, refColumn=name), reference=null, srcAlias=rrcube,
pickedFor=fact_table)]
{noformat}
* Storage table resolver logs
{noformat}
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Storage tables containing
Partition it = '2015-06-19' are: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Adding existing partitionit
= '2015-06-19'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Looking for look ahead
process time partitions for it = '2015-06-19'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Update period is the least
update period
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - candidate storage tables for
searching partitions: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Storage tables containing
Partition it = '2015-06-20' are: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Adding existing partitionit
= '2015-06-20'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Looking for look ahead
process time partitions for it = '2015-06-20'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Update period is the least
update period
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - candidate storage tables for
searching partitions: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Storage tables containing
Partition it = '2015-06-21' are: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Adding existing partitionit
= '2015-06-21'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Looking for look ahead
process time partitions for it = '2015-06-21'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Update period is the least
update period
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - candidate storage tables for
searching partitions: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Storage tables containing
Partition it = '2015-06-22' are: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Adding existing partitionit
= '2015-06-22'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Looking for look ahead
process time partitions for it = '2015-06-22'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Update period is the least
update period
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - candidate storage tables for
searching partitions: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Storage tables containing
Partition it = '2015-06-23' are: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Adding existing partitionit
= '2015-06-23'
{noformat}
was (Author: amareshwari):
I'm looking at improving logs on estimate path, after looking at one of the
estimate calls, I'm posting the frequent log that comes in the log files. Does
anyone think following logs are too much information and we should reduce?
* Logs to do with denormalized column and its corresponding optional dimension :
{noformat}
27 Jul 2015 08:50:19,465 [estimate-96] INFO
org.apache.lens.cube.parse.DenormalizationResolver - Adding denormalized
column for column:xxxx for table:fact_table
27 Jul 2015 08:50:19,465 [estimate-95] INFO
org.apache.lens.cube.parse.CubeQueryContext - Adding optional dimension:dimx
optDim:CubeQueryContext.OptionalDimCtx(colQueried=[xxxx],
requiredForCandidates=[....], isRequiredInJoinChain=false) for column:xxxx
isRef:true
{noformat}
27 Jul 2015 08:50:32,883 [estimate-95] INFO
org.apache.lens.cube.parse.DenormalizationResolver - Picked reference for xxxx
ref:[DenormalizationResolver.PickedReference(ref=ReferencedDimAtrribute(super=BaseDimAttribute(super=xxxx:Creative
Name#start:2011-04-01-00:0.0, type=string,
numOfDistinctValues=Optional.absent()), references=[], isJoinKey=false,
chainName=chainx, refColumn=name), reference=null, srcAlias=rrcube,
pickedFor=fact_table)]
* Storage table resolver logs
{noformat}
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Storage tables containing
Partition it = '2015-06-19' are: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Adding existing partitionit
= '2015-06-19'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Looking for look ahead
process time partitions for it = '2015-06-19'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Update period is the least
update period
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - candidate storage tables for
searching partitions: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Storage tables containing
Partition it = '2015-06-20' are: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Adding existing partitionit
= '2015-06-20'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Looking for look ahead
process time partitions for it = '2015-06-20'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Update period is the least
update period
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - candidate storage tables for
searching partitions: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Storage tables containing
Partition it = '2015-06-21' are: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Adding existing partitionit
= '2015-06-21'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Looking for look ahead
process time partitions for it = '2015-06-21'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Update period is the least
update period
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - candidate storage tables for
searching partitions: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Storage tables containing
Partition it = '2015-06-22' are: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Adding existing partitionit
= '2015-06-22'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Looking for look ahead
process time partitions for it = '2015-06-22'
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Update period is the least
update period
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - candidate storage tables for
searching partitions: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Storage tables containing
Partition it = '2015-06-23' are: [s1_fact_table]
27 Jul 2015 08:50:32,878 [estimate-95] INFO
org.apache.lens.cube.parse.StorageTableResolver - Adding existing partitionit
= '2015-06-23'
{noformat}
> Review logging in estimate path
> -------------------------------
>
> Key: LENS-426
> URL: https://issues.apache.org/jira/browse/LENS-426
> Project: Apache Lens
> Issue Type: Improvement
> Components: cube, server
> Reporter: Jaideep Dhok
> Assignee: Amareshwari Sriramadasu
> Fix For: 2.3
>
>
> # Review logs which might be doing expensive toString computations
> # Review logs inside big loops
> # Check log levels
> # Use parameterised logging wherever possible
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
