[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-12 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684077#comment-16684077
 ] 

Hadoop QA commented on OOZIE-3381:
--

PreCommit-OOZIE-Build started


> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-12 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684221#comment-16684221
 ] 

Hadoop QA commented on OOZIE-3381:
--


Testing JIRA OOZIE-3381

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 3063
.Tests failed : 1
.Tests in error   : 0
.Tests timed out  : 0

{color:red}-1{color} [ERROR] There are [1] test failures in [core]. Listing 
only the first [5] ones
testRetryConsoleUrlForked:org.apache.oozie.command.wf.TestWorkflowActionRetryInfoXCommand

Check console output for the full list of errors/failures
.{color:orange}Tests failed at first run:{color}
TestPurgeXCommand#testPurgeBundleWithCoordChildWithWFChild3MoreThanLimit
TestZKUUIDService#testFallback
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/915/



> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and 

[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-14 Thread Andras Piros (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16686507#comment-16686507
 ] 

Andras Piros commented on OOZIE-3381:
-

Test failure is unrelated and caused by OOZIE-3384. [~kmarton] can you please 
review the patch? Thanks!

> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-15 Thread Julia Kinga Marton (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687849#comment-16687849
 ] 

Julia Kinga Marton commented on OOZIE-3381:
---

[~andras.piros], can you please upload the patch to the Review Board?

> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-15 Thread Andras Piros (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687910#comment-16687910
 ] 

Andras Piros commented on OOZIE-3381:
-

[~kmarton] sure thing, [here u go|https://reviews.apache.org/r/69348/].

> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-15 Thread Julia Kinga Marton (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688195#comment-16688195
 ] 

Julia Kinga Marton commented on OOZIE-3381:
---

Thank you [~andras.piros]! I left some comments in the review. Mostly nice to 
have ones.

> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-20 Thread Andras Piros (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693044#comment-16693044
 ] 

Andras Piros commented on OOZIE-3381:
-

Thanks for the review [~kmarton]! Updated patch based on review comments.

> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch, OOZIE-3381.002.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-20 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693058#comment-16693058
 ] 

Hadoop QA commented on OOZIE-3381:
--

PreCommit-OOZIE-Build started


> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch, OOZIE-3381.002.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-20 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693172#comment-16693172
 ] 

Hadoop QA commented on OOZIE-3381:
--


Testing JIRA OOZIE-3381

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in 
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3075
.{color:orange}Tests failed at first run:{color}
TestPurgeXCommand#testPurgeCoordWithWFChild2
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/918/



> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch, OOZIE-3381.002.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-22 Thread Julia Kinga Marton (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16695758#comment-16695758
 ] 

Julia Kinga Marton commented on OOZIE-3381:
---

Thank you [~andras.piros] for the patch! +1

> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch, OOZIE-3381.002.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (OOZIE-3381) [coordinator] Enhance logging of CoordElFunctions

2018-11-22 Thread Julia Kinga Marton (JIRA)


[ 
https://issues.apache.org/jira/browse/OOZIE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16695761#comment-16695761
 ] 

Julia Kinga Marton commented on OOZIE-3381:
---

committed to master.

> [coordinator] Enhance logging of CoordElFunctions
> -
>
> Key: OOZIE-3381
> URL: https://issues.apache.org/jira/browse/OOZIE-3381
> Project: Oozie
>  Issue Type: Task
>  Components: coordinator
>Affects Versions: 5.1.0
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3381.001.patch, OOZIE-3381.002.patch
>
>
> When using [{{$\{coord:latest\(n\)}}} coordinator EL 
> function|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.6.6._coord:latestint_n_EL_Function_for_Synchronous_Datasets]
>  inside an input dataset dependency, it's often the case that more 
> information is needed how many HDFS URIs are being checked for each 
> {{}}.
> Right now we don't have this information. While debugging and fine tuning 
> parameters like {{dataset frequency}}, {{initial-instance}}, and {{data-in}} 
> {{instance}}, it would be very useful to know how many HDFS roundtrips are 
> issues by the current settings {{CoordELFunctions#coord_latestRange_sync()}} 
> and {{CoordELFunctions#coord_futureRange_sync()}} having called 
> {{DFSClient#exists()}}. We need appropriate logging there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)