[jira] [Updated] (OOZIE-2657) Clean up redundant access modifiers from oozie interfaces
[
https://issues.apache.org/jira/browse/OOZIE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna updated OOZIE-2657:
--
Fix Version/s: 4.3.0
> Clean up redundant access modifiers from oozie interfaces
> -
>
> Key: OOZIE-2657
> URL: https://issues.apache.org/jira/browse/OOZIE-2657
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Minor
> Labels: code-cleanup
> Fix For: 4.3.0
>
> Attachments: OOZIE-2657-00.patch
>
>
> By default java {{interface}} fields are {{public static final}}. Some of the
> Oozie interface still explicitly declare these redundant access modifier. I
> think we can clean it.
> Similarly, methods declared in the interface are {{public}}. This redundant
> declaration can be cleaned up.
> {code}
> public interface RestConstants {
> public static final String VERSIONS = "versions";
> public static final String JOB = "job";
> ...
> }
> public interface JsonTags {
> public static final String OOZIE_SAFE_MODE = "safeMode"; //Applicable for
> V0 only
> public static final String OOZIE_SYSTEM_MODE = "systemMode";
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2657) Clean up redundant access modifiers from oozie interfaces
[
https://issues.apache.org/jira/browse/OOZIE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439654#comment-15439654
]
Abhishek Bafna commented on OOZIE-2657:
---
Testcase failure is not related. Test passes locally.
> Clean up redundant access modifiers from oozie interfaces
> -
>
> Key: OOZIE-2657
> URL: https://issues.apache.org/jira/browse/OOZIE-2657
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Minor
> Labels: code-cleanup
> Attachments: OOZIE-2657-00.patch
>
>
> By default java {{interface}} fields are {{public static final}}. Some of the
> Oozie interface still explicitly declare these redundant access modifier. I
> think we can clean it.
> Similarly, methods declared in the interface are {{public}}. This redundant
> declaration can be cleaned up.
> {code}
> public interface RestConstants {
> public static final String VERSIONS = "versions";
> public static final String JOB = "job";
> ...
> }
> public interface JsonTags {
> public static final String OOZIE_SAFE_MODE = "safeMode"; //Applicable for
> V0 only
> public static final String OOZIE_SYSTEM_MODE = "systemMode";
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2657) Clean up redundant access modifiers from oozie interfaces
[
https://issues.apache.org/jira/browse/OOZIE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15439233#comment-15439233
]
Abhishek Bafna commented on OOZIE-2657:
---
[~rkanter], [~jaydeepvishwakarma] Can you guys please take? Thanks.
> Clean up redundant access modifiers from oozie interfaces
> -
>
> Key: OOZIE-2657
> URL: https://issues.apache.org/jira/browse/OOZIE-2657
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Minor
> Labels: code-cleanup
> Attachments: OOZIE-2657-00.patch
>
>
> By default java {{interface}} fields are {{public static final}}. Some of the
> Oozie interface still explicitly declare these redundant access modifier. I
> think we can clean it.
> Similarly, methods declared in the interface are {{public}}. This redundant
> declaration can be cleaned up.
> {code}
> public interface RestConstants {
> public static final String VERSIONS = "versions";
> public static final String JOB = "job";
> ...
> }
> public interface JsonTags {
> public static final String OOZIE_SAFE_MODE = "safeMode"; //Applicable for
> V0 only
> public static final String OOZIE_SYSTEM_MODE = "systemMode";
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2657) Clean up redundant access modifiers from oozie interfaces
[
https://issues.apache.org/jira/browse/OOZIE-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna updated OOZIE-2657:
--
Attachment: OOZIE-2657-00.patch
> Clean up redundant access modifiers from oozie interfaces
> -
>
> Key: OOZIE-2657
> URL: https://issues.apache.org/jira/browse/OOZIE-2657
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Minor
> Labels: code-cleanup
> Attachments: OOZIE-2657-00.patch
>
>
> By default java {{interface}} fields are {{public static final}}. Some of the
> Oozie interface still explicitly declare these redundant access modifier. I
> think we can clean it.
> Similarly, methods declared in the interface are {{public}}. This redundant
> declaration can be cleaned up.
> {code}
> public interface RestConstants {
> public static final String VERSIONS = "versions";
> public static final String JOB = "job";
> ...
> }
> public interface JsonTags {
> public static final String OOZIE_SAFE_MODE = "safeMode"; //Applicable for
> V0 only
> public static final String OOZIE_SYSTEM_MODE = "systemMode";
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OOZIE-2657) Clean up redundant access modifiers from oozie interfaces
Abhishek Bafna created OOZIE-2657:
-
Summary: Clean up redundant access modifiers from oozie interfaces
Key: OOZIE-2657
URL: https://issues.apache.org/jira/browse/OOZIE-2657
Project: Oozie
Issue Type: Bug
Components: core
Reporter: Abhishek Bafna
Assignee: Abhishek Bafna
Priority: Minor
By default java {{interface}} fields are {{public static final}}. Some of the
Oozie interface still explicitly declare these redundant access modifier. I
think we can clean it.
Similarly, methods declared in the interface are {{public}}. This redundant
declaration can be cleaned up.
{code}
public interface RestConstants {
public static final String VERSIONS = "versions";
public static final String JOB = "job";
...
}
public interface JsonTags {
public static final String OOZIE_SAFE_MODE = "safeMode"; //Applicable for
V0 only
public static final String OOZIE_SYSTEM_MODE = "systemMode";
...
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2041) Add an admin command to run the PurgeXCommand
[
https://issues.apache.org/jira/browse/OOZIE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna updated OOZIE-2041:
--
Attachment: OOZIE-2041-00.patch
> Add an admin command to run the PurgeXCommand
> -
>
> Key: OOZIE-2041
> URL: https://issues.apache.org/jira/browse/OOZIE-2041
> Project: Oozie
> Issue Type: New Feature
> Components: core
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Abhishek Bafna
> Attachments: OOZIE-2041-00.patch
>
>
> Some users may find it useful to be able to run the PurgeXCommand on-demand.
> We can add a new admin command to run this. e.g.
> {noformat}
> oozie admin -purge
> {noformat}
> With no args, it can use the "older than" values from oozie-site, but we
> could have it take some arguments to override those for the command.
> Another thing to worry about is making sure that we don't run this if either
> it's already running (i.e. the user sent it twice) or if the scheduled
> PurgeService is already running it. On top of that, we may need extra
> consideration for HA setups where we currently only run it on the leader.
> We should probably also have a to not schedule the PurgeService in ooize-site
> for users who only want to run it manually.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2041) Add an admin command to run the PurgeXCommand
[
https://issues.apache.org/jira/browse/OOZIE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438942#comment-15438942
]
Abhishek Bafna commented on OOZIE-2041:
---
By default {{PurgeService}} executes every hour to clean up old oozie
workflows, coordinators and bundles from Oozie DB. With the new {{oozie admin
-purge wf=1\;coord=2\;bundle=3\;limit=100\;oldCoordAction=true/false>}}
command, users (admin users) can execute {{PurgeService}} on demand.
# Allow users to specify older than values for (workflow, coordinator and
bundle) in {{days}}, limit and turn {{on/off}} for old coordinator actions. At
least one parameter needs to be specified.
# If user does not specify one of parameter than {{PurgeService}} will use the
default value of it from {{oozie-default.xml}} or {{oozie-site.xml}}.
# The execution of the Purge Command will require a lock for execution, so at
the same time two {{PurgeXCommand}} will not execute. In the case, when lock is
not acquired, it will queue the command.
# Currently purge command is implemented as blocking command. Users will need
to wait until command execution completes.
# In the HA mode, purge command can be executed only on the leader Oozie server.
# purge command can be enabled/disabled using
{{oozie.service.PurgeService.enable.command.line}} property. By default it is
enabled.
> Add an admin command to run the PurgeXCommand
> -
>
> Key: OOZIE-2041
> URL: https://issues.apache.org/jira/browse/OOZIE-2041
> Project: Oozie
> Issue Type: New Feature
> Components: core
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Abhishek Bafna
>
> Some users may find it useful to be able to run the PurgeXCommand on-demand.
> We can add a new admin command to run this. e.g.
> {noformat}
> oozie admin -purge
> {noformat}
> With no args, it can use the "older than" values from oozie-site, but we
> could have it take some arguments to override those for the command.
> Another thing to worry about is making sure that we don't run this if either
> it's already running (i.e. the user sent it twice) or if the scheduled
> PurgeService is already running it. On top of that, we may need extra
> consideration for HA setups where we currently only run it on the leader.
> We should probably also have a to not schedule the PurgeService in ooize-site
> for users who only want to run it manually.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2525) SchemaChecker fails with NPE
[
https://issues.apache.org/jira/browse/OOZIE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438537#comment-15438537
]
Abhishek Bafna commented on OOZIE-2525:
---
+1 {{non-binding}}.
> SchemaChecker fails with NPE
>
>
> Key: OOZIE-2525
> URL: https://issues.apache.org/jira/browse/OOZIE-2525
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Robert Kanter
> Fix For: 4.3.0
>
> Attachments: OOZIE-2525.001.patch
>
>
> {code}
> 2016-05-03 14:38:03,851 WARN SchedulerService:523 [pool-1-thread-2] -
> SERVER[-] Error executing runnable [SchemaCheckerRunnable], null
> java.lang.NullPointerException
> at
> org.apache.oozie.service.SchemaCheckerService.access$002(SchemaCheckerService.java:29)
> at
> org.apache.oozie.service.SchemaCheckerService$SchemaCheckerRunnable.run(SchemaCheckerService.java:120)
> at
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It;s trying to use SchemaCheckerService before it is initialized.
> {code:title=SchemaCheckerService.java}
> Services.get().get(SchemaCheckerService.class).status = "DISABLED (not leader
> in HA)";
> Services.get().get(SchemaCheckerService.class).lastCheck =
> "N/A";
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OOZIE-2041) Add an admin command to run the PurgeXCommand
[
https://issues.apache.org/jira/browse/OOZIE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna reassigned OOZIE-2041:
-
Assignee: Abhishek Bafna
> Add an admin command to run the PurgeXCommand
> -
>
> Key: OOZIE-2041
> URL: https://issues.apache.org/jira/browse/OOZIE-2041
> Project: Oozie
> Issue Type: New Feature
> Components: core
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Abhishek Bafna
>
> Some users may find it useful to be able to run the PurgeXCommand on-demand.
> We can add a new admin command to run this. e.g.
> {noformat}
> oozie admin -purge
> {noformat}
> With no args, it can use the "older than" values from oozie-site, but we
> could have it take some arguments to override those for the command.
> Another thing to worry about is making sure that we don't run this if either
> it's already running (i.e. the user sent it twice) or if the scheduled
> PurgeService is already running it. On top of that, we may need extra
> consideration for HA setups where we currently only run it on the leader.
> We should probably also have a to not schedule the PurgeService in ooize-site
> for users who only want to run it manually.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2041) Add an admin command to run the PurgeXCommand
[
https://issues.apache.org/jira/browse/OOZIE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436684#comment-15436684
]
Abhishek Bafna commented on OOZIE-2041:
---
[~rkanter] Can I take up this issue?
> Add an admin command to run the PurgeXCommand
> -
>
> Key: OOZIE-2041
> URL: https://issues.apache.org/jira/browse/OOZIE-2041
> Project: Oozie
> Issue Type: New Feature
> Components: core
>Affects Versions: trunk
>Reporter: Robert Kanter
>
> Some users may find it useful to be able to run the PurgeXCommand on-demand.
> We can add a new admin command to run this. e.g.
> {noformat}
> oozie admin -purge
> {noformat}
> With no args, it can use the "older than" values from oozie-site, but we
> could have it take some arguments to override those for the command.
> Another thing to worry about is making sure that we don't run this if either
> it's already running (i.e. the user sent it twice) or if the scheduled
> PurgeService is already running it. On top of that, we may need extra
> consideration for HA setups where we currently only run it on the leader.
> We should probably also have a to not schedule the PurgeService in ooize-site
> for users who only want to run it manually.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2650) Retry coord start on database exceptions
[
https://issues.apache.org/jira/browse/OOZIE-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436646#comment-15436646
]
Abhishek Bafna commented on OOZIE-2650:
---
I think we can define a new error code instead of using {{E1304, "Duplicate
bundle application coordinator name"}}
{code}
@Override
protected void verifyPrecondition() throws CommandException {
super.verifyPrecondition();
if (wfId != null) {
LOG.warn("Workflow is already submitted for Coord Action [{0}]",
getParentId());
throw new CommandException(ErrorCode.E1304, getParentId());
}
}
{code}
> Retry coord start on database exceptions
>
>
> Key: OOZIE-2650
> URL: https://issues.apache.org/jira/browse/OOZIE-2650
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Attachments: OOZIE-2650-1.patch
>
>
> {code:title=CoordActionStartXCommand.java}
> updateList.add(new UpdateEntry(
> WorkflowJobQuery.UPDATE_WORKFLOW_PARENT_MODIFIED,
> wfJob));
> updateList.add(new UpdateEntry(
> CoordActionQuery.UPDATE_COORD_ACTION_FOR_START,
> coordAction));
> try {
> executor.executeBatchInsertUpdateDelete(insertList,
> updateList, null);
> queue(new
> CoordActionNotificationXCommand(coordAction), 100);
> if (EventHandlerService.isEnabled()) {
> generateEvent(coordAction, user, appName,
> wfJob.getStartTime());
> }
> }
> catch (JPAExecutorException je) {
> throw new CommandException(je);
> }
> .
> ...
>
> finally {
> if (makeFail == true) { // No DB exception occurs
>
>
>
> queue(new
> CoordActionReadyXCommand(coordAction.getJobId()));
> }
> }
> {code}
> If there is any Database issue while starting coord action, we fail the coord
> action. We should retry.
> CoordActionStartXCommand submits workflow. Workflow gets linked to the coord
> action if workflow submission succeeds. But if coord action update fails due
> to database exception, recovery service should be able to recover it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2617) Read properties files in action configurations
[
https://issues.apache.org/jira/browse/OOZIE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436618#comment-15436618
]
Abhishek Bafna commented on OOZIE-2617:
---
+1 {{non-binding}}.
> Read properties files in action configurations
> --
>
> Key: OOZIE-2617
> URL: https://issues.apache.org/jira/browse/OOZIE-2617
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: trunk
>Reporter: Taklon Stephen Wu
> Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2617-1.patch, OOZIE-2617-2.patch,
> OOZIE-2617-3.patch
>
>
> Current logic of acton configuration only READ xml files within Oozie action
> directory, but for pig and other applications, properties files are supported
> and may be default configuration file format. To simplify the logic of
> reusing these properties files when using Oozie with them, we add properties
> file reading in action conf.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2517) Add support for startCreatedTime and endCreatedTime filters for coord and bundles
[
https://issues.apache.org/jira/browse/OOZIE-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15434162#comment-15434162
]
Abhishek Bafna commented on OOZIE-2517:
---
Thanks [~jaydeepvishwakarma] for review and committing.
> Add support for startCreatedTime and endCreatedTime filters for coord and
> bundles
> -
>
> Key: OOZIE-2517
> URL: https://issues.apache.org/jira/browse/OOZIE-2517
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2517-001.patch, OOZIE-2517-003.patch,
> OOZIE-2517-005.patch, OOZIE-2517-007.patch, OOZIE-2517-008.patch,
> OOZIE-2517-009.patch, OOZIE-2517-010.patch
>
>
> Support for the 'startCreatedTime' and 'endCreatedTime' is not implemented
> for the coordinators and bundle job.
> Curl command with jobtype coordinator and bundle fails where as jobtype=wf
> executes successfully.
> https://oozie.apache.org/docs/4.2.0/WebServicesAPI.html#Jobs_Information
> {noformat}
> curl
> "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=bundle";
> curl
> "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=coordinator";
> curl
> "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=wf";
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2517) Add support for startCreatedTime and endCreatedTime filters for coord and bundles
[
https://issues.apache.org/jira/browse/OOZIE-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15432689#comment-15432689
]
Abhishek Bafna commented on OOZIE-2517:
---
[~jaydeepvishwakarma] Ping.
> Add support for startCreatedTime and endCreatedTime filters for coord and
> bundles
> -
>
> Key: OOZIE-2517
> URL: https://issues.apache.org/jira/browse/OOZIE-2517
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2517-001.patch, OOZIE-2517-003.patch,
> OOZIE-2517-005.patch, OOZIE-2517-007.patch, OOZIE-2517-008.patch,
> OOZIE-2517-009.patch, OOZIE-2517-010.patch
>
>
> Support for the 'startCreatedTime' and 'endCreatedTime' is not implemented
> for the coordinators and bundle job.
> Curl command with jobtype coordinator and bundle fails where as jobtype=wf
> executes successfully.
> https://oozie.apache.org/docs/4.2.0/WebServicesAPI.html#Jobs_Information
> {noformat}
> curl
> "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=bundle";
> curl
> "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=coordinator";
> curl
> "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=wf";
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0
[
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426216#comment-15426216
]
Abhishek Bafna commented on OOZIE-2613:
---
I am not able to get these two testcases working. I can use some help/feedback
from the fellow developers in the community.
{noformat}
org.apache.oozie.action.hadoop.TestJavaActionExecutor.testACLModifyJob
org.apache.oozie.action.hadoop.TestHiveMain.testMain
{noformat}
{{TestJavaActionExecutor.testACLModifyJob}} is failing because FileSystem is
initialized with the {{test}} user and launcher job is launched with
{{user.name}} (due to {{XHCatTestCase#getTestUser}}, added into this patch).
The Hive testcase works fine with hadoop-2 profile, but fails with hadoop-1
profile.
{noformat}
2016-08-18 14:43:47,507 [main] ERROR org.apache.hadoop.hive.ql.exec.Task -
Failed with exception Unable to rename:
hdfs://localhost:56649/user/test/95bd1a4c-2629-47bd-a347-f3901b757832/output/.hive-staging_hive_2016-08-18_14-41-58_397_1680638062022362363-1/-ext-1
to: /user/test/95bd1a4c-2629-47bd-a347-f3901b757832/output
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename:
hdfs://localhost:56649/user/test/95bd1a4c-2629-47bd-a347-f3901b757832/output/.hive-staging_hive_2016-08-18_14-41-58_397_1680638062022362363-1/-ext-1
to: /user/test/95bd1a4c-2629-47bd-a347-f3901b757832/output
at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:114)
at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:222)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1650)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1409)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1192)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:311)
at
org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:409)
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:425)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:334)
at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:311)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:49)
at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:69)
at
org.apache.oozie.action.hadoop.TestHiveMain.call(TestHiveMain.java:138)
at
org.apache.oozie.action.hadoop.TestHiveMain.call(TestHiveMain.java:34)
at
org.apache.oozie.action.hadoop.MainTestCase$1.run(MainTestCase.java:33)
at
org.apache.oozie.action.hadoop.MainTestCase$1.run(MainTestCase.java:31)
at java.security.AccessController.doPrivileged(Native Method)
{noformat}
For the hadoop-1, would it be a good idea to keep the hive version same as
{{hive-0.13.1}}.
cc: [~rkanter] [~jaydeepvishwakarma] [~puru] [~rohini] [~asasvari] [~gezapeti]
[~satishsaley]
> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch,
> OOZIE-2613-02.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2231) Upgrade curator to latest version 2.11.0
[
https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna updated OOZIE-2231:
--
Attachment: OOZIE-2231-03.patch
[~rkanter], Uploading a patch with {{curator-2.11.0}} and {{guava-16.0.1}}.
1. The {{Stopwatch#elapsedMillis()}} is not available in guava-16. Updated
Stopwatch initialization instead of a deprecated constructor to static method
{{Stopwatch.createStarted()}}.
2. Indentation fixed. Refactored the indentation for property section of the
main {{pom.xml}}.
3. Defined a global {{}}.
4. This should not be a problem. I was able to execute Oozie example workflows
without any issue. This is how lib's look like with patch.
{noformat}
$find . -name guava*
./share/lib/hive/guava-16.0.1.jar
./share/lib/spark/guava-14.0.1.jar
./share/lib/pig/guava-11.0.2.jar
./oozie-client/lib/guava-16.0.1.jar
./lib/guava-16.0.1.jar
./oozie-server/webapps/oozie/WEB-INF/lib/guava-16.0.1.jar
{noformat}
[~puru], [~rohini] Please have a look and let me know your feedback. Thanks.
> Upgrade curator to latest version 2.11.0
>
>
> Key: OOZIE-2231
> URL: https://issues.apache.org/jira/browse/OOZIE-2231
> Project: Oozie
> Issue Type: Bug
> Components: HA
>Reporter: Purshotam Shah
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch,
> OOZIE-2231-02.patch, OOZIE-2231-02.patch, OOZIE-2231-03.patch
>
>
> It have some fix related to InterProcessReadWriteLock, ChildReaper,
> LeaderSelector which we use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2231) Upgrade curator to latest version 2.11.0
[
https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424932#comment-15424932
]
Abhishek Bafna commented on OOZIE-2231:
---
[~rkanter] I think we need to bring in guava-16, because curator-2.11.0 uses
it. Otherwise many ZK related testcases fails with following exception: (some
other errors)
{noformat}
java.lang.NoSuchMethodError:
com.google.common.collect.Sets.newConcurrentHashSet()Ljava/util/Set;
at
org.apache.curator.framework.recipes.locks.ChildReaper.(ChildReaper.java:63)
at
org.apache.curator.framework.recipes.locks.ChildReaper.(ChildReaper.java:124)
at org.apache.oozie.service.ZKLocksService.init(ZKLocksService.java:71)
at
org.apache.oozie.service.TestZKLocksService.testRegisterUnregister(TestZKLocksService.java:47)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
{noformat}
I will be prepare a patch and testing the UTs and all the examples which we
bundle with Oozie for this. Let me know if this sounds good to you. If there is
any other testing I should be doing, please let me know. Thanks.
> Upgrade curator to latest version 2.11.0
>
>
> Key: OOZIE-2231
> URL: https://issues.apache.org/jira/browse/OOZIE-2231
> Project: Oozie
> Issue Type: Bug
> Components: HA
>Reporter: Purshotam Shah
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch,
> OOZIE-2231-02.patch, OOZIE-2231-02.patch
>
>
> It have some fix related to InterProcessReadWriteLock, ChildReaper,
> LeaderSelector which we use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2231) Upgrade curator to latest version 2.11.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424318#comment-15424318 ] Abhishek Bafna commented on OOZIE-2231: --- I will upload a correct patch soon. [~rkanter] Please ignore my previous comment. Thanks. > Upgrade curator to latest version 2.11.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch, OOZIE-2231-02.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0
[ https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Bafna updated OOZIE-2613: -- Attachment: OOZIE-2613-02.patch > Upgrade hive version from 0.13.1 to 1.2.0 > - > > Key: OOZIE-2613 > URL: https://issues.apache.org/jira/browse/OOZIE-2613 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch, > OOZIE-2613-02.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2231) Upgrade curator to latest version 2.11.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Bafna updated OOZIE-2231: -- Attachment: OOZIE-2231-02.patch > Upgrade curator to latest version 2.11.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch, OOZIE-2231-02.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2231) Upgrade curator to latest version 2.11.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Bafna updated OOZIE-2231: -- Summary: Upgrade curator to latest version 2.11.0 (was: Upgrade curator to latest version 2.8.0) > Upgrade curator to latest version 2.11.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2231) Upgrade curator to latest version 2.8.0
[
https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna updated OOZIE-2231:
--
Attachment: OOZIE-2231-02.patch
Thanks [~rkanter] for the feedback.
I have incorporated all the points you have mentioned. (We don't need to use
Guava 16).
I have done a bit of refactoring for properties defined in the main {{pom.xml}}
> Upgrade curator to latest version 2.8.0
> ---
>
> Key: OOZIE-2231
> URL: https://issues.apache.org/jira/browse/OOZIE-2231
> Project: Oozie
> Issue Type: Bug
> Components: HA
>Reporter: Purshotam Shah
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch,
> OOZIE-2231-02.patch
>
>
> It have some fix related to InterProcessReadWriteLock, ChildReaper,
> LeaderSelector which we use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2544) Typo -- but it appears on the main Google page for the site
[
https://issues.apache.org/jira/browse/OOZIE-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423957#comment-15423957
]
Abhishek Bafna commented on OOZIE-2544:
---
Thanks [~walt] for reporting the issue.
Thanks [~rkanter] for review and publishing it.
> Typo -- but it appears on the main Google page for the site
> ---
>
> Key: OOZIE-2544
> URL: https://issues.apache.org/jira/browse/OOZIE-2544
> Project: Oozie
> Issue Type: Bug
> Components: site
>Affects Versions: trunk
> Environment: Web site
>Reporter: Walt Stoneburner
>Assignee: Abhishek Bafna
>Priority: Trivial
> Attachments: OOZIE-2544.patch
>
> Original Estimate: 2m
> Remaining Estimate: 2m
>
> There is a typo on the main http://oozie.apache.org page that describes the
> product. In fact, if one asks Google about Oozie, it is in the main banner
> that Google supplies about a product or website.
> {quote}
> Index: content/apt/index.apt.vm
> ===
> --- content/apt/index.apt.vm (revision 1746160)
> +++ content/apt/index.apt.vm (working copy)
> @@ -23,7 +23,7 @@
>
>Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions.
>
> - Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availabilty.
> + Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availability.
>
>Oozie is integrated with the rest of the Hadoop stack supporting several
> types of Hadoop jobs out of the box
>(such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop and
> Distcp) as well as system specific jobs
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OOZIE-1174) Upgrade to antlr 3.4 when we switch to Hive 0.11.0
[ https://issues.apache.org/jira/browse/OOZIE-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Bafna resolved OOZIE-1174. --- Resolution: Done > Upgrade to antlr 3.4 when we switch to Hive 0.11.0 > -- > > Key: OOZIE-1174 > URL: https://issues.apache.org/jira/browse/OOZIE-1174 > Project: Oozie > Issue Type: Improvement > Components: tests >Reporter: Robert Kanter > > In Hive 0.11.0, they're upgrading to antlr 3.4 so we shouldn't need to use > antlr 3.0.1 anymore. > (This may not be for a while, but just so we don't forget) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-1174) Upgrade to antlr 3.4 when we switch to Hive 0.11.0
[
https://issues.apache.org/jira/browse/OOZIE-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422680#comment-15422680
]
Abhishek Bafna commented on OOZIE-1174:
---
We are moved to {{antlr-3.4}}.
Main pom.xml
https://github.com/apache/oozie/blob/master/pom.xml#L1302-L1306
Core Module pom.xml:
https://github.com/apache/oozie/blob/master/core/pom.xml#L287-L292
Closing the issue. Please re-open if required. Thanks.
> Upgrade to antlr 3.4 when we switch to Hive 0.11.0
> --
>
> Key: OOZIE-1174
> URL: https://issues.apache.org/jira/browse/OOZIE-1174
> Project: Oozie
> Issue Type: Improvement
> Components: tests
>Reporter: Robert Kanter
>
> In Hive 0.11.0, they're upgrading to antlr 3.4 so we shouldn't need to use
> antlr 3.0.1 anymore.
> (This may not be for a while, but just so we don't forget)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2632) Provide database dump/load functionality to make database migration easier
[ https://issues.apache.org/jira/browse/OOZIE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422631#comment-15422631 ] Abhishek Bafna commented on OOZIE-2632: --- [~gezapeti] Can you please upload the patch to RB and add the link to Jira. Thanks. > Provide database dump/load functionality to make database migration easier > -- > > Key: OOZIE-2632 > URL: https://issues.apache.org/jira/browse/OOZIE-2632 > Project: Oozie > Issue Type: New Feature >Reporter: Peter Cseh >Assignee: Peter Cseh > Attachments: OOZIE-2632.001.patch, OOZIE-2632.002.patch, > OOZIE-2632.003.patch > > > When there is a need to change the database under Oozie one has to submit all > the workflows again or migrate the data manually which is not easy because of > the binary blobs stored in the DB. > oozie-setup.sh could have a db export/import command that would save the > contents of the Db to a file and read it back from it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2544) Typo -- but it appears on the main Google page for the site
[
https://issues.apache.org/jira/browse/OOZIE-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna updated OOZIE-2544:
--
Attachment: OOZIE-2544.patch
> Typo -- but it appears on the main Google page for the site
> ---
>
> Key: OOZIE-2544
> URL: https://issues.apache.org/jira/browse/OOZIE-2544
> Project: Oozie
> Issue Type: Bug
> Components: site
>Affects Versions: trunk
> Environment: Web site
>Reporter: Walt Stoneburner
>Assignee: Abhishek Bafna
>Priority: Trivial
> Attachments: OOZIE-2544.patch
>
> Original Estimate: 2m
> Remaining Estimate: 2m
>
> There is a typo on the main http://oozie.apache.org page that describes the
> product. In fact, if one asks Google about Oozie, it is in the main banner
> that Google supplies about a product or website.
> {quote}
> Index: content/apt/index.apt.vm
> ===
> --- content/apt/index.apt.vm (revision 1746160)
> +++ content/apt/index.apt.vm (working copy)
> @@ -23,7 +23,7 @@
>
>Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions.
>
> - Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availabilty.
> + Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availability.
>
>Oozie is integrated with the rest of the Hadoop stack supporting several
> types of Hadoop jobs out of the box
>(such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop and
> Distcp) as well as system specific jobs
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2544) Typo -- but it appears on the main Google page for the site
[
https://issues.apache.org/jira/browse/OOZIE-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna updated OOZIE-2544:
--
Attachment: (was: OOZIE-2544.patch)
> Typo -- but it appears on the main Google page for the site
> ---
>
> Key: OOZIE-2544
> URL: https://issues.apache.org/jira/browse/OOZIE-2544
> Project: Oozie
> Issue Type: Bug
> Components: site
>Affects Versions: trunk
> Environment: Web site
>Reporter: Walt Stoneburner
>Assignee: Abhishek Bafna
>Priority: Trivial
> Original Estimate: 2m
> Remaining Estimate: 2m
>
> There is a typo on the main http://oozie.apache.org page that describes the
> product. In fact, if one asks Google about Oozie, it is in the main banner
> that Google supplies about a product or website.
> {quote}
> Index: content/apt/index.apt.vm
> ===
> --- content/apt/index.apt.vm (revision 1746160)
> +++ content/apt/index.apt.vm (working copy)
> @@ -23,7 +23,7 @@
>
>Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions.
>
> - Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availabilty.
> + Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availability.
>
>Oozie is integrated with the rest of the Hadoop stack supporting several
> types of Hadoop jobs out of the box
>(such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop and
> Distcp) as well as system specific jobs
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2544) Typo -- but it appears on the main Google page for the site
[
https://issues.apache.org/jira/browse/OOZIE-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422554#comment-15422554
]
Abhishek Bafna commented on OOZIE-2544:
---
[~jaydeepvishwakarma], [~rkanter] Can you please take a look at the patch.
The patch fixes the typo into the oozie site home page. http://oozie.apache.org/
I have not marked the {{Submit Patch}} as it is not related to the Oozie code.
Please advice if any other approach is required. Thanks.
> Typo -- but it appears on the main Google page for the site
> ---
>
> Key: OOZIE-2544
> URL: https://issues.apache.org/jira/browse/OOZIE-2544
> Project: Oozie
> Issue Type: Bug
> Components: site
>Affects Versions: trunk
> Environment: Web site
>Reporter: Walt Stoneburner
>Assignee: Abhishek Bafna
>Priority: Trivial
> Original Estimate: 2m
> Remaining Estimate: 2m
>
> There is a typo on the main http://oozie.apache.org page that describes the
> product. In fact, if one asks Google about Oozie, it is in the main banner
> that Google supplies about a product or website.
> {quote}
> Index: content/apt/index.apt.vm
> ===
> --- content/apt/index.apt.vm (revision 1746160)
> +++ content/apt/index.apt.vm (working copy)
> @@ -23,7 +23,7 @@
>
>Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions.
>
> - Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availabilty.
> + Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availability.
>
>Oozie is integrated with the rest of the Hadoop stack supporting several
> types of Hadoop jobs out of the box
>(such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop and
> Distcp) as well as system specific jobs
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2544) Typo -- but it appears on the main Google page for the site
[
https://issues.apache.org/jira/browse/OOZIE-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna updated OOZIE-2544:
--
Attachment: OOZIE-2544.patch
> Typo -- but it appears on the main Google page for the site
> ---
>
> Key: OOZIE-2544
> URL: https://issues.apache.org/jira/browse/OOZIE-2544
> Project: Oozie
> Issue Type: Bug
> Components: site
>Affects Versions: trunk
> Environment: Web site
>Reporter: Walt Stoneburner
>Priority: Trivial
> Attachments: OOZIE-2544.patch
>
> Original Estimate: 2m
> Remaining Estimate: 2m
>
> There is a typo on the main http://oozie.apache.org page that describes the
> product. In fact, if one asks Google about Oozie, it is in the main banner
> that Google supplies about a product or website.
> {quote}
> Index: content/apt/index.apt.vm
> ===
> --- content/apt/index.apt.vm (revision 1746160)
> +++ content/apt/index.apt.vm (working copy)
> @@ -23,7 +23,7 @@
>
>Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions.
>
> - Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availabilty.
> + Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availability.
>
>Oozie is integrated with the rest of the Hadoop stack supporting several
> types of Hadoop jobs out of the box
>(such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop and
> Distcp) as well as system specific jobs
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OOZIE-2544) Typo -- but it appears on the main Google page for the site
[
https://issues.apache.org/jira/browse/OOZIE-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna reassigned OOZIE-2544:
-
Assignee: Abhishek Bafna
> Typo -- but it appears on the main Google page for the site
> ---
>
> Key: OOZIE-2544
> URL: https://issues.apache.org/jira/browse/OOZIE-2544
> Project: Oozie
> Issue Type: Bug
> Components: site
>Affects Versions: trunk
> Environment: Web site
>Reporter: Walt Stoneburner
>Assignee: Abhishek Bafna
>Priority: Trivial
> Attachments: OOZIE-2544.patch
>
> Original Estimate: 2m
> Remaining Estimate: 2m
>
> There is a typo on the main http://oozie.apache.org page that describes the
> product. In fact, if one asks Google about Oozie, it is in the main banner
> that Google supplies about a product or website.
> {quote}
> Index: content/apt/index.apt.vm
> ===
> --- content/apt/index.apt.vm (revision 1746160)
> +++ content/apt/index.apt.vm (working copy)
> @@ -23,7 +23,7 @@
>
>Oozie Workflow jobs are Directed Acyclical Graphs (DAGs) of actions.
>
> - Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availabilty.
> + Oozie Coordinator jobs are recurrent Oozie Workflow jobs triggered by time
> (frequency) and data availability.
>
>Oozie is integrated with the rest of the Hadoop stack supporting several
> types of Hadoop jobs out of the box
>(such as Java map-reduce, Streaming map-reduce, Pig, Hive, Sqoop and
> Distcp) as well as system specific jobs
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0
[ https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Bafna updated OOZIE-2613: -- Attachment: OOZIE-2613-01.patch > Upgrade hive version from 0.13.1 to 1.2.0 > - > > Key: OOZIE-2613 > URL: https://issues.apache.org/jira/browse/OOZIE-2613 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: Abhishek Bafna >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2613-00.patch, OOZIE-2613-01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-1793) Improve find bugs reporting for Oozie
[
https://issues.apache.org/jira/browse/OOZIE-1793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna updated OOZIE-1793:
--
Attachment: OOZIE-1793-01.patch
Attaching the rebase patch.
verified the patch with {{mvn clean verify -DskipTests}}.
+1 {{non-binding}}.
> Improve find bugs reporting for Oozie
> -
>
> Key: OOZIE-1793
> URL: https://issues.apache.org/jira/browse/OOZIE-1793
> Project: Oozie
> Issue Type: Sub-task
> Components: build
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Fix For: 4.3.0
>
> Attachments: OOZIE-1793-01.patch, OOZIE-1793.patch, OOZIE-1793.patch
>
>
> I couldn't figure out how to run findbugs against Oozie. And even if it does
> work, it looks like it's just going to create an XML file.
> Based on some minor refactoring to this
> [tutorial|http://www.petrikainulainen.net/programming/maven/findbugs-maven-plugin-tutorial/]
> and this [stack overflow
> message|http://stackoverflow.com/questions/7035112/any-easy-way-to-generate-a-findbug-html-report-from-maven-without-sitesite/10365954#10365954],
> I was able to improve our findbugs reporting. Now, when you run {{mvn
> verify}} (which we use to generate a checkstyle report), it will also run
> findbugs and transform the resulting XML files into html files that are human
> readable.
> The hard part here will be actually going through the reports and fixing
> these bugs; but that's for other JIRAs :)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2036) Drop support for Java 1.6
[ https://issues.apache.org/jira/browse/OOZIE-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418536#comment-15418536 ] Abhishek Bafna commented on OOZIE-2036: --- Thanks [~jaydeepvishwakarma] for review and committing it. > Drop support for Java 1.6 > - > > Key: OOZIE-2036 > URL: https://issues.apache.org/jira/browse/OOZIE-2036 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Peter Cseh > Fix For: 4.3.0 > > Attachments: OOZIE-2036-001.patch, OOZIE-2036-02.patch > > > Hadoop is planning on dropping support for JDK 6, which is EoL, with Hadoop > 2.7. We should probably do the same in Oozie trunk. > This task involves: > - Only allow building on JDK 7+ > - Making the build target Java 7 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OOZIE-2525) SchemaChecker fails with NPE
[
https://issues.apache.org/jira/browse/OOZIE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418398#comment-15418398
]
Abhishek Bafna edited comment on OOZIE-2525 at 8/12/16 5:58 AM:
Thanks [~rkanter] for the patch. Marking {{submit patch}} for issue, so jenkins
can run and get included into the 4.3.0 release.
was (Author: abhishekbafna):
Thanks [~rkanter] for the patch. Marking submitting the patch for issue, so
jenkins can run and get included into the 4.3.0 release.
> SchemaChecker fails with NPE
>
>
> Key: OOZIE-2525
> URL: https://issues.apache.org/jira/browse/OOZIE-2525
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Robert Kanter
> Fix For: 4.3.0
>
> Attachments: OOZIE-2525.001.patch
>
>
> {code}
> 2016-05-03 14:38:03,851 WARN SchedulerService:523 [pool-1-thread-2] -
> SERVER[-] Error executing runnable [SchemaCheckerRunnable], null
> java.lang.NullPointerException
> at
> org.apache.oozie.service.SchemaCheckerService.access$002(SchemaCheckerService.java:29)
> at
> org.apache.oozie.service.SchemaCheckerService$SchemaCheckerRunnable.run(SchemaCheckerService.java:120)
> at
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It;s trying to use SchemaCheckerService before it is initialized.
> {code:title=SchemaCheckerService.java}
> Services.get().get(SchemaCheckerService.class).status = "DISABLED (not leader
> in HA)";
> Services.get().get(SchemaCheckerService.class).lastCheck =
> "N/A";
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2525) SchemaChecker fails with NPE
[
https://issues.apache.org/jira/browse/OOZIE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418398#comment-15418398
]
Abhishek Bafna commented on OOZIE-2525:
---
Thanks [~rkanter] for the patch. Marking submitting the patch for issue, so
jenkins can run and get included into the 4.3.0 release.
> SchemaChecker fails with NPE
>
>
> Key: OOZIE-2525
> URL: https://issues.apache.org/jira/browse/OOZIE-2525
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Robert Kanter
> Fix For: 4.3.0
>
> Attachments: OOZIE-2525.001.patch
>
>
> {code}
> 2016-05-03 14:38:03,851 WARN SchedulerService:523 [pool-1-thread-2] -
> SERVER[-] Error executing runnable [SchemaCheckerRunnable], null
> java.lang.NullPointerException
> at
> org.apache.oozie.service.SchemaCheckerService.access$002(SchemaCheckerService.java:29)
> at
> org.apache.oozie.service.SchemaCheckerService$SchemaCheckerRunnable.run(SchemaCheckerService.java:120)
> at
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It;s trying to use SchemaCheckerService before it is initialized.
> {code:title=SchemaCheckerService.java}
> Services.get().get(SchemaCheckerService.class).status = "DISABLED (not leader
> in HA)";
> Services.get().get(SchemaCheckerService.class).lastCheck =
> "N/A";
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2231) Upgrade curator to latest version 2.8.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15417705#comment-15417705 ] Abhishek Bafna commented on OOZIE-2231: --- failed testcase are not relevant, passes locally. Thanks. > Upgrade curator to latest version 2.8.0 > --- > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2231) Upgrade curator to latest version 2.8.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15417360#comment-15417360 ] Abhishek Bafna commented on OOZIE-2231: --- [~rkanter] Sure. I will look into it and do it. Thanks. > Upgrade curator to latest version 2.8.0 > --- > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Assignee: Abhishek Bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2231) Upgrade curator to latest version 2.8.0
[
https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Bafna updated OOZIE-2231:
--
Attachment: OOZIE-2231-01.patch
Keeping the same guava version for pig as {{pig-0.16}} is using {{guava-11.0}}.
> Upgrade curator to latest version 2.8.0
> ---
>
> Key: OOZIE-2231
> URL: https://issues.apache.org/jira/browse/OOZIE-2231
> Project: Oozie
> Issue Type: Bug
> Components: HA
>Reporter: Purshotam Shah
>Assignee: Abhishek Bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch
>
>
> It have some fix related to InterProcessReadWriteLock, ChildReaper,
> LeaderSelector which we use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2036) Drop support for Java 1.6
[ https://issues.apache.org/jira/browse/OOZIE-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15417087#comment-15417087 ] Abhishek Bafna commented on OOZIE-2036: --- testcase failure is intermittent. Passed locally. [~rkanter], [~rohini], [~puru], [~jaydeepvishwakarma] can this patch be reviewed and committed. So that we can start some of the other blocked issues (upgrades). Thanks. > Drop support for Java 1.6 > - > > Key: OOZIE-2036 > URL: https://issues.apache.org/jira/browse/OOZIE-2036 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Peter Cseh > Fix For: 4.3.0 > > Attachments: OOZIE-2036-001.patch, OOZIE-2036-02.patch > > > Hadoop is planning on dropping support for JDK 6, which is EoL, with Hadoop > 2.7. We should probably do the same in Oozie trunk. > This task involves: > - Only allow building on JDK 7+ > - Making the build target Java 7 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2488) Upgrade to latest OpenJPA version
[ https://issues.apache.org/jira/browse/OOZIE-2488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2488: -- Attachment: OOZIE-2488-00.patch > Upgrade to latest OpenJPA version > - > > Key: OOZIE-2488 > URL: https://issues.apache.org/jira/browse/OOZIE-2488 > Project: Oozie > Issue Type: Improvement > Components: core >Reporter: Robert Kanter > Fix For: 4.3.0 > > Attachments: OOZIE-2488-00.patch > > > Oozie is currently using OpenJPA 2.2.2, which is from April 2013. Version > 2.3.0 is from Nov 2013, but there was a recent 2.4.1 release in Feb 2016 > (that's a long gap!). Given that this is a critical dependency, it might be > a good idea to take a look at the chnagelog, do some testing, and update the > dependency. I'm sure there's lots of bug fixes that would be great to have. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OOZIE-2488) Upgrade to latest OpenJPA version
[ https://issues.apache.org/jira/browse/OOZIE-2488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna reassigned OOZIE-2488: - Assignee: abhishek bafna > Upgrade to latest OpenJPA version > - > > Key: OOZIE-2488 > URL: https://issues.apache.org/jira/browse/OOZIE-2488 > Project: Oozie > Issue Type: Improvement > Components: core >Reporter: Robert Kanter >Assignee: abhishek bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2488-00.patch > > > Oozie is currently using OpenJPA 2.2.2, which is from April 2013. Version > 2.3.0 is from Nov 2013, but there was a recent 2.4.1 release in Feb 2016 > (that's a long gap!). Given that this is a critical dependency, it might be > a good idea to take a look at the chnagelog, do some testing, and update the > dependency. I'm sure there's lots of bug fixes that would be great to have. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2231) Upgrade curator to latest version 2.8.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2231: -- Attachment: OOZIE-2231-00.patch Attaching patch with curator 2.11.0 and google guava 16.0.1 version. > Upgrade curator to latest version 2.8.0 > --- > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah > Fix For: 4.3.0 > > Attachments: OOZIE-2231-00.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2036) Drop support for Java 1.6
[ https://issues.apache.org/jira/browse/OOZIE-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2036: -- Attachment: OOZIE-2036-02.patch Attaching the rebase patch and added the properties for java version. > Drop support for Java 1.6 > - > > Key: OOZIE-2036 > URL: https://issues.apache.org/jira/browse/OOZIE-2036 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Peter Cseh > Fix For: 4.3.0 > > Attachments: OOZIE-2036-001.patch, OOZIE-2036-02.patch > > > Hadoop is planning on dropping support for JDK 6, which is EoL, with Hadoop > 2.7. We should probably do the same in Oozie trunk. > This task involves: > - Only allow building on JDK 7+ > - Making the build target Java 7 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2538) Update HttpClient versions to close security vulnerabilities
[ https://issues.apache.org/jira/browse/OOZIE-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15416873#comment-15416873 ] abhishek bafna commented on OOZIE-2538: --- Testcase failure is not relevant. [~jaydeepvishwakarma], [~rkanter] can you please take a look. > Update HttpClient versions to close security vulnerabilities > > > Key: OOZIE-2538 > URL: https://issues.apache.org/jira/browse/OOZIE-2538 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2538-01.patch, OOZIE-2538.patch > > > We learned that > https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-5262 : > http/conn/ssl/SSLConnectionSocketFactory.java in Apache HttpComponents > HttpClient before 4.3.6 ignores the http.socket.timeout configuration setting > during an SSL handshake, which allows remote attackers to cause a denial of > service (HTTPS call hang) via unspecified vectors. > Also, Commons HttpClient project is now end of life, and is no longer being > developed. It has been replaced by the Apache HttpComponents project in its > HttpClient and HttpCore modules, which offer better performance and more > flexibility. http://hc.apache.org/httpclient-3.x/ > Hence, HttpClient version should be updated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2525) SchemaChecker fails with NPE
[
https://issues.apache.org/jira/browse/OOZIE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15416511#comment-15416511
]
abhishek bafna commented on OOZIE-2525:
---
[~rkanter] Yes. we can do a {{!= null}} but then this service is configured to
run every week, so we can not skip it during the startup. Other way could be,
putting a hard coded {{sleep(seconds)}}, But I guess that will be same as
defining a initial delay.
In most of the cases we have defined a hard coded 10 second of initial delay.
Few services have configurable initial delay as well.
> SchemaChecker fails with NPE
>
>
> Key: OOZIE-2525
> URL: https://issues.apache.org/jira/browse/OOZIE-2525
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>
> {code}
> 2016-05-03 14:38:03,851 WARN SchedulerService:523 [pool-1-thread-2] -
> SERVER[-] Error executing runnable [SchemaCheckerRunnable], null
> java.lang.NullPointerException
> at
> org.apache.oozie.service.SchemaCheckerService.access$002(SchemaCheckerService.java:29)
> at
> org.apache.oozie.service.SchemaCheckerService$SchemaCheckerRunnable.run(SchemaCheckerService.java:120)
> at
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It;s trying to use SchemaCheckerService before it is initialized.
> {code:title=SchemaCheckerService.java}
> Services.get().get(SchemaCheckerService.class).status = "DISABLED (not leader
> in HA)";
> Services.get().get(SchemaCheckerService.class).lastCheck =
> "N/A";
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2628) Fix Workflow EL function return types in Documentation
[
https://issues.apache.org/jira/browse/OOZIE-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15416483#comment-15416483
]
abhishek bafna commented on OOZIE-2628:
---
Thanks [~jaydeepvishwakarma] for reviewing and committing.
> Fix Workflow EL function return types in Documentation
> --
>
> Key: OOZIE-2628
> URL: https://issues.apache.org/jira/browse/OOZIE-2628
> Project: Oozie
> Issue Type: Bug
> Components: docs
>Affects Versions: 4.2.0
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: 4.2.0, 4.3.0
>
> Attachments: OOZIE-2628-00.patch
>
>
> These workflow EL functions return type is documented as {{int}} where as
> they return {{String}}.
> {noformat}
> int wf:actionExternalId(String node)
> It returns the external Id for an action node, or an empty string if the
> action has not being executed or it has not completed yet.
> int wf:actionTrackerUri(String node)
> It returns the tracker URI for an action node, or an empty string if the
> action has not being executed or it has not completed yet.
> int wf:actionExternalStatus(String node)
> It returns the external status for an action node, or an empty string if the
> action has not being executed or it has not completed yet.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2525) SchemaChecker fails with NPE
[
https://issues.apache.org/jira/browse/OOZIE-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15415117#comment-15415117
]
abhishek bafna commented on OOZIE-2525:
---
As [~puru] mentioned above, SchemaCheckerService worker thread starts
immediately and before SchemaCheckerService instance is stored (put) into the
Services Map. Because of this race condition, NPE is happening.
We can add an initial delay of 5 second or something. We need to use the SEC
unit instead of HOUR. HOUR to SEC conversion needs to be done manually.
[~puru], [~rkanter] let me know what you guys think. Thanks.
> SchemaChecker fails with NPE
>
>
> Key: OOZIE-2525
> URL: https://issues.apache.org/jira/browse/OOZIE-2525
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>
> {code}
> 2016-05-03 14:38:03,851 WARN SchedulerService:523 [pool-1-thread-2] -
> SERVER[-] Error executing runnable [SchemaCheckerRunnable], null
> java.lang.NullPointerException
> at
> org.apache.oozie.service.SchemaCheckerService.access$002(SchemaCheckerService.java:29)
> at
> org.apache.oozie.service.SchemaCheckerService$SchemaCheckerRunnable.run(SchemaCheckerService.java:120)
> at
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It;s trying to use SchemaCheckerService before it is initialized.
> {code:title=SchemaCheckerService.java}
> Services.get().get(SchemaCheckerService.class).status = "DISABLED (not leader
> in HA)";
> Services.get().get(SchemaCheckerService.class).lastCheck =
> "N/A";
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2628) Fix Workflow EL function return types in Documentation
[
https://issues.apache.org/jira/browse/OOZIE-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15412960#comment-15412960
]
abhishek bafna commented on OOZIE-2628:
---
Patch contains only doc changes.
> Fix Workflow EL function return types in Documentation
> --
>
> Key: OOZIE-2628
> URL: https://issues.apache.org/jira/browse/OOZIE-2628
> Project: Oozie
> Issue Type: Bug
> Components: docs
>Affects Versions: 4.2.0
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: 4.2.0, 4.3.0
>
> Attachments: OOZIE-2628-00.patch
>
>
> These workflow EL functions return type is documented as {{int}} where as
> they return {{String}}.
> {noformat}
> int wf:actionExternalId(String node)
> It returns the external Id for an action node, or an empty string if the
> action has not being executed or it has not completed yet.
> int wf:actionTrackerUri(String node)
> It returns the tracker URI for an action node, or an empty string if the
> action has not being executed or it has not completed yet.
> int wf:actionExternalStatus(String node)
> It returns the external status for an action node, or an empty string if the
> action has not being executed or it has not completed yet.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2628) Fix Workflow EL function return types in Documentation
[
https://issues.apache.org/jira/browse/OOZIE-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2628:
--
Attachment: OOZIE-2628-00.patch
> Fix Workflow EL function return types in Documentation
> --
>
> Key: OOZIE-2628
> URL: https://issues.apache.org/jira/browse/OOZIE-2628
> Project: Oozie
> Issue Type: Bug
> Components: docs
>Affects Versions: 4.2.0
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: 4.2.0, 4.3.0
>
> Attachments: OOZIE-2628-00.patch
>
>
> These workflow EL functions return type is documented as {{int}} where as
> they return {{String}}.
> {noformat}
> int wf:actionExternalId(String node)
> It returns the external Id for an action node, or an empty string if the
> action has not being executed or it has not completed yet.
> int wf:actionTrackerUri(String node)
> It returns the tracker URI for an action node, or an empty string if the
> action has not being executed or it has not completed yet.
> int wf:actionExternalStatus(String node)
> It returns the external status for an action node, or an empty string if the
> action has not being executed or it has not completed yet.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OOZIE-2628) Fix Workflow EL function return types in Documentation
abhishek bafna created OOZIE-2628:
-
Summary: Fix Workflow EL function return types in Documentation
Key: OOZIE-2628
URL: https://issues.apache.org/jira/browse/OOZIE-2628
Project: Oozie
Issue Type: Bug
Components: docs
Reporter: abhishek bafna
Assignee: abhishek bafna
Fix For: 4.3.0, 4.2.0
These workflow EL functions return type is documented as {{int}} where as they
return {{String}}.
{noformat}
int wf:actionExternalId(String node)
It returns the external Id for an action node, or an empty string if the action
has not being executed or it has not completed yet.
int wf:actionTrackerUri(String node)
It returns the tracker URI for an action node, or an empty string if the action
has not being executed or it has not completed yet.
int wf:actionExternalStatus(String node)
It returns the external status for an action node, or an empty string if the
action has not being executed or it has not completed yet.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2538) Update HttpClient versions to close security vulnerabilities
[ https://issues.apache.org/jira/browse/OOZIE-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2538: -- Attachment: OOZIE-2538-01.patch > Update HttpClient versions to close security vulnerabilities > > > Key: OOZIE-2538 > URL: https://issues.apache.org/jira/browse/OOZIE-2538 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2538-01.patch, OOZIE-2538.patch > > > We learned that > https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-5262 : > http/conn/ssl/SSLConnectionSocketFactory.java in Apache HttpComponents > HttpClient before 4.3.6 ignores the http.socket.timeout configuration setting > during an SSL handshake, which allows remote attackers to cause a denial of > service (HTTPS call hang) via unspecified vectors. > Also, Commons HttpClient project is now end of life, and is no longer being > developed. It has been replaced by the Apache HttpComponents project in its > HttpClient and HttpCore modules, which offer better performance and more > flexibility. http://hc.apache.org/httpclient-3.x/ > Hence, HttpClient version should be updated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0
[
https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409093#comment-15409093
]
abhishek bafna commented on OOZIE-2613:
---
The patch includes the changes from OOZIE-2618, otherwise hive2 example fails
for {{HiveConf}} CNF exception.
> Upgrade hive version from 0.13.1 to 1.2.0
> -
>
> Key: OOZIE-2613
> URL: https://issues.apache.org/jira/browse/OOZIE-2613
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2613-00.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0
[ https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2613: -- Attachment: OOZIE-2613-00.patch > Upgrade hive version from 0.13.1 to 1.2.0 > - > > Key: OOZIE-2613 > URL: https://issues.apache.org/jira/browse/OOZIE-2613 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2613-00.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-1173) Refactor: use ParamChecker inXOozieClient
[
https://issues.apache.org/jira/browse/OOZIE-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-1173:
--
Attachment: OOZIE-1173-01.patch
Fixed the testcases.
> Refactor: use ParamChecker inXOozieClient
> -
>
> Key: OOZIE-1173
> URL: https://issues.apache.org/jira/browse/OOZIE-1173
> Project: Oozie
> Issue Type: Task
> Components: client, core
>Reporter: jun aoki
>Assignee: abhishek bafna
>Priority: Trivial
> Labels: newbie
> Fix For: 4.3.0
>
> Attachments: OOZIE-1173-00.patch, OOZIE-1173-01.patch
>
>
> In
> [XOozieClient.java|http://svn.apache.org/viewvc/oozie/trunk/client/src/main/java/org/apache/oozie/client/XOozieClient.java?revision=1401367&view=markup],
> you can find a piece of code like this and should be replaced with
> ParamChecker.
> {code}
> if (conf == null) {
> throw new IllegalArgumentException("conf cannot be null");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2036) Drop support for Java 1.6
[
https://issues.apache.org/jira/browse/OOZIE-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407088#comment-15407088
]
abhishek bafna commented on OOZIE-2036:
---
[~gezapeti] Similar to {{targetJavaVersion}} property, we should define one for
{{sourceJavaVersion}}.
> Drop support for Java 1.6
> -
>
> Key: OOZIE-2036
> URL: https://issues.apache.org/jira/browse/OOZIE-2036
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Peter Cseh
> Fix For: 4.3.0
>
> Attachments: OOZIE-2036-001.patch
>
>
> Hadoop is planning on dropping support for JDK 6, which is EoL, with Hadoop
> 2.7. We should probably do the same in Oozie trunk.
> This task involves:
> - Only allow building on JDK 7+
> - Making the build target Java 7
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2493) TestDistcpMain deletes action.xml from wrong filesystem
[
https://issues.apache.org/jira/browse/OOZIE-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407082#comment-15407082
]
abhishek bafna commented on OOZIE-2493:
---
Thanks [~rkanter] for the review and committing to master.
> TestDistcpMain deletes action.xml from wrong filesystem
> ---
>
> Key: OOZIE-2493
> URL: https://issues.apache.org/jira/browse/OOZIE-2493
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2493-02.patch, OOZIE-2493-03.patch,
> OOZIE-2493.patch
>
>
> In the TestDistcpMain#testMain test case, it is deleting a file using HDFS
> FileSystem instance which actually created on a local FileSystem.
> {code}
> // getting file hadoop system instance
> FileSystem fs = getFileSystem();
> ...
> // creating a storing file on local file system
> File actionXml = new File(getTestCaseDir(), "action.xml");
> ...
> // Now trying to delete the file on hadoop file system
> fs.delete(new Path(getTestCaseDir(), "action.xml"), true);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2569) Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher
[
https://issues.apache.org/jira/browse/OOZIE-2569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2569:
--
Attachment: OOZIE-2569-03.patch
Updated the patch with default as {{false}}.
> Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher
> ---
>
> Key: OOZIE-2569
> URL: https://issues.apache.org/jira/browse/OOZIE-2569
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2569-00.patch, OOZIE-2569-02.patch,
> OOZIE-2569-03.patch
>
>
> Oozie Spark action requires yarn-site, core-site, hdfs-site and mapred-site
> to be present in the classpath for launching the spark applications.
> Currently we manually copy these files into oozie spark sharelib.
> Until it is fixed through the spark, we can copy the
> {{oozie.action.conf.xml}} file as yarn-site, core-site, hdfs-site and
> mapred-site, which will avoid manual steps of copying those files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OOZIE-2493) TestDiscp leaves action.xml behind on local filesystem
[
https://issues.apache.org/jira/browse/OOZIE-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15406518#comment-15406518
]
abhishek bafna edited comment on OOZIE-2493 at 8/3/16 7:56 PM:
---
[~rkanter] I looked into the classes inheriting from the {{MainTestCase}}.
Except {{TestDistcpMain}}, all the others are writing into the {{action.xml}}
just once, so they are not performing the {{delete}} operation on the file.
Because, It is performing two tests in a single testcase.
Essentially the problem is, after the testcase is completed, in the
{{tearDown}} method we are just resetting the {{XTestCase#testCaseDir}},
instead we should be removing the content of the directory then might set it to
null. Same goes for the {{XFsTestCase#tearDown}}. Let me know what do you thing
about this approach.
was (Author: abhishekbafna):
[~rkanter] I looked into the classes inheriting from the {{MainTestCase}}.
Except {{TestDistcpMain}}, all the others are writing into the {{action.xml}}
just once, so they are not performing the {{delete}} operation on the file.
Essentially the problem is, after the testcase is completed, in the
{{tearDown}} method we are just resetting the {{XTestCase#testCaseDir}},
instead we should be removing the content of the directory then might set it to
null. Same goes for the {{XFsTestCase#tearDown}}. Let me know what do you thing
about this approach.
> TestDiscp leaves action.xml behind on local filesystem
> --
>
> Key: OOZIE-2493
> URL: https://issues.apache.org/jira/browse/OOZIE-2493
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2493-02.patch, OOZIE-2493-03.patch,
> OOZIE-2493.patch
>
>
> In the TestDistcpMain#testMain test case, it is deleting a file using HDFS
> FileSystem instance which actually created on a local FileSystem.
> {code}
> // getting file hadoop system instance
> FileSystem fs = getFileSystem();
> ...
> // creating a storing file on local file system
> File actionXml = new File(getTestCaseDir(), "action.xml");
> ...
> // Now trying to delete the file on hadoop file system
> fs.delete(new Path(getTestCaseDir(), "action.xml"), true);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2493) TestDiscp leaves action.xml behind on local filesystem
[
https://issues.apache.org/jira/browse/OOZIE-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15406518#comment-15406518
]
abhishek bafna commented on OOZIE-2493:
---
[~rkanter] I looked into the classes inheriting from the {{MainTestCase}}.
Except {{TestDistcpMain}}, all the others are writing into the {{action.xml}}
just once, so they are not performing the {{delete}} operation on the file.
Essentially the problem is, after the testcase is completed, in the
{{tearDown}} method we are just resetting the {{XTestCase#testCaseDir}},
instead we should be removing the content of the directory then might set it to
null. Same goes for the {{XFsTestCase#tearDown}}. Let me know what do you thing
about this approach.
> TestDiscp leaves action.xml behind on local filesystem
> --
>
> Key: OOZIE-2493
> URL: https://issues.apache.org/jira/browse/OOZIE-2493
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2493-02.patch, OOZIE-2493-03.patch,
> OOZIE-2493.patch
>
>
> In the TestDistcpMain#testMain test case, it is deleting a file using HDFS
> FileSystem instance which actually created on a local FileSystem.
> {code}
> // getting file hadoop system instance
> FileSystem fs = getFileSystem();
> ...
> // creating a storing file on local file system
> File actionXml = new File(getTestCaseDir(), "action.xml");
> ...
> // Now trying to delete the file on hadoop file system
> fs.delete(new Path(getTestCaseDir(), "action.xml"), true);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OOZIE-1176) DISCLAIMER.txt talks about incubator status
[ https://issues.apache.org/jira/browse/OOZIE-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna resolved OOZIE-1176. --- Resolution: Duplicate Fix Version/s: (was: trunk) 4.3.0 > DISCLAIMER.txt talks about incubator status > --- > > Key: OOZIE-1176 > URL: https://issues.apache.org/jira/browse/OOZIE-1176 > Project: Oozie > Issue Type: Bug > Components: docs >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Udai Kiran Potluri >Priority: Trivial > Labels: newbie > Fix For: 4.3.0 > > > The DISCLAIMER.txt file in the root dir talks about Oozie being in incubator > status and gives the http://incubator.apache.org/oozie url. > We should either update the disclaimer (is there a standard Apache disclaimer > that should go here?) or simply delete the file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-1173) Refactor: use ParamChecker inXOozieClient
[
https://issues.apache.org/jira/browse/OOZIE-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-1173:
--
Assignee: abhishek bafna
Affects Version/s: (was: trunk)
Fix Version/s: (was: trunk)
4.3.0
Component/s: core
> Refactor: use ParamChecker inXOozieClient
> -
>
> Key: OOZIE-1173
> URL: https://issues.apache.org/jira/browse/OOZIE-1173
> Project: Oozie
> Issue Type: Task
> Components: client, core
>Reporter: jun aoki
>Assignee: abhishek bafna
>Priority: Trivial
> Labels: newbie
> Fix For: 4.3.0
>
> Attachments: OOZIE-1173-00.patch
>
>
> In
> [XOozieClient.java|http://svn.apache.org/viewvc/oozie/trunk/client/src/main/java/org/apache/oozie/client/XOozieClient.java?revision=1401367&view=markup],
> you can find a piece of code like this and should be replaced with
> ParamChecker.
> {code}
> if (conf == null) {
> throw new IllegalArgumentException("conf cannot be null");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-1173) Refactor: use ParamChecker inXOozieClient
[
https://issues.apache.org/jira/browse/OOZIE-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-1173:
--
Attachment: OOZIE-1173-00.patch
> Refactor: use ParamChecker inXOozieClient
> -
>
> Key: OOZIE-1173
> URL: https://issues.apache.org/jira/browse/OOZIE-1173
> Project: Oozie
> Issue Type: Task
> Components: client
>Affects Versions: trunk
>Reporter: jun aoki
>Priority: Trivial
> Labels: newbie
> Fix For: trunk
>
> Attachments: OOZIE-1173-00.patch
>
>
> In
> [XOozieClient.java|http://svn.apache.org/viewvc/oozie/trunk/client/src/main/java/org/apache/oozie/client/XOozieClient.java?revision=1401367&view=markup],
> you can find a piece of code like this and should be replaced with
> ParamChecker.
> {code}
> if (conf == null) {
> throw new IllegalArgumentException("conf cannot be null");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2569) Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher
[
https://issues.apache.org/jira/browse/OOZIE-2569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2569:
--
Attachment: OOZIE-2569-02.patch
[~satishsaley] Thanks for the feedback.
Made the hadoop configuration for spark action configurable. By default this is
be enabled (true) so that user can have it out of the box.
[~satishsaley], [~rkanter] Please do provide your feedback regarding the
default behavior of change. Thanks.
> Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher
> ---
>
> Key: OOZIE-2569
> URL: https://issues.apache.org/jira/browse/OOZIE-2569
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2569-00.patch, OOZIE-2569-02.patch
>
>
> Oozie Spark action requires yarn-site, core-site, hdfs-site and mapred-site
> to be present in the classpath for launching the spark applications.
> Currently we manually copy these files into oozie spark sharelib.
> Until it is fixed through the spark, we can copy the
> {{oozie.action.conf.xml}} file as yarn-site, core-site, hdfs-site and
> mapred-site, which will avoid manual steps of copying those files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2036) Drop support for Java 1.6
[ https://issues.apache.org/jira/browse/OOZIE-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15406162#comment-15406162 ] abhishek bafna commented on OOZIE-2036: --- [~rkanter] Can you please add the patch for this. Thanks. > Drop support for Java 1.6 > - > > Key: OOZIE-2036 > URL: https://issues.apache.org/jira/browse/OOZIE-2036 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Robert Kanter > Fix For: 4.3.0 > > > Hadoop is planning on dropping support for JDK 6, which is EoL, with Hadoop > 2.7. We should probably do the same in Oozie trunk. > This task involves: > - Only allow building on JDK 7+ > - Making the build target Java 7 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2565) [Oozie web Console] Make the timezones in settings tab to be sorted by default
[ https://issues.apache.org/jira/browse/OOZIE-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2565: -- Fix Version/s: 4.3.0 > [Oozie web Console] Make the timezones in settings tab to be sorted by default > -- > > Key: OOZIE-2565 > URL: https://issues.apache.org/jira/browse/OOZIE-2565 > Project: Oozie > Issue Type: New Feature > Components: site >Affects Versions: trunk >Reporter: Chandan >Priority: Minor > Labels: easyfix, features, newbie > Fix For: 4.3.0 > > Attachments: OOZIE-2565.patch > > > Currently, the timezone field in the settings tab in the oozie web console is > not sorted. > This makes it difficult to select the intended timezone as the user has to > scroll through all the timezones until he finds the intended one. > Is the field is sorted, this will help him to navigate to the concerned > timezone. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2520) SortBy filter for ordering the jobs query results
[ https://issues.apache.org/jira/browse/OOZIE-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2520: -- Fix Version/s: 4.3.0 > SortBy filter for ordering the jobs query results > - > > Key: OOZIE-2520 > URL: https://issues.apache.org/jira/browse/OOZIE-2520 > Project: Oozie > Issue Type: Improvement > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2520-01.patch, OOZIE-2520-02.patch, > OOZIE-2520-03.patch > > > Currently jobs query results are order by job creation time. It is good to > have a filter option for ordering the results by lastModifiedTime or > createdTime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2462) When calling ParamChecker.notNull() in CoordActionsIgnoreXCommand.java, "Action" should be passed instead of "Action cannot be null".
[
https://issues.apache.org/jira/browse/OOZIE-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2462:
--
Fix Version/s: 4.3.0
> When calling ParamChecker.notNull() in CoordActionsIgnoreXCommand.java,
> "Action" should be passed instead of "Action cannot be null".
> -
>
> Key: OOZIE-2462
> URL: https://issues.apache.org/jira/browse/OOZIE-2462
> Project: Oozie
> Issue Type: Bug
>Reporter: Manjunath Ballur
>Priority: Trivial
> Labels: newbie
> Fix For: 4.3.0
>
> Attachments: OOZIE-2462-001.patch, OOZIE-2462-002.patch,
> OOZIE-2462-002.patch
>
>
> In ParamChecker::notNull() method, if the passed object is null, then
> following exception is thrown:
> if (obj == null) {
> throw new IllegalArgumentException(name + " cannot be null");
> }
> Here, "name" is the name of the object being checked.
> In "CoordActionsIgnoreXCommand::checkAllActionsStatus()" method, following
> piece of code calls "ParamChecker::notNull()" method:
> ParamChecker.notNull(action, "Action cannot be null");
> The above statement should be changed to:
> ParamChecker.notNull(action, "Action");
> With the current code, when "action" is null, then the message printed will
> be:
> "Action cannot be null cannot be null"
> "cannot be null" is printed twice.
> Hence, we need to remove "cannot be null" while calling
> "ParamChecker::notNull()" method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2390) Rerun with failed option removing completed output data
[ https://issues.apache.org/jira/browse/OOZIE-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2390: -- Fix Version/s: 4.3.0 > Rerun with failed option removing completed output data > --- > > Key: OOZIE-2390 > URL: https://issues.apache.org/jira/browse/OOZIE-2390 > Project: Oozie > Issue Type: Bug >Reporter: Jaydeep Vishwakarma >Assignee: Jaydeep Vishwakarma >Priority: Critical > Fix For: 4.3.0 > > Attachments: OOZIE-2390.patch > > > Rerun with failed option is used to run workflow from failed node. During the > time of rerun cleanup operation is removing the output directories which is > incorrect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2440) Exponential re-try policy for workflow action
[ https://issues.apache.org/jira/browse/OOZIE-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2440: -- Fix Version/s: 4.3.0 > Exponential re-try policy for workflow action > - > > Key: OOZIE-2440 > URL: https://issues.apache.org/jira/browse/OOZIE-2440 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley >Priority: Minor > Fix For: 4.3.0 > > Attachments: OOZIE-2440-2.patch, OOZIE-2440-3.patch, > OOZIE-2440-4.patch, OOZIE-2440-5.patch > > > Currently the user can mention retry interval and maximum number of retries. > We will add another element in action tag through which the user can mention > policy for the retry. Policy could be exponential or periodic. Periodic will > be the default as it is the current policy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2539) Incorrect property key is used for 'hive log4j configuration file for execution mode'
[ https://issues.apache.org/jira/browse/OOZIE-2539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2539: -- Fix Version/s: 4.3.0 > Incorrect property key is used for 'hive log4j configuration file for > execution mode' > - > > Key: OOZIE-2539 > URL: https://issues.apache.org/jira/browse/OOZIE-2539 > Project: Oozie > Issue Type: Bug >Reporter: abhishek bafna >Assignee: abhishek bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2539.patch > > > The correct property key for hive configuration file for execution mode is > *hive.exec.log4j.file* instead we are setting *hive.log4j.exec.file*. We > should correct this. > This is working for now, because, If the property is not set, then logging > will be initialized using *hive-exec-log4j.properties* found on the classpath. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2613) Upgrade hive version from 0.13.1 to 1.2.0
[ https://issues.apache.org/jira/browse/OOZIE-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2613: -- Fix Version/s: 4.3.0 > Upgrade hive version from 0.13.1 to 1.2.0 > - > > Key: OOZIE-2613 > URL: https://issues.apache.org/jira/browse/OOZIE-2613 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Fix For: 4.3.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2602) Upgrade oozie to pig 0.16.0
[ https://issues.apache.org/jira/browse/OOZIE-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2602: -- Fix Version/s: 4.3.0 > Upgrade oozie to pig 0.16.0 > --- > > Key: OOZIE-2602 > URL: https://issues.apache.org/jira/browse/OOZIE-2602 > Project: Oozie > Issue Type: Improvement >Reporter: Narayan Periwal >Assignee: Narayan Periwal > Fix For: 4.3.0 > > Attachments: oozie-2602.patch > > > Currently the pig version in Oozie is 0.12.1 > We can upgrade oozie to pig 0.16.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2231) Upgrade curator to latest version 2.8.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2231: -- Fix Version/s: 4.3.0 > Upgrade curator to latest version 2.8.0 > --- > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah > Fix For: 4.3.0 > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2488) Upgrade to latest OpenJPA version
[ https://issues.apache.org/jira/browse/OOZIE-2488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2488: -- Affects Version/s: (was: trunk) Fix Version/s: (was: trunk) 4.3.0 > Upgrade to latest OpenJPA version > - > > Key: OOZIE-2488 > URL: https://issues.apache.org/jira/browse/OOZIE-2488 > Project: Oozie > Issue Type: Improvement > Components: core >Reporter: Robert Kanter > Fix For: 4.3.0 > > > Oozie is currently using OpenJPA 2.2.2, which is from April 2013. Version > 2.3.0 is from Nov 2013, but there was a recent 2.4.1 release in Feb 2016 > (that's a long gap!). Given that this is a critical dependency, it might be > a good idea to take a look at the chnagelog, do some testing, and update the > dependency. I'm sure there's lots of bug fixes that would be great to have. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2409) hive2 action with hive 1.2.1 failed
[
https://issues.apache.org/jira/browse/OOZIE-2409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2409:
--
Fix Version/s: 4.3.0
> hive2 action with hive 1.2.1 failed
> ---
>
> Key: OOZIE-2409
> URL: https://issues.apache.org/jira/browse/OOZIE-2409
> Project: Oozie
> Issue Type: Bug
> Components: build
>Affects Versions: 4.2.0
>Reporter: KWON BYUNGCHANG
>Priority: Minor
> Fix For: 4.3.0
>
> Attachments: OOZIE-2409.01.patch, OOZIE-2409.02.patch,
> OOZIE-2409.patch
>
>
> hive2 action with hive 1.2.1 failed.
> stack trace
> {code}
> 015-11-26 15:47:39,141 WARN Hive2ActionExecutor:523 - SERVER[myhost.com]
> USER[myuser] GROUP[-] TOKEN[] APP[test]
> JOB[401-151125114930017-oozie-irte-W]
> ACTION[401-151125114930017-oozie-irte-W@HIVEJOB] Launcher exception:
> jline/console/history/History
> java.lang.NoClassDefFoundError: jline/console/history/History
> at
> org.apache.oozie.action.hadoop.Hive2Main.runBeeline(Hive2Main.java:240)
> at org.apache.oozie.action.hadoop.Hive2Main.run(Hive2Main.java:223)
> at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
> at org.apache.oozie.action.hadoop.Hive2Main.main(Hive2Main.java:56)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {code}
> hive 1.2.1 need to jline-2.12.jar.
> But hive2 shareblib has jline-0.9.94.jar.
> jline-0.9.94.jar does not have jline/console/history/History class.
> I have changed {{sharelib/hive2/pom.xml}} temporarily.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2606) Set spark.yarn.jars to fix Spark 2.0 with Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2606: -- Fix Version/s: (was: trunk) 4.3.0 > Set spark.yarn.jars to fix Spark 2.0 with Oozie > --- > > Key: OOZIE-2606 > URL: https://issues.apache.org/jira/browse/OOZIE-2606 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.2.0 >Reporter: Jonathan Kelly > Labels: spark, spark2.0.0 > Fix For: 4.3.0 > > Attachments: OOZIE-2606.patch > > > Oozie adds all of the jars in the Oozie Spark sharelib to the > DistributedCache such that all jars will be present in the current working > directory of the YARN container (as well as in the container classpath). > However, this is not quite enough to make Spark 2.0 work, since Spark 2.0 by > default looks for the jars in assembly/target/scala-2.11/jars [1] (as if it > is a locally built distribution for development) and will not find them in > the current working directory. > To fix this, we can set spark.yarn.jars to *.jar so that it finds the jars in > the current working directory rather than looking in the wrong place. [2] > [1] > https://github.com/apache/spark/blob/v2.0.0-rc2/launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java#L357 > [2] > https://github.com/apache/spark/blob/v2.0.0-rc2/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L476 > Note: This property will be ignored by Spark 1.x. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2036) Drop support for Java 1.6
[ https://issues.apache.org/jira/browse/OOZIE-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2036: -- Fix Version/s: 4.3.0 > Drop support for Java 1.6 > - > > Key: OOZIE-2036 > URL: https://issues.apache.org/jira/browse/OOZIE-2036 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Robert Kanter > Fix For: 4.3.0 > > > Hadoop is planning on dropping support for JDK 6, which is EoL, with Hadoop > 2.7. We should probably do the same in Oozie trunk. > This task involves: > - Only allow building on JDK 7+ > - Making the build target Java 7 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2037) Add TLSv1.1,TLSv1.2
[ https://issues.apache.org/jira/browse/OOZIE-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2037: -- Fix Version/s: 4.3.0 > Add TLSv1.1,TLSv1.2 > --- > > Key: OOZIE-2037 > URL: https://issues.apache.org/jira/browse/OOZIE-2037 > Project: Oozie > Issue Type: Sub-task > Components: security >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Robert Kanter > Labels: newbie > Fix For: 4.3.0 > > Attachments: OOZIE-2037.001.patch > > > OOZIE-2034 required us to specifically list the versions of TLS that Oozie > supports. Java 7 supports TLSv1.1 and TLSv1.2, so we should add them to the > list. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-1793) Improve find bugs reporting for Oozie
[
https://issues.apache.org/jira/browse/OOZIE-1793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-1793:
--
Fix Version/s: 4.3.0
> Improve find bugs reporting for Oozie
> -
>
> Key: OOZIE-1793
> URL: https://issues.apache.org/jira/browse/OOZIE-1793
> Project: Oozie
> Issue Type: Sub-task
> Components: build
>Affects Versions: trunk
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Fix For: 4.3.0
>
> Attachments: OOZIE-1793.patch, OOZIE-1793.patch
>
>
> I couldn't figure out how to run findbugs against Oozie. And even if it does
> work, it looks like it's just going to create an XML file.
> Based on some minor refactoring to this
> [tutorial|http://www.petrikainulainen.net/programming/maven/findbugs-maven-plugin-tutorial/]
> and this [stack overflow
> message|http://stackoverflow.com/questions/7035112/any-easy-way-to-generate-a-findbug-html-report-from-maven-without-sitesite/10365954#10365954],
> I was able to improve our findbugs reporting. Now, when you run {{mvn
> verify}} (which we use to generate a checkstyle report), it will also run
> findbugs and transform the resulting XML files into html files that are human
> readable.
> The hard part here will be actually going through the reports and fixing
> these bugs; but that's for other JIRAs :)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2496) Testcase failure due to environment specific line.separator value
[
https://issues.apache.org/jira/browse/OOZIE-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15405608#comment-15405608
]
abhishek bafna commented on OOZIE-2496:
---
testcase failure is not related.
{noformat}
-1 COMPILE
-1 HEAD does not compile
-1 patch does not compile
{noformat}
The above failure are because of unable to download maven dependency for
{{gwt-links}}. This is happening very frequently for last couple of weeks.
> Testcase failure due to environment specific line.separator value
> -
>
> Key: OOZIE-2496
> URL: https://issues.apache.org/jira/browse/OOZIE-2496
> Project: Oozie
> Issue Type: Bug
> Components: tests
> Environment: Windows
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2496-01.patch, OOZIE-2496-02.patch,
> OOZIE-2496-03.patch
>
>
> The "line.separator" returns operating system specific line separator. The
> following testcase uses hard coded "\n" while comparing the return values in
> the assertion, which makes them to fail on the windows environment.
> {noformat}
> org.apache.oozie.client.TestOozieCLI#testOozieStatus
> org.apache.oozie.client.TestOozieCLI#testServerBuildVersion
> org.apache.oozie.client.TestOozieCLI#testClientBuildVersion
> {noformat}
> In the following testcase, we are using "line.separator" where as
> _TimestampedMessageParser_ sends the logs after appending "\n".
> {noformat}
> org.apache.oozie.util.TestXLogUserFilterParam#testloglevel_DEBUF_INFO
> org.apache.oozie.util.TestXLogUserFilterParam#testloglevel_Error
> org.apache.oozie.util.TestXLogUserFilterParam#testTextSearch
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2582) Populating external child Ids for action failures
[ https://issues.apache.org/jira/browse/OOZIE-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2582: -- Fix Version/s: 4.3.0 > Populating external child Ids for action failures > - > > Key: OOZIE-2582 > URL: https://issues.apache.org/jira/browse/OOZIE-2582 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2582-00.patch, OOZIE-2582-01.patch, > OOZIE-2582-02.patch, OOZIE-2582-03.patch, OOZIE-2582-04.patch, > OOZIE-2582-05.patch > > > Currently Oozie external child ids are populated into workflow bean, when the > job/action completes successfully. It should populate external child ids in > case of job failures as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2513) log.scan.duration should not be used for error and audit logs
[ https://issues.apache.org/jira/browse/OOZIE-2513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2513: -- Fix Version/s: 4.3.0 > log.scan.duration should not be used for error and audit logs > - > > Key: OOZIE-2513 > URL: https://issues.apache.org/jira/browse/OOZIE-2513 > Project: Oozie > Issue Type: Bug >Reporter: Purshotam Shah >Assignee: Purshotam Shah > Fix For: 4.3.0 > > Attachments: OOZIE-2513-V1.patch, OOZIE-2513-V2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2498) Oozie CallerId configuration for downstream components
[ https://issues.apache.org/jira/browse/OOZIE-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2498: -- Fix Version/s: 4.3.0 > Oozie CallerId configuration for downstream components > --- > > Key: OOZIE-2498 > URL: https://issues.apache.org/jira/browse/OOZIE-2498 > Project: Oozie > Issue Type: Improvement > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Labels: patch > Fix For: 4.3.0 > > Attachments: OOZIE-2498-01.patch > > > The idea is to add callerId to every component, so we can track the chain of > application which cause the underlining operation. A typical chain is > Oozie->Pig|Hive->Tez->Hdfs. With proper callerId logging, we can trace Hdfs > operation back to Oozie workflow which triggers it. > For Oozie, It will pass its workflow id as callerId to other components > (Pig/Hive). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2569) Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher
[
https://issues.apache.org/jira/browse/OOZIE-2569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2569:
--
Fix Version/s: 4.3.0
> Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher
> ---
>
> Key: OOZIE-2569
> URL: https://issues.apache.org/jira/browse/OOZIE-2569
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2569-00.patch
>
>
> Oozie Spark action requires yarn-site, core-site, hdfs-site and mapred-site
> to be present in the classpath for launching the spark applications.
> Currently we manually copy these files into oozie spark sharelib.
> Until it is fixed through the spark, we can copy the
> {{oozie.action.conf.xml}} file as yarn-site, core-site, hdfs-site and
> mapred-site, which will avoid manual steps of copying those files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2521) Filter options are case sensitive for jobtye=bundle
[ https://issues.apache.org/jira/browse/OOZIE-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2521: -- Fix Version/s: 4.3.0 > Filter options are case sensitive for jobtye=bundle > --- > > Key: OOZIE-2521 > URL: https://issues.apache.org/jira/browse/OOZIE-2521 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Fix For: 4.3.0 > > > The filter options are case sensitive for jobtype=bundle, where as for > coordinators and workflow it is not. This could be confusing for the user and > should be consistent across the jobs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2538) Update HttpClient versions to close security vulnerabilities
[ https://issues.apache.org/jira/browse/OOZIE-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2538: -- Fix Version/s: 4.3.0 > Update HttpClient versions to close security vulnerabilities > > > Key: OOZIE-2538 > URL: https://issues.apache.org/jira/browse/OOZIE-2538 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2538.patch > > > We learned that > https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2015-5262 : > http/conn/ssl/SSLConnectionSocketFactory.java in Apache HttpComponents > HttpClient before 4.3.6 ignores the http.socket.timeout configuration setting > during an SSL handshake, which allows remote attackers to cause a denial of > service (HTTPS call hang) via unspecified vectors. > Also, Commons HttpClient project is now end of life, and is no longer being > developed. It has been replaced by the Apache HttpComponents project in its > HttpClient and HttpCore modules, which offer better performance and more > flexibility. http://hc.apache.org/httpclient-3.x/ > Hence, HttpClient version should be updated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2496) Testcase failure due to environment specific line.separator value
[
https://issues.apache.org/jira/browse/OOZIE-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2496:
--
Fix Version/s: 4.3.0
> Testcase failure due to environment specific line.separator value
> -
>
> Key: OOZIE-2496
> URL: https://issues.apache.org/jira/browse/OOZIE-2496
> Project: Oozie
> Issue Type: Bug
> Components: tests
> Environment: Windows
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2496-01.patch, OOZIE-2496-02.patch,
> OOZIE-2496-03.patch
>
>
> The "line.separator" returns operating system specific line separator. The
> following testcase uses hard coded "\n" while comparing the return values in
> the assertion, which makes them to fail on the windows environment.
> {noformat}
> org.apache.oozie.client.TestOozieCLI#testOozieStatus
> org.apache.oozie.client.TestOozieCLI#testServerBuildVersion
> org.apache.oozie.client.TestOozieCLI#testClientBuildVersion
> {noformat}
> In the following testcase, we are using "line.separator" where as
> _TimestampedMessageParser_ sends the logs after appending "\n".
> {noformat}
> org.apache.oozie.util.TestXLogUserFilterParam#testloglevel_DEBUF_INFO
> org.apache.oozie.util.TestXLogUserFilterParam#testloglevel_Error
> org.apache.oozie.util.TestXLogUserFilterParam#testTextSearch
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2552) Update ActiveMQ version for security and other fixes
[ https://issues.apache.org/jira/browse/OOZIE-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2552: -- Fix Version/s: 4.3.0 > Update ActiveMQ version for security and other fixes > > > Key: OOZIE-2552 > URL: https://issues.apache.org/jira/browse/OOZIE-2552 > Project: Oozie > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Robert Kanter >Assignee: Attila Sasvari > Fix For: 4.3.0 > > Attachments: OOZIE-2552-001.patch, OOZIE-2552-002.patch, > OOZIE-2552-003.patch > > > We're currently using ActiveMQ 5.10.2. We should use a newer version, which > includes various security and bug fixes. > We are currently stuck on 5.10.2 because it's the last version that supports > Java 6, but once OOZIE-2036 is done, we should get the latest version that > supports Java 7. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2500) -DtestJarSimple option mentioned in minioozie doc does not work
[
https://issues.apache.org/jira/browse/OOZIE-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2500:
--
Fix Version/s: 4.3.0
> -DtestJarSimple option mentioned in minioozie doc does not work
> ---
>
> Key: OOZIE-2500
> URL: https://issues.apache.org/jira/browse/OOZIE-2500
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: abhishek bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2500-01.patch, out.txt
>
>
> [Minioozie doc|https://oozie.apache.org/docs/4.2.0/ENG_MiniOozie.html] says
> to run {{$ mvn clean install -DskipTests -DtestJarSimple}} to populate local
> maven repo with required jars. But the command fails with following message:
> {code}
> [INFO] 100 errors
> [INFO] -
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Oozie Main .. SUCCESS [ 1.339
> s]
> [INFO] Apache Oozie Hadoop Utils .. SUCCESS [ 0.999
> s]
> [INFO] Apache Oozie Hadoop Distcp hadoop-1-4.3.0-SNAPSHOT . SUCCESS [ 0.138
> s]
> [INFO] Apache Oozie Hadoop Auth hadoop-1-4.3.0-SNAPSHOT ... SUCCESS [ 0.244
> s]
> [INFO] Apache Oozie Hadoop Libs ... SUCCESS [ 0.052
> s]
> [INFO] Apache Oozie Client SUCCESS [ 3.282
> s]
> [INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ 2.257
> s]
> [INFO] Apache Oozie Share Lib HCatalog SUCCESS [ 2.088
> s]
> [INFO] Apache Oozie Share Lib Distcp .. SUCCESS [ 0.693
> s]
> [INFO] Apache Oozie Core .. SUCCESS [ 20.951
> s]
> [INFO] Apache Oozie Share Lib Streaming ... FAILURE [ 2.688
> s]
> [INFO] Apache Oozie Share Lib Pig . SKIPPED
> [INFO] Apache Oozie Share Lib Hive SKIPPED
> [INFO] Apache Oozie Share Lib Hive 2 .. SKIPPED
> [INFO] Apache Oozie Share Lib Sqoop ... SKIPPED
> [INFO] Apache Oozie Examples .. SKIPPED
> [INFO] Apache Oozie Share Lib Spark ... SKIPPED
> [INFO] Apache Oozie Share Lib . SKIPPED
> [INFO] Apache Oozie Docs .. SKIPPED
> [INFO] Apache Oozie WebApp SKIPPED
> [INFO] Apache Oozie Tools . SKIPPED
> [INFO] Apache Oozie MiniOozie . SKIPPED
> [INFO] Apache Oozie Distro SKIPPED
> [INFO] Apache Oozie ZooKeeper Security Tests .. SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 35.474 s
> [INFO] Finished at: 2016-04-04T16:04:41-07:00
> [INFO] Final Memory: 161M/1262M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.3.2:testCompile
> (default-testCompile) on project oozie-sharelib-streaming: Compilation
> failure: Compilation failure:
> [ERROR]
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestStreamingMain.java:[31,39]
> error: cannot find symbol
> [ERROR] class MainTestCase
> [ERROR]
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java:[76,49]
> error: cannot find symbol
> [ERROR] class ActionExecutorTestCase
> [ERROR]
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java:[342,14]
> error: cannot find symbol
> [ERROR] class TestMapReduceActionExecutor
> [ERROR]
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java:[365,14]
> error: cannot find symbol
> [ERROR] class TestMapReduceActionExecutor
> [ERROR]
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestMapReduceActionExecutor.java:[389,38]
> error: cannot find symbol
> [ERROR] class TestMapReduceActionExecutor
> [ERROR]
> /Users/saley/src/oozie/sharelib/streaming/src/test/java/org/apache/oozie/action/hadoop/TestStreamingMain.java:[34,24]
> error: cannot find symbol
> [ERROR] class TestStreamingMain
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2572) SLA DURATION miss not shown when job is running for longer than expected time
[ https://issues.apache.org/jira/browse/OOZIE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2572: -- Fix Version/s: 4.3.0 > SLA DURATION miss not shown when job is running for longer than expected time > - > > Key: OOZIE-2572 > URL: https://issues.apache.org/jira/browse/OOZIE-2572 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: 4.3.0 > > Attachments: OOZIE-2572-1.patch, OOZIE-2572-2.patch, > OOZIE-2572-3.patch, OOZIE-2572-4.patch, OOZIE-2572-5.patch > > > If job is running for beyond expected duration, Oozie UI is not slowing SLA > for duration miss. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2573) dataset url which contains spaces can not be handled rightly
[ https://issues.apache.org/jira/browse/OOZIE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2573: -- Fix Version/s: 4.3.0 > dataset url which contains spaces can not be handled rightly > - > > Key: OOZIE-2573 > URL: https://issues.apache.org/jira/browse/OOZIE-2573 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 4.2.0 >Reporter: WangMeng > Fix For: 4.3.0 > > Attachments: OOZIE-2573.01.patch > > > Currently , Oozie does not escape special character-spaces , this will lead > to dataset url which contains spaces can not be handled rightly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2517) Add support for startCreatedTime and endCreatedTime filters for coord and bundles
[
https://issues.apache.org/jira/browse/OOZIE-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2517:
--
Fix Version/s: 4.3.0
> Add support for startCreatedTime and endCreatedTime filters for coord and
> bundles
> -
>
> Key: OOZIE-2517
> URL: https://issues.apache.org/jira/browse/OOZIE-2517
> Project: Oozie
> Issue Type: Bug
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2517-001.patch, OOZIE-2517-003.patch,
> OOZIE-2517-005.patch, OOZIE-2517-007.patch, OOZIE-2517-008.patch,
> OOZIE-2517-009.patch, OOZIE-2517-010.patch
>
>
> Support for the 'startCreatedTime' and 'endCreatedTime' is not implemented
> for the coordinators and bundle job.
> Curl command with jobtype coordinator and bundle fails where as jobtype=wf
> executes successfully.
> https://oozie.apache.org/docs/4.2.0/WebServicesAPI.html#Jobs_Information
> {noformat}
> curl
> "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=bundle";
> curl
> "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=coordinator";
> curl
> "http://host:11000/oozie/v2/jobs?filter=startCreatedTime%3D2016-04-28T00%3A00Z&jobtype=wf";
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2338) Invalid configuration defined reported for some valid configs
[
https://issues.apache.org/jira/browse/OOZIE-2338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2338:
--
Fix Version/s: 4.3.0
> Invalid configuration defined reported for some valid configs
> -
>
> Key: OOZIE-2338
> URL: https://issues.apache.org/jira/browse/OOZIE-2338
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Fix For: 4.3.0
>
> Attachments: OOZIE-2338.001.patch, OOZIE-2338.002.patch
>
>
> OOZIE-1890 moved all default config values from the code and oozie-site into
> oozie-default; it also added a WARN message on startup when a config is found
> in oozie-site that's not defined in oozie-default.
> However, this can produce some false positives. In particular, we observed
> these:
> {noformat}
> 2015-08-18 01:16:32,365 WARN org.apache.oozie.service.ConfigurationService:
> SERVER[] Invalid configuration defined, [oozie.email.smtp.auth]
> 2015-08-18 01:16:32,365 WARN org.apache.oozie.service.ConfigurationService:
> SERVER[] Invalid configuration defined, [oozie.email.smtp.host]
> 2015-08-18 01:16:32,365 WARN org.apache.oozie.service.ConfigurationService:
> SERVER[] Invalid configuration defined,
> [oozie.service.HCatAccessorService.hcat.configuration]
> 2015-08-18 01:16:32,365 WARN org.apache.oozie.service.ConfigurationService:
> SERVER[] Invalid configuration defined, [oozie.email.from.address]
> 2015-08-18 01:16:32,366 WARN org.apache.oozie.service.ConfigurationService:
> SERVER[] Invalid configuration defined,
> [oozie.service.ProxyUserService.proxyuser.hue.hosts]
> 2015-08-18 01:16:32,366 WARN org.apache.oozie.service.ConfigurationService:
> SERVER[] Invalid configuration defined,
> [oozie.service.GroupsService.hadoop.security.group.mapping]
> 2015-08-18 01:16:32,366 WARN org.apache.oozie.service.ConfigurationService:
> SERVER[] Invalid configuration defined, [oozie.email.smtp.port]
> 2015-08-18 01:16:32,366 WARN org.apache.oozie.service.ConfigurationService:
> SERVER[] Invalid configuration defined,
> [oozie.service.ProxyUserService.proxyuser.hue.groups]
> {noformat}
> While harmful, these messages may be concerning or confusing for users.
> Some of these are simply missing from oozie-default (because they have no
> default value?), while others such as the ProxyUserService configs, will
> require special handling because the property name can vary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2617) Read properties files in action configurations
[ https://issues.apache.org/jira/browse/OOZIE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2617: -- Fix Version/s: (was: trunk) 4.3.0 > Read properties files in action configurations > -- > > Key: OOZIE-2617 > URL: https://issues.apache.org/jira/browse/OOZIE-2617 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: trunk >Reporter: Taklon Stephen Wu > Labels: patch > Fix For: 4.3.0 > > Attachments: OOZIE-2617-1.patch, OOZIE-2617-2.patch, > OOZIE-2617-3.patch > > > Current logic of acton configuration only READ xml files within Oozie action > directory, but for pig and other applications, properties files are supported > and may be default configuration file format. To simplify the logic of > reusing these properties files when using Oozie with them, we add properties > file reading in action conf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2574) Oozie to support replication-enabled mysql urls
[
https://issues.apache.org/jira/browse/OOZIE-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2574:
--
Fix Version/s: 4.3.0
> Oozie to support replication-enabled mysql urls
> ---
>
> Key: OOZIE-2574
> URL: https://issues.apache.org/jira/browse/OOZIE-2574
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Reporter: abhishek bafna
>Assignee: abhishek bafna
> Fix For: 4.3.0
>
> Attachments: OOZIE-2574-00.patch
>
>
> Currently Oozie does not support replication-enabled mysql urls. As per the
> mysql JDBC specification we can use
> {{jdbc:mysql:replication://master_host:port,slave_host:port/database}}.
> https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-master-slave-replication-connection.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2508) Documentation change for Coord action rerun [OOZIE-1735]
[
https://issues.apache.org/jira/browse/OOZIE-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2508:
--
Fix Version/s: 4.3.0
> Documentation change for Coord action rerun [OOZIE-1735]
>
>
> Key: OOZIE-2508
> URL: https://issues.apache.org/jira/browse/OOZIE-2508
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: 4.3.0
>
> Attachments: OOZIE-2508-1.patch, OOZIE-2508-2.patch,
> OOZIE-2508-3.patch, OOZIE-2508-4.patch, OOZIE-2508-5.patch
>
>
> One user
>
> 1) Killed a coordinator job.
> 2) Rerun one of the actions of the killed coordinator.
> As of [Oozie
> Documentation|http://oozie.apache.org/docs/4.2.0/DG_CoordinatorRerun.html]
> Rerun should not work.
> {quote}
> Coordinator actions cannot be rerun if the coordinator job is in the KILLED
> or FAILED state.
> {quote}
> But [OOZIE-1735|https://issues.apache.org/jira/browse/OOZIE-1735] added the
> support for resuming of failed coordinator job and rerun of a failed
> coordinator action.
> Therefore, documentation needs to be updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2433) oozie restart required if oozie metrics to graphing tool broken
[ https://issues.apache.org/jira/browse/OOZIE-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] abhishek bafna updated OOZIE-2433: -- Fix Version/s: 4.3.0 > oozie restart required if oozie metrics to graphing tool broken > --- > > Key: OOZIE-2433 > URL: https://issues.apache.org/jira/browse/OOZIE-2433 > Project: Oozie > Issue Type: Bug >Reporter: Sanjeev T >Assignee: Narayan Periwal > Fix For: 4.3.0 > > Attachments: OOZIE-2433.patch > > > When we start oozie service it starts emitting oozie metrics, expose to the > host > * oozie.external_monitoring.address > If we have issue on the external host or service disruption for a while, the > metrics are not exposed, it requires oozie restart > * Can we avoid oozie restart, let oozie do health check and expose metrics > * Enable logging for the oozie metrics emitting to external host -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2273) MiniOozie does not work outside of Oozie
[
https://issues.apache.org/jira/browse/OOZIE-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2273:
--
Fix Version/s: 4.3.0
> MiniOozie does not work outside of Oozie
>
>
> Key: OOZIE-2273
> URL: https://issues.apache.org/jira/browse/OOZIE-2273
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Affects Versions: 4.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
> Fix For: 4.3.0
>
> Attachments: OOZIE-2273.001.patch
>
>
> {{MiniOozie}} doesn't appear to work correctly outside of Oozie, due to some
> hardcoded paths/assumptions. We should fix this.
> As a quick workaround, I added the "core" folder and sub folders/files. For
> example:
> {noformat}
> project-root
> |- src
> |- pom.xml
> |- (other typical maven stuff)
> |- core
> |- |- src
> |- |- |- test
> |- |- |- |- resources
> |- |- |- |- |- hadoop-config.xml (a copy of this file [1])
> {noformat}
> Also, the Metastore doesn't work properly either (not sure why). It can be
> disabled by adding {{System.setProperty("oozie.test.metastore.server",
> "false");}} in the {{setUp()}} method, as done at \[2], but we should fix
> that as well.
> \[1]
> https://raw.githubusercontent.com/apache/oozie/master/core/src/test/resources/hadoop-config.xml
> \[2]
> https://github.com/apache/oozie/blob/master/minitest/src/test/java/org/apache/oozie/test/WorkflowTest.java#L45
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2518) Oozie is creating many files and directories in /tmp per day
[
https://issues.apache.org/jira/browse/OOZIE-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
abhishek bafna updated OOZIE-2518:
--
Fix Version/s: (was: trunk)
4.3.0
> Oozie is creating many files and directories in /tmp per day
>
>
> Key: OOZIE-2518
> URL: https://issues.apache.org/jira/browse/OOZIE-2518
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 4.2.0
>Reporter: Sergey Svinarchuk
>Assignee: Sergey Svinarchuk
> Labels: patch
> Fix For: 4.3.0
>
> Attachments: OOZIE-2518.patch
>
>
> Oozie generate many tmp files, I have over 2000 files in /tmp from oozie,
> that have been created over the last 7 days:
> {code}
> drwxr-xr-x 3 root root 22 Apr 22 14:19 /tmp/oozie-war-packing-24575
> drwxr-xr-x 3 root root 22 Apr 22 14:15 /tmp/oozie-war-packing-12211
> drwxr-xr-x 3 root root 22 Apr 22 14:10 /tmp/oozie-war-packing-32238
> drwxr-xr-x 2 test test6 Apr 22 13:54
> /tmp/oozie-test8457897583638353023.dir
> -rw-r--r-- 1 test test0 Apr 22 13:54 /tmp/ooziedb-2680155694357966339.sql
> drwxr-xr-x 2 test test6 Apr 22 13:53 /tmp/oozie-test142692242798969000.dir
> -rw-r--r-- 1 test test0 Apr 22 13:53 /tmp/ooziedb-9104680045020610294.sql
> drwxr-xr-x 2 test test6 Apr 22 13:53
> /tmp/oozie-test5885601734493698256.dir
> -rw-r--r-- 1 test test0 Apr 22 13:53 /tmp/ooziedb-4675739133078261538.sql
> drwxr-xr-x 2 test test6 Apr 22 13:53
> /tmp/oozie-test5565479595579419225.dir
> [root@test-node ext-conf]# ls -ldt /tmp/ooz* | tail
> drwxr-xr-x 3 root root 22 Apr 13 08:17 /tmp/oozie-war-packing-22705
> drwxr-xr-x 3 root root 22 Apr 13 08:16 /tmp/oozie-war-packing-18292
> drwxr-xr-x 3 root root 22 Apr 13 08:09 /tmp/oozie-war-packing-20289
> drwxr-xr-x 3 root root 22 Apr 13 08:09 /tmp/oozie-war-packing-18417
> drwxr-xr-x 3 root root 22 Apr 13 08:08 /tmp/oozie-war-packing-14712
> drwxr-xr-x 3 root root 22 Apr 13 08:05 /tmp/oozie-war-packing-25206
> drwxr-xr-x 3 root root 22 Apr 13 07:53 /tmp/oozie-war-packing-968
> drwxr-xr-x 3 root root 22 Apr 13 07:52 /tmp/oozie-war-packing-27771
> drwxr-xr-x 3 root root 22 Apr 13 07:45 /tmp/oozie-war-packing-30342
> drwxr-xr-x 3 root root 22 Apr 13 07:45 /tmp/oozie-war-packing-29086
> {code}
> This files can be moved into oozie tmp dir (something like this
> /tmp/oozieTmp) and also we could cleanup old files
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
