[jira] [Commented] (OOZIE-1178) Workflow Application Master in YARN
[
https://issues.apache.org/jira/browse/OOZIE-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581199#comment-13581199
]
Tianyou Li commented on OOZIE-1178:
---
Hi [~acmurthy],
Is there any progress on submitting YAPP proposal to Apache Incubator? The team
here is hoping to participate in YAPP project development and make
contributions. By chance I saw project
Tez{http://wiki.apache.org/incubator/TezProposal}, do you see any relationship
between YAPP and Tez? Thanks.
Workflow Application Master in YARN
---
Key: OOZIE-1178
URL: https://issues.apache.org/jira/browse/OOZIE-1178
Project: Oozie
Issue Type: New Feature
Reporter: Bo Wang
Attachments: MAPREDUCE-4495-v1.1.patch, MAPREDUCE-4495-v1.patch,
MapReduceWorkflowAM.pdf, yapp_proposal.txt
It is useful to have a workflow application master, which will be capable of
running a DAG of jobs. The workflow client submits a DAG request to the AM
and then the AM will manage the life cycle of this application in terms of
requesting the needed resources from the RM, and starting, monitoring and
retrying the application's individual tasks.
Compared to running Oozie with the current MapReduce Application Master,
these are some of the advantages:
- Less number of consumed resources, since only one application master will
be spawned for the whole workflow.
- Reuse of resources, since the same resources can be used by multiple
consecutive jobs in the workflow (no need to request/wait for resources for
every individual job from the central RM).
- More optimization opportunities in terms of collective resource requests.
- Optimization opportunities in terms of rewriting and composing jobs in the
workflow (e.g. pushing down Mappers).
- This Application Master can be reused/extended by higher systems like Pig
and hive to provide an optimized way of running their workflows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OOZIE-1223) too many extjs cookies can cause the web UI to stop responding
[ https://issues.apache.org/jira/browse/OOZIE-1223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13581384#comment-13581384 ] Eduardo Ferreira commented on OOZIE-1223: - Hi there, here are more details about the problem I experience as I browse around the Oozie Web Console until it stops responding. The UI itself still responds in the sense of allowing me to switch tabs (Workflow Jobs, Coordinator Jobs, etc) and sort entries by clicking on the subtabs (Job ID, Name, Status, etc). The problem is that the UI does not bring up any secondary windows or frames or whatever, i.e. when I click an entry (job, action, etc) under Coordinator Jobs for example, I expect it to display a window with the list of instances and respective status corresponding to that coordinator, but it does not display anything. When I clear cookies corresponding to my Oozie server, the problem goes away. The number of cookies varies from time to time, but here's the list of cookies I see in the browser right now (13 cookies): Name: ys-ext-comp-1002 Content: o%3Acolumns%3Da%253Ao%25253Aid%25253Ds%2525253Anumberer%25255Ewidth%25253Dn%2525253A23%255Eo%25253Aid%25253Ds%2525253Aid%25255Ewidth%25253Dn%2525253A190%255Eo%25253Aid%25253Dn%2525253A2%25255Ewidth%25253Dn%2525253A100%255Eo%25253Aid%25253Dn%2525253A3%25255Ewidth%25253Dn%2525253A70%255Eo%25253Aid%25253Dn%2525253A4%25255Ewidth%25253Dn%2525253A30%255Eo%25253Aid%25253Dn%2525253A5%25255Ewidth%25253Dn%2525253A60%255Eo%25253Aid%25253Dn%2525253A6%25255Ewidth%25253Dn%2525253A60%255Eo%25253Aid%25253Dn%2525253A7%25255Ewidth%25253Dn%2525253A170%255Eo%25253Aid%25253Dn%2525253A8%25255Ewidth%25253Dn%2525253A170%255Eo%25253Aid%25253Dn%2525253A9%25255Ewidth%25253Dn%2525253A170%255Eo%25253Aid%25253Dn%2525253A10%25255Ewidth%25253Dn%2525253A170%5Esort%3Do%253Afield%253Ds%25253AappName%255Edirection%253Ds%25253ADESC Domain: aspenuzi1.cnn.vgtf.net Path: / Send for: Any kind of connection Accessible to script: Yes Created:Tuesday, February 19, 2013 10:34:47 AM Expires:Tuesday, February 26, 2013 10:34:38 AM Name: ys-ext-comp-1006 Content: o%3Acolumns%3Da%253Ao%25253Aid%25253Ds%2525253Anumberer%25255Ewidth%25253Dn%2525253A23%255Eo%25253Aid%25253Ds%2525253Aid%25255Ewidth%25253Dn%2525253A230%255Eo%25253Aid%25253Dn%2525253A2%25255Ewidth%25253Dn%2525253A246%255Eo%25253Aid%25253Dn%2525253A3%25255Ewidth%25253Dn%2525253A80%255Eo%25253Aid%25253Dn%2525253A4%25255Ewidth%25253Dn%2525253A80%255Eo%25253Aid%25253Dn%2525253A5%25255Ewidth%25253Dn%2525253A80%255Eo%25253Aid%25253Dn%2525253A6%25255Ewidth%25253Dn%2525253A70%255Eo%25253Aid%25253Dn%2525253A7%25255Ewidth%25253Dn%2525253A60%255Eo%25253Aid%25253Dn%2525253A8%25255Ewidth%25253Dn%2525253A170%255Eo%25253Aid%25253Dn%2525253A9%25255Ewidth%25253Dn%2525253A170%5Esort%3Do%253Afield%253Ds%25253AcoordJobName%255Edirection%253Ds%25253AASC Domain: aspenuzi1.cnn.vgtf.net Path: / Send for: Any kind of connection Accessible to script: Yes Created:Tuesday, February 19, 2013 10:34:50 AM Expires:Tuesday, February 26, 2013 10:34:38 AM Name: ys-ext-comp-1142 Content: o%3Acolumns%3Da%253Ao%25253Aid%25253Ds%2525253Anumberer%25255Ewidth%25253Dn%2525253A23%255Eo%25253Aid%25253Ds%2525253Aid%25255Ewidth%25253Dn%2525253A240%255Eo%25253Aid%25253Dn%2525253A2%25255Ewidth%25253Dn%2525253A80%255Eo%25253Aid%25253Dn%2525253A3%25255Ewidth%25253Dn%2525253A220%255Eo%25253Aid%25253Dn%2525253A4%25255Ewidth%25253Dn%2525253A80%255Eo%25253Aid%25253Dn%2525253A5%25255Ewidth%25253Dn%2525253A160%255Eo%25253Aid%25253Dn%2525253A6%25255Ewidth%25253Dn%2525253A160%255Eo%25253Aid%25253Dn%2525253A7%25255Ewidth%25253Dn%2525253A170%5Esort%3Do%253Afield%253Ds%25253Astatus%255Edirection%253Ds%25253AASC Domain: aspenuzi1.cnn.vgtf.net Path: / Send for: Any kind of connection Accessible to script: Yes Created:Wednesday, February 13, 2013 3:20:44 PM Expires:Wednesday, February 20, 2013 3:20:41 PM Name: ys-ext-comp-1175 Content: o%3Acolumns%3Da%253Ao%25253Aid%25253Ds%2525253Anumberer%25255Ewidth%25253Dn%2525253A23%255Eo%25253Aid%25253Ds%2525253Aid%25255Ewidth%25253Dn%2525253A240%255Eo%25253Aid%25253Dn%2525253A2%25255Ewidth%25253Dn%2525253A80%255Eo%25253Aid%25253Dn%2525253A3%25255Ewidth%25253Dn%2525253A220%255Eo%25253Aid%25253Dn%2525253A4%25255Ewidth%25253Dn%2525253A80%255Eo%25253Aid%25253Dn%2525253A5%25255Ewidth%25253Dn%2525253A160%255Eo%25253Aid%25253Dn%2525253A6%25255Ewidth%25253Dn%2525253A160%255Eo%25253Aid%25253Dn%2525253A7%25255Ewidth%25253Dn%2525253A170%5Esort%3Do%253Afield%253Ds%25253Astatus%255Edirection%253Ds%25253AASC Domain: aspenuzi1.cnn.vgtf.net Path: / Send for: Any kind of connection Accessible to script: Yes Created:Wednesday, February 13, 2013 3:20:59 PM Expires:Wednesday, February 20, 2013 3:20:57 PM Name: ys-ext-comp-1208 Content:
[jira] [Updated] (OOZIE-1045) Parameterize unresolved-instances tag currently hardcoded
[ https://issues.apache.org/jira/browse/OOZIE-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated OOZIE-1045: -- Issue Type: Improvement (was: Sub-task) Parent: (was: OOZIE-561) Parameterize unresolved-instances tag currently hardcoded --- Key: OOZIE-1045 URL: https://issues.apache.org/jira/browse/OOZIE-1045 Project: Oozie Issue Type: Improvement Reporter: Ryota Egashira Assignee: Ryota Egashira Fix For: trunk Attachments: OOZIE-1045.patch currently unresolved-instances is hardcoded. Use static variable for it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OOZIE-1207) Optimize current EL resolution in case of start-instance and end-instance
[
https://issues.apache.org/jira/browse/OOZIE-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated OOZIE-1207:
--
Attachment: OOZIE-1207-1.patch
Attaching patch based on hcat-intre branch. Will rebase patch to trunk if
required once hcat-intre is merged with trunk.
Optimize current EL resolution in case of start-instance and end-instance
-
Key: OOZIE-1207
URL: https://issues.apache.org/jira/browse/OOZIE-1207
Project: Oozie
Issue Type: Improvement
Reporter: Rohini Palaniswamy
Attachments: OOZIE-1207-1.patch
start-instance$
{coord:current(-23)}
/start-instance
end-instance$
{coord:current(0)}
/end-instance
Assuming all 24 instances are available, Oozie makes 300(1+2+3+..24) calls
instead of 24 calls. This leads to some very high CPU usage when number of
current instances is high.
OOZIE-1073 fixes similar issue for latest and future
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-1036) Utility class to parse HCat URI
[ https://issues.apache.org/jira/browse/OOZIE-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari resolved OOZIE-1036. -- Resolution: Fixed Utility class to parse HCat URI --- Key: OOZIE-1036 URL: https://issues.apache.org/jira/browse/OOZIE-1036 Project: Oozie Issue Type: Sub-task Reporter: Ryota Egashira Assignee: Ryota Egashira Labels: hcatalog, oozie Fix For: trunk Attachments: OOZIE-1036-v2.patch uri-template for HCat uri-template hcat://[HcatServer:Port]/db/table/?datestamp=$YEAR$MONTH$DAY$HOUR;region=us/uri-template This utility class parses HCat URI, does checking,and store information(server, db, table, partitions) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-1033) Generic utility class to register/unregister a JMS message handler
[ https://issues.apache.org/jira/browse/OOZIE-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari resolved OOZIE-1033. -- Resolution: Fixed Generic utility class to register/unregister a JMS message handler -- Key: OOZIE-1033 URL: https://issues.apache.org/jira/browse/OOZIE-1033 Project: Oozie Issue Type: Sub-task Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Fix For: trunk This task includes: 1. Provide two methods to register and unregister a message ;listener with generic message handler for a specific topic. 2. Use the 'default' end point , when no endpoint is passed. 3. A new generic message handler interface that anyone can implement and pass for message handling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-1044) Parameterize uris tag currently hardcoded
[ https://issues.apache.org/jira/browse/OOZIE-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari resolved OOZIE-1044. -- Resolution: Fixed Parameterize uris tag currently hardcoded --- Key: OOZIE-1044 URL: https://issues.apache.org/jira/browse/OOZIE-1044 Project: Oozie Issue Type: Sub-task Reporter: Ryota Egashira Assignee: Ryota Egashira Fix For: trunk Attachments: OOZIE-1044-v2.patch currently uris is hardcoded. Use static variable for it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-1043) Add logic to register to Missing Dependency Structure in coord action materialization
[ https://issues.apache.org/jira/browse/OOZIE-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari resolved OOZIE-1043. -- Resolution: Fixed Add logic to register to Missing Dependency Structure in coord action materialization - Key: OOZIE-1043 URL: https://issues.apache.org/jira/browse/OOZIE-1043 Project: Oozie Issue Type: Sub-task Reporter: Ryota Egashira Assignee: Ryota Egashira Labels: hcatalog, oozie Fix For: trunk Attachments: OOZIE-1043-v5.patch in materialization of coord action, following logic needs to be added - check existence of partition against HCat - register info of missing partition to Missing Dependency Data Structure -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-1061) Add new EL function to retrieve HCatalog server, DB and table name
[ https://issues.apache.org/jira/browse/OOZIE-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari resolved OOZIE-1061. -- Resolution: Fixed Add new EL function to retrieve HCatalog server, DB and table name -- Key: OOZIE-1061 URL: https://issues.apache.org/jira/browse/OOZIE-1061 Project: Oozie Issue Type: Sub-task Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Fix For: trunk -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-1068) Metadata Accessor service for HCatalog
[ https://issues.apache.org/jira/browse/OOZIE-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari resolved OOZIE-1068. -- Resolution: Fixed Metadata Accessor service for HCatalog -- Key: OOZIE-1068 URL: https://issues.apache.org/jira/browse/OOZIE-1068 Project: Oozie Issue Type: Sub-task Environment: Provide Metadata accessor service and meta data client wrapper. Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Fix For: trunk Provide Metadata accessor service and meta data client wrapper. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-1168) Reliability and retry for JMS connections
[ https://issues.apache.org/jira/browse/OOZIE-1168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari resolved OOZIE-1168. -- Resolution: Fixed Fix Version/s: trunk Reliability and retry for JMS connections - Key: OOZIE-1168 URL: https://issues.apache.org/jira/browse/OOZIE-1168 Project: Oozie Issue Type: Sub-task Reporter: Rohini Palaniswamy Assignee: Rohini Palaniswamy Fix For: trunk Need to handle connection failures and reconnect (backed off retry) to JMS when listening to notifications. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-1086) Command to check the missing partitions directly against HCatalog server
[ https://issues.apache.org/jira/browse/OOZIE-1086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari resolved OOZIE-1086. -- Resolution: Fixed Command to check the missing partitions directly against HCatalog server Key: OOZIE-1086 URL: https://issues.apache.org/jira/browse/OOZIE-1086 Project: Oozie Issue Type: Sub-task Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Fix For: trunk The same will be used in multiple scenarios: 1. As part of action materialization. 2. Recovery service -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OOZIE-1217) Address review comments in OOZIE-1210
[ https://issues.apache.org/jira/browse/OOZIE-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virag Kothari resolved OOZIE-1217. -- Resolution: Fixed Address review comments in OOZIE-1210 - Key: OOZIE-1217 URL: https://issues.apache.org/jira/browse/OOZIE-1217 Project: Oozie Issue Type: Sub-task Reporter: Rohini Palaniswamy Assignee: Rohini Palaniswamy Fix For: trunk Attachments: OOZIE-1217-1.patch, OOZIE-1217-2.patch, OOZIE-1217-3.patch Address review comments in https://issues.apache.org/jira/browse/OOZIE-1200?focusedCommentId=13574146page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13574146 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
