Towards Apache Apex 3.6.0 release
Hi, It has been four month since 3.5.0 Apex Core release. There are several new features added to core after 3.5.0. I would like to propose the 3.6.0 release of Apex Core, to make these features available to users. The list of issues fixed in 3.6.0 are: https://issues.apache.org/jira/issues/?jql=project%20% 3D%20APEXCORE%20AND%20status%20in%20(Resolved%2C%20Closed)% 20AND%20fixVersion%20%3D%203.6.0%20ORDER%20BY%20status%20ASC Apart from above JIRAs, which bug-fixes/features people will like to see in this release. If you feel your JIRA should be included then please set fix version to 3.6.0 with estimated time for work completion, also discuss it here. Some of the pending pull requests could be incorporated in the release. I feel following JIRA should be part of release APEXCORE-649, APEXCORE-678. Let me know about your thoughts. Thanks, Tushar.
[jira] [Closed] (APEXCORE-561) Container info needs to be persisted even after application has been killed
[ https://issues.apache.org/jira/browse/APEXCORE-561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay M Pujare closed APEXCORE-561. Resolution: Won't Do This is no longer needed and will use the timeline server as suggested > Container info needs to be persisted even after application has been killed > --- > > Key: APEXCORE-561 > URL: https://issues.apache.org/jira/browse/APEXCORE-561 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > > This is required so that info can be displayed (by Apex cli for example) even > for killed apps. > At least one piece of info missing, which is the finishedTime. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Reopened] (APEXCORE-561) Container info needs to be persisted even after application has been killed
[ https://issues.apache.org/jira/browse/APEXCORE-561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay M Pujare reopened APEXCORE-561: -- Need to be closed with correct type > Container info needs to be persisted even after application has been killed > --- > > Key: APEXCORE-561 > URL: https://issues.apache.org/jira/browse/APEXCORE-561 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare > > This is required so that info can be displayed (by Apex cli for example) even > for killed apps. > At least one piece of info missing, which is the finishedTime. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (APEXCORE-493) Purge the checkpoints from the containers
[ https://issues.apache.org/jira/browse/APEXCORE-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov resolved APEXCORE-493. - Resolution: Won't Do > Purge the checkpoints from the containers > - > > Key: APEXCORE-493 > URL: https://issues.apache.org/jira/browse/APEXCORE-493 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Sandesh >Assignee: Sandesh > > Currently, purging of the checkpoints happens from the Stram, but it could be > done in StreamingContainers. > Idea is to reduce the responsibilities of the Stram and to distribute the > work across the containers. > Purging during Dynamic partitions can still be done in Stram. > Inspiration - > https://issues.apache.org/jira/browse/APEXCORE-222 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (APEXCORE-683) Apex client should support application packages on HDFS
[
https://issues.apache.org/jira/browse/APEXCORE-683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vlad Rozov resolved APEXCORE-683.
-
Resolution: Implemented
Fix Version/s: 3.6.0
> Apex client should support application packages on HDFS
> ---
>
> Key: APEXCORE-683
> URL: https://issues.apache.org/jira/browse/APEXCORE-683
> Project: Apache Apex Core
> Issue Type: Improvement
>Reporter: Vlad Rozov
>Assignee: Vlad Rozov
> Fix For: 3.6.0
>
>
> apex client supports only local file system, it is natural for native yarn
> application to support hdfs and other dfs file systems:
> {noformat}
> hdfs dfs -ls hdfs:///user/vrozov
> Found 4 items
> drwx-- - vrozov supergroup 0 2016-12-24 16:00
> hdfs:///user/vrozov/.Trash
> drwxrwxrwx - vrozov supergroup 0 2017-02-28 19:20
> hdfs:///user/vrozov/.flink
> -rw-r--r-- 3 vrozov supergroup 18862977 2017-03-25 09:20
> hdfs:///user/vrozov/appPackage.apa
> drwxrwxrwx - vrozov supergroup 0 2016-10-25 21:49
> hdfs:///user/vrozov/datatorrent
> [vrozov@node0 apex]$ apex
> Apex CLI 3.6.0-SNAPSHOT 25.03.2017 @ 09:17:45 PDT rev: b4a4e05 branch: master
> apex> get-app-package-info hdfs:///user/vrozov/appPackage.apa
> java.io.FileNotFoundException: hdfs:/user/vrozov/appPackage.apa (No such file
> or directory)
> at java.util.zip.ZipFile.open(Native Method)
> at java.util.zip.ZipFile.(ZipFile.java:215)
> at java.util.zip.ZipFile.(ZipFile.java:145)
> at java.util.jar.JarFile.(JarFile.java:154)
> at java.util.jar.JarFile.(JarFile.java:118)
> at com.datatorrent.stram.client.AppPackage.(AppPackage.java:185)
> at com.datatorrent.stram.client.AppPackage.(AppPackage.java:258)
> at
> com.datatorrent.stram.cli.ApexCli.newAppPackageInstance(ApexCli.java:467)
> at
> com.datatorrent.stram.cli.ApexCli$GetAppPackageInfoCommand.execute(ApexCli.java:3502)
> at com.datatorrent.stram.cli.ApexCli$3.run(ApexCli.java:1520)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (APEXCORE-678) Shutdown of application should start from input nodes
[
https://issues.apache.org/jira/browse/APEXCORE-678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tushar Gosavi updated APEXCORE-678:
---
Fix Version/s: 3.6.0
> Shutdown of application should start from input nodes
> -
>
> Key: APEXCORE-678
> URL: https://issues.apache.org/jira/browse/APEXCORE-678
> Project: Apache Apex Core
> Issue Type: Bug
>Reporter: Bhupesh Chawda
>Assignee: Bhupesh Chawda
> Fix For: 3.6.0
>
>
> Streaming container calls shutdown() for all nodes instead of just input
> nodes.
> {code}
> private void stopInputNodes()
> {
> for (Entry e : nodes.entrySet()) {
> Node node = e.getValue();
> if (node instanceof InputNode) {
> final Thread thread = e.getValue().context.getThread();
> if (thread == null || !thread.isAlive()) {
> continue;
> }
> }
> node.shutdown(true);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] apex-core pull request #500: Apexcore 550.better error message on java versi...
Github user oliverwnk closed the pull request at: https://github.com/apache/apex-core/pull/500 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (APEXCORE-699) Investigate versioning for plugins
[ https://issues.apache.org/jira/browse/APEXCORE-699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pramod Immaneni reassigned APEXCORE-699: Assignee: Tushar Gosavi > Investigate versioning for plugins > -- > > Key: APEXCORE-699 > URL: https://issues.apache.org/jira/browse/APEXCORE-699 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Pramod Immaneni >Assignee: Tushar Gosavi > > Having versioning information in the plugin would help in dealing with > compatibility with older plugins when the plugin interface changes. This > needs to be investigated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (APEXCORE-699) Investigate versioning for plugins
Pramod Immaneni created APEXCORE-699: Summary: Investigate versioning for plugins Key: APEXCORE-699 URL: https://issues.apache.org/jira/browse/APEXCORE-699 Project: Apache Apex Core Issue Type: Improvement Reporter: Pramod Immaneni Having versioning information in the plugin would help in dealing with compatibility with older plugins when the plugin interface changes. This needs to be investigated. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (APEXCORE-456) Explicitly limit Server.Subscriber to one way communication
[ https://issues.apache.org/jira/browse/APEXCORE-456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov resolved APEXCORE-456. - Resolution: Fixed Fix Version/s: 3.6.0 > Explicitly limit Server.Subscriber to one way communication > --- > > Key: APEXCORE-456 > URL: https://issues.apache.org/jira/browse/APEXCORE-456 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Vlad Rozov >Assignee: Vlad Rozov > Fix For: 3.6.0 > > > In the DAG data always flows from upstream to downstream operators and buffer > server Server.Subscriber class may explicitly shutdown input stream, does not > need to allocate read buffer and listen to OP_READ events. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-660) Documentation for Control tuple support changes
[ https://issues.apache.org/jira/browse/APEXCORE-660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-660: Fix Version/s: (was: 3.6.0) > Documentation for Control tuple support changes > --- > > Key: APEXCORE-660 > URL: https://issues.apache.org/jira/browse/APEXCORE-660 > Project: Apache Apex Core > Issue Type: Sub-task >Reporter: Bhupesh Chawda >Assignee: Bhupesh Chawda > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
