[GitHub] apex-core pull request #469: APEXCORE-644 get-app-package-operators with par...
Github user asfgit closed the pull request at: https://github.com/apache/apex-core/pull/469 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] apex-core pull request #471: APEXCORE-597 BufferServer needs to shutdown all...
GitHub user vrozov opened a pull request: https://github.com/apache/apex-core/pull/471 APEXCORE-597 BufferServer needs to shutdown all created execution services @PramodSSImmaneni, @tweise, @tushargosavi or @sandeshh Please review You can merge this pull request into a Git repository by running: $ git pull https://github.com/vrozov/apex-core APEXCORE-597 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/apex-core/pull/471.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #471 commit c376681dff0ac0abf68bf394c23981c8e9da409c Author: Vlad Rozov Date: 2017-02-13T05:17:34Z APEXCORE-597 BufferServer needs to shutdown all created execution services --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (APEXCORE-437) Output error message if address of Kafka broker is invalid
[ https://issues.apache.org/jira/browse/APEXCORE-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Munagala V. Ramanath closed APEXCORE-437. - Resolution: Invalid Recreated as APEXMALHAR-2405 > Output error message if address of Kafka broker is invalid > -- > > Key: APEXCORE-437 > URL: https://issues.apache.org/jira/browse/APEXCORE-437 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Munagala V. Ramanath >Assignee: Siyuan Hua > > When using the new Kafka 0.9 operator, if the zookeeper address is supplied > instead of the broker address, the App Master fails with no error messages > and the application gets stuck in the ACCEPTED state. > Should print an error message to the logs when this happens. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (APEXMALHAR-2405) Output error message if address of Kafka broker is invalid
[ https://issues.apache.org/jira/browse/APEXMALHAR-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15863070#comment-15863070 ] Munagala V. Ramanath commented on APEXMALHAR-2405: -- Previously created erroneously as APEXCORE-437. > Output error message if address of Kafka broker is invalid > -- > > Key: APEXMALHAR-2405 > URL: https://issues.apache.org/jira/browse/APEXMALHAR-2405 > Project: Apache Apex Malhar > Issue Type: Improvement > Components: adapters message bus >Reporter: Munagala V. Ramanath > > When using the new Kafka (0.9 API) operator, if the zookeeper address is > supplied instead of the broker address, the App Master fails with no error > messages and the application gets stuck in the ACCEPTED state. > Should print an error message to the logs when this happens. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (APEXMALHAR-2405) Output error message if address of Kafka broker is invalid
Munagala V. Ramanath created APEXMALHAR-2405: Summary: Output error message if address of Kafka broker is invalid Key: APEXMALHAR-2405 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2405 Project: Apache Apex Malhar Issue Type: Improvement Components: adapters message bus Reporter: Munagala V. Ramanath When using the new Kafka (0.9 API) operator, if the zookeeper address is supplied instead of the broker address, the App Master fails with no error messages and the application gets stuck in the ACCEPTED state. Should print an error message to the logs when this happens. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (APEXCORE-437) Output error message if address of Kafka broker is invalid
[ https://issues.apache.org/jira/browse/APEXCORE-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15863044#comment-15863044 ] Sandesh commented on APEXCORE-437: -- [~dtram] Can you please create this jira in Malhar? > Output error message if address of Kafka broker is invalid > -- > > Key: APEXCORE-437 > URL: https://issues.apache.org/jira/browse/APEXCORE-437 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Munagala V. Ramanath >Assignee: Siyuan Hua > > When using the new Kafka 0.9 operator, if the zookeeper address is supplied > instead of the broker address, the App Master fails with no error messages > and the application gets stuck in the ACCEPTED state. > Should print an error message to the logs when this happens. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-597) BufferServer needs to shutdown all created execution services
[ https://issues.apache.org/jira/browse/APEXCORE-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-597: Summary: BufferServer needs to shutdown all created execution services (was: Container does not terminate after shutdown request from master.) > BufferServer needs to shutdown all created execution services > - > > Key: APEXCORE-597 > URL: https://issues.apache.org/jira/browse/APEXCORE-597 > Project: Apache Apex Core > Issue Type: Bug > Components: Buffer Server >Reporter: Tushar Gosavi >Assignee: Vlad Rozov >Priority: Minor > > JVM does not shutdown cleanly after receiving shutdown request from Stram. > The issue may be because BufferServer is not shutdown. The issue is seen > after commit d1646e42bdf5594ef34070594733a7ca10123a3f > The demo application to recreate the issue is at > https://github.com/tushargosavi/apex-malhar/blob/shutdownapp/demos/pi/src/main/java/com/datatorrent/demos/pi/Application.java -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (APEXCORE-597) Container does not terminate after shutdown request from master.
[ https://issues.apache.org/jira/browse/APEXCORE-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov reassigned APEXCORE-597: --- Assignee: Vlad Rozov > Container does not terminate after shutdown request from master. > > > Key: APEXCORE-597 > URL: https://issues.apache.org/jira/browse/APEXCORE-597 > Project: Apache Apex Core > Issue Type: Bug > Components: Buffer Server >Reporter: Tushar Gosavi >Assignee: Vlad Rozov >Priority: Minor > > JVM does not shutdown cleanly after receiving shutdown request from Stram. > The issue may be because BufferServer is not shutdown. The issue is seen > after commit d1646e42bdf5594ef34070594733a7ca10123a3f > The demo application to recreate the issue is at > https://github.com/tushargosavi/apex-malhar/blob/shutdownapp/demos/pi/src/main/java/com/datatorrent/demos/pi/Application.java -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-597) Container does not terminate after shutdown request from master.
[ https://issues.apache.org/jira/browse/APEXCORE-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-597: Priority: Minor (was: Major) > Container does not terminate after shutdown request from master. > > > Key: APEXCORE-597 > URL: https://issues.apache.org/jira/browse/APEXCORE-597 > Project: Apache Apex Core > Issue Type: Bug > Components: Buffer Server >Reporter: Tushar Gosavi >Priority: Minor > > JVM does not shutdown cleanly after receiving shutdown request from Stram. > The issue may be because BufferServer is not shutdown. The issue is seen > after commit d1646e42bdf5594ef34070594733a7ca10123a3f > The demo application to recreate the issue is at > https://github.com/tushargosavi/apex-malhar/blob/shutdownapp/demos/pi/src/main/java/com/datatorrent/demos/pi/Application.java -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-597) Container does not terminate after shutdown request from master.
[ https://issues.apache.org/jira/browse/APEXCORE-597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-597: Component/s: Buffer Server > Container does not terminate after shutdown request from master. > > > Key: APEXCORE-597 > URL: https://issues.apache.org/jira/browse/APEXCORE-597 > Project: Apache Apex Core > Issue Type: Bug > Components: Buffer Server >Reporter: Tushar Gosavi >Priority: Minor > > JVM does not shutdown cleanly after receiving shutdown request from Stram. > The issue may be because BufferServer is not shutdown. The issue is seen > after commit d1646e42bdf5594ef34070594733a7ca10123a3f > The demo application to recreate the issue is at > https://github.com/tushargosavi/apex-malhar/blob/shutdownapp/demos/pi/src/main/java/com/datatorrent/demos/pi/Application.java -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (APEXCORE-601) provide log level in the event
[ https://issues.apache.org/jira/browse/APEXCORE-601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov resolved APEXCORE-601. - Resolution: Duplicate > provide log level in the event > --- > > Key: APEXCORE-601 > URL: https://issues.apache.org/jira/browse/APEXCORE-601 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Sanjay M Pujare > > Provide log levels for stram events. Such as INFO, WARN, ERROR > Eg: > 1. Start Container, Start Operator are INFO level events > 2. OperatorError is ERROR level event > 3. Stop Container, Stop Operator are WARN level events > These levels are used in the DT log file entries so the intent is to make the > same level available to the event consumer. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-600) Dynamic change of operator property not adhering to constraint annotations
[
https://issues.apache.org/jira/browse/APEXCORE-600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vlad Rozov updated APEXCORE-600:
Priority: Minor (was: Major)
> Dynamic change of operator property not adhering to constraint annotations
> --
>
> Key: APEXCORE-600
> URL: https://issues.apache.org/jira/browse/APEXCORE-600
> Project: Apache Apex Core
> Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Francis Fernandes
>Assignee: Francis Fernandes
>Priority: Minor
>
> Create an operator with a field having some constraint
> {code}
> @ Min(0)
> protected int nonNegativeVal = 10;
> {code}
> When launching the application the constraint is adhered and an exception is
> thrown which prevents the application from launching:
> {code}
>
> dt.operator.test.prop.nonNegativeVal
> -10
>
> {code}
> The application is launched with a valid value and enters into running state.
> The constraint is not adhered to on updating the property with an invalid
> (negative) value.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (APEXCORE-606) Suggestion: Optimise Kryo Output
[ https://issues.apache.org/jira/browse/APEXCORE-606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-606: Priority: Minor (was: Major) > Suggestion: Optimise Kryo Output > > > Key: APEXCORE-606 > URL: https://issues.apache.org/jira/browse/APEXCORE-606 > Project: Apache Apex Core > Issue Type: New Feature >Reporter: bright chen >Assignee: bright chen >Priority: Minor > > The kryo Output has some limitation > - The size of the data is limited. kryo write data to the buffer, it will > throw the overflow exception if the data exceed the size > - The Output.toBytes() will copy the data to temporary buffer and output, > it will decrease the performance and introduce garbage collection. > When I was tuning Spillable Data structure and Manage State. I create a > mechanism to share and reuse the memory to avoid above problem. And it can > be reused in core serialization with small change. Please see jira: > https://issues.apache.org/jira/browse/APEXMALHAR-2190 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-607) Add unit tests for BlackListedBasedResourceRequestHandler
[ https://issues.apache.org/jira/browse/APEXCORE-607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-607: Priority: Minor (was: Major) > Add unit tests for BlackListedBasedResourceRequestHandler > - > > Key: APEXCORE-607 > URL: https://issues.apache.org/jira/browse/APEXCORE-607 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Sandesh >Priority: Minor > > There is no unit test coverage for the following class > BlackListedBasedResourceRequestHandler. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-608) Streaming Containers use stale RPC proxy after connection is closed
[ https://issues.apache.org/jira/browse/APEXCORE-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-608: Priority: Minor (was: Major) > Streaming Containers use stale RPC proxy after connection is closed > --- > > Key: APEXCORE-608 > URL: https://issues.apache.org/jira/browse/APEXCORE-608 > Project: Apache Apex Core > Issue Type: Bug >Reporter: Vlad Rozov >Assignee: Vlad Rozov >Priority: Minor > > When Application is killed and Application Master is terminated, Streaming > Containers initiate container exit sequence and use disconnected RPC proxy to > report errors back to already terminated Application Master. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-609) Drop DataList blocks when BufferServer spooling is disabled
[ https://issues.apache.org/jira/browse/APEXCORE-609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-609: Summary: Drop DataList blocks when BufferServer spooling is disabled (was: Container runs out of memory when buffer spooling is disabled) > Drop DataList blocks when BufferServer spooling is disabled > --- > > Key: APEXCORE-609 > URL: https://issues.apache.org/jira/browse/APEXCORE-609 > Project: Apache Apex Core > Issue Type: Improvement > Components: Buffer Server >Reporter: Pramod Immaneni >Assignee: Pramod Immaneni > > When buffer spooling is disabled and the specified buffer in memory limit is > reached, because of fault tolerance requirements, additional memory is > allocated to store the new data. This causes container to run out of memory. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-609) Container runs out of memory when buffer spooling is disabled
[ https://issues.apache.org/jira/browse/APEXCORE-609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-609: Issue Type: Improvement (was: Bug) > Container runs out of memory when buffer spooling is disabled > - > > Key: APEXCORE-609 > URL: https://issues.apache.org/jira/browse/APEXCORE-609 > Project: Apache Apex Core > Issue Type: Improvement > Components: Buffer Server >Reporter: Pramod Immaneni >Assignee: Pramod Immaneni > > When buffer spooling is disabled and the specified buffer in memory limit is > reached, because of fault tolerance requirements, additional memory is > allocated to store the new data. This causes container to run out of memory. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-615) Out of sequence tuple exception - kill the whole app
[ https://issues.apache.org/jira/browse/APEXCORE-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-615: Priority: Minor (was: Major) > Out of sequence tuple exception - kill the whole app > > > Key: APEXCORE-615 > URL: https://issues.apache.org/jira/browse/APEXCORE-615 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Sandesh >Assignee: Sandesh >Priority: Minor > > In case of out of sequence tuple, killing the containers will not solve the > problem as container will continue to fail after recovery. Killing the app > is a better option. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-619) recovery window id in future for terminating state less operators during relaunch.
[ https://issues.apache.org/jira/browse/APEXCORE-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-619: Priority: Minor (was: Major) > recovery window id in future for terminating state less operators during > relaunch. > -- > > Key: APEXCORE-619 > URL: https://issues.apache.org/jira/browse/APEXCORE-619 > Project: Apache Apex Core > Issue Type: Bug >Reporter: Tushar Gosavi >Assignee: Tushar Gosavi >Priority: Minor > > With following DAG > A -> B -> C > C is a stateless operator. If this application is killed and restarted after > long time between kill and restart, then recovery window id of C is too high > compare to A and B. This is because recovery windowid is computed from > current timestamp for stateless operators in updateRecoveryCheckpoints. > The problem this causes > - Operator C does not process any data till windowId of B reached to recovery > window id of C. > - If other operators are not able to keep up then C gets killed because it is > detected as blocked operator. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-620) Support Fat Jar without POM, allowing Gradle and other build systems
[ https://issues.apache.org/jira/browse/APEXCORE-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-620: Priority: Minor (was: Major) > Support Fat Jar without POM, allowing Gradle and other build systems > > > Key: APEXCORE-620 > URL: https://issues.apache.org/jira/browse/APEXCORE-620 > Project: Apache Apex Core > Issue Type: New Feature >Reporter: Jose Casillas >Priority: Minor > > For a project that is built with Gradle to be used in Apex, we must convert > our suite of libraries to Maven or resort to: > apex> launch -ignorepom -libjars x,y,z,... > Can the launcher simply accept a jar and include all of the dependency jars > within it, without requiring a POM or explicit declaration with -libjars? > As a stop-gap, allowing -libjars to support a directory of dependencies would > free us from specifying each one in the launch statement. > The fat jar support would allow build systems other than Maven to power Apex > applications. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-621) populate TIMEOUT_WINDOW_COUNT for thread local operators from downstreams.
[ https://issues.apache.org/jira/browse/APEXCORE-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-621: Priority: Minor (was: Major) > populate TIMEOUT_WINDOW_COUNT for thread local operators from downstreams. > -- > > Key: APEXCORE-621 > URL: https://issues.apache.org/jira/browse/APEXCORE-621 > Project: Apache Apex Core > Issue Type: Improvement >Reporter: Tushar Gosavi >Assignee: Deepak Narkhede >Priority: Minor > > A -> B -> C -> D > In above dag if we have set TIMEOUT_WINDOW_COUNT on 'C' and 'B' and 'C' are > in thread local, then 'B' uses default TIMEOUT_WINDOW_COUNT attribute and > marked as blocked opeator while C is performing a time cosuming operation. > The problem is more visible when operator B is partitioned and unifiers are > deployed thread local to C, in this case unifiers are declared are blocked, > and users need to remember to set TIMEOUT_WINDOW_COUNT on unifiers. > Instead platform could inherit TIMEOUT_WINDOW_COUNT attribute from downstream > operator in case of threadlocal/container local case to avoid getting > detected as blocked early. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-622) event files are overwritten when master restarts.
[ https://issues.apache.org/jira/browse/APEXCORE-622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-622: Priority: Minor (was: Major) > event files are overwritten when master restarts. > - > > Key: APEXCORE-622 > URL: https://issues.apache.org/jira/browse/APEXCORE-622 > Project: Apache Apex Core > Issue Type: Bug >Reporter: Tushar Gosavi >Priority: Minor > > The events file written in application_path/events are overwritten in case of > master restart. The important debugging information is lost while debugging > cause of master restart. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-624) Shutdown does not work because of incorrect logic in the AppMaster
[ https://issues.apache.org/jira/browse/APEXCORE-624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-624: Priority: Minor (was: Major) > Shutdown does not work because of incorrect logic in the AppMaster > -- > > Key: APEXCORE-624 > URL: https://issues.apache.org/jira/browse/APEXCORE-624 > Project: Apache Apex Core > Issue Type: Bug >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare >Priority: Minor > > com.datatorrent.stram.StreamingAppMasterService.execute() calculates > numRequestedContainers incorrectly in some cases (e.g. RM container > allocation failure) which prevents an application from shutting down when it > is requested externally. An example is where we ask RM to remove previous > container allocation request (where the count should be decremented but is > NOT) and add a new one (where the count should be and IS incremented). > Another example is the "alreadyAllocated" case where we release the container > and still increment numRequestedContainers which seems wrong. > This bug is showing up in multiple Apex deployments. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (APEXCORE-628) One Yarn with Multiple Apex Applications
[ https://issues.apache.org/jira/browse/APEXCORE-628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov closed APEXCORE-628. --- Resolution: Not A Bug > One Yarn with Multiple Apex Applications > > > Key: APEXCORE-628 > URL: https://issues.apache.org/jira/browse/APEXCORE-628 > Project: Apache Apex Core > Issue Type: Bug >Reporter: santhoshi > > Launching more than one (multiple) apex engine in one node with multiple > terminals and one yarn running. > 8042 port is used by one application which is making other application not to > run as port 8042 is already in use. > How to run more than one apex engine with same yarn? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (APEXCORE-628) One Yarn with Multiple Apex Applications
[ https://issues.apache.org/jira/browse/APEXCORE-628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov updated APEXCORE-628: Fix Version/s: (was: 3.6.0) > One Yarn with Multiple Apex Applications > > > Key: APEXCORE-628 > URL: https://issues.apache.org/jira/browse/APEXCORE-628 > Project: Apache Apex Core > Issue Type: Bug >Reporter: santhoshi > > Launching more than one (multiple) apex engine in one node with multiple > terminals and one yarn running. > 8042 port is used by one application which is making other application not to > run as port 8042 is already in use. > How to run more than one apex engine with same yarn? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Reopened] (APEXCORE-628) One Yarn with Multiple Apex Applications
[ https://issues.apache.org/jira/browse/APEXCORE-628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vlad Rozov reopened APEXCORE-628: - > One Yarn with Multiple Apex Applications > > > Key: APEXCORE-628 > URL: https://issues.apache.org/jira/browse/APEXCORE-628 > Project: Apache Apex Core > Issue Type: Bug >Reporter: santhoshi > Fix For: 3.6.0 > > > Launching more than one (multiple) apex engine in one node with multiple > terminals and one yarn running. > 8042 port is used by one application which is making other application not to > run as port 8042 is already in use. > How to run more than one apex engine with same yarn? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (APEXCORE-642) Catch all exceptions in StreamingAppMasterService.serviceInit() and create a StramEvent
[ https://issues.apache.org/jira/browse/APEXCORE-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862989#comment-15862989 ] Vlad Rozov commented on APEXCORE-642: - I'd prefer to log exception in the main as it is the top most method and logging exception in serviceInit will lead to double logging as serviceInit() is not called directly by StreamingAppMaster, so there may be exceptions raised by a middle man and it will be impossible to distinguish exceptions raised by a middle man from exceptions raised by StreamingAppMasterService and already logged. It is OK to create StramEvent in the serviceInit() assuming that there is a way to record the event. As events are reported asynchronously, reporting an event and terminating app master will likely lead to the event being lost. > Catch all exceptions in StreamingAppMasterService.serviceInit() and create a > StramEvent > --- > > Key: APEXCORE-642 > URL: https://issues.apache.org/jira/browse/APEXCORE-642 > Project: Apache Apex Core > Issue Type: Bug >Reporter: Sanjay M Pujare >Assignee: Sanjay M Pujare >Priority: Minor > Original Estimate: 48h > Remaining Estimate: 48h > > When the AM (Stram) starts, it executes > StreamingAppMasterService.serviceInit() to perform service initialization as > per the Hadoop service contract. In this, the Stram deserializes the DAG > which can fail (e.g. bad jar versions or other deserialization issues) and > any exception is thrown is not logged in Apex log files or events. It is > proposed to catch these exceptions and log them to the dt log file as well as > Stram events so the Apex user knows about these exceptions. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
