[GitHub] flink pull request #3276: [FLINK-5718] [core] TaskManagers exit the JVM on f...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/3276


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #3276: [FLINK-5718] [core] TaskManagers exit the JVM on f...

[StephanEwen]

Github user StephanEwen commented on a diff in the pull request:

https://github.com/apache/flink/pull/3276#discussion_r100128721
  
--- Diff: docs/setup/config.md ---
@@ -86,7 +86,7 @@ The default fraction for managed memory can be adjusted 
using the `taskmanager.m
 
 - `taskmanager.memory.segment-size`: The size of memory buffers used by 
the memory manager and the network stack in bytes (DEFAULT: 32768 (= 32 
KiBytes)).
 
-- `taskmanager.memory.preallocate`: Can be either of `true` or `false`. 
Specifies whether task managers should allocate all managed memory when 
starting up. (DEFAULT: false). When `taskmanager.memory.off-heap` is set to 
`true`, then it is advised that this configuration is also set to `true`.  If 
this configuration is set to `false` cleaning up of the allocated offheap 
memory happens only when the configured JVM parameter MaxDirectMemorySize is 
reached by triggering a full GC.
+- `taskmanager.memory.preallocate`: Can be either of `true` or `false`. 
Specifies whether task managers should allocate all managed memory when 
starting up. (DEFAULT: false). When `taskmanager.memory.off-heap` is set to 
`true`, then it is advised that this configuration is also set to `true`.  If 
this configuration is set to `false` cleaning up of the allocated offheap 
memory happens only when the configured JVM parameter MaxDirectMemorySize is 
reached by triggering a full GC. **Note:** For streaming setups, we highly 
recommend to set this value to `false` as the core state backends currently do 
not use the managed memory.
--- End diff --

That would probably be good


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #3276: [FLINK-5718] [core] TaskManagers exit the JVM on f...

[greghogan]

Github user greghogan commented on a diff in the pull request:

https://github.com/apache/flink/pull/3276#discussion_r99674991
  
--- Diff: docs/setup/config.md ---
@@ -86,7 +86,7 @@ The default fraction for managed memory can be adjusted 
using the `taskmanager.m
 
 - `taskmanager.memory.segment-size`: The size of memory buffers used by 
the memory manager and the network stack in bytes (DEFAULT: 32768 (= 32 
KiBytes)).
 
-- `taskmanager.memory.preallocate`: Can be either of `true` or `false`. 
Specifies whether task managers should allocate all managed memory when 
starting up. (DEFAULT: false). When `taskmanager.memory.off-heap` is set to 
`true`, then it is advised that this configuration is also set to `true`.  If 
this configuration is set to `false` cleaning up of the allocated offheap 
memory happens only when the configured JVM parameter MaxDirectMemorySize is 
reached by triggering a full GC.
+- `taskmanager.memory.preallocate`: Can be either of `true` or `false`. 
Specifies whether task managers should allocate all managed memory when 
starting up. (DEFAULT: false). When `taskmanager.memory.off-heap` is set to 
`true`, then it is advised that this configuration is also set to `true`.  If 
this configuration is set to `false` cleaning up of the allocated offheap 
memory happens only when the configured JVM parameter MaxDirectMemorySize is 
reached by triggering a full GC. **Note:** For streaming setups, we highly 
recommend to set this value to `false` as the core state backends currently do 
not use the managed memory.
--- End diff --

Should this warning also be added to `flink-conf.yaml`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #3276: [FLINK-5718] [core] TaskManagers exit the JVM on f...

[StephanEwen]

GitHub user StephanEwen opened a pull request:

https://github.com/apache/flink/pull/3276

[FLINK-5718] [core] TaskManagers exit the JVM on fatal exceptions.

*This adds a feature requested by a user for production stability.*

Certain exceptions should not be attempted to be handled by the 
TaskManager, because they indicate that the JVM is corrupt. When the task 
throws such an exception, the TaskManager simply forcefully and immediately 
exits the JVM.

Optionally, the `OutOfMemoryError` can also be set to cause such immediate 
JVM termination, via the `taskmanager.jvm-exit-on-oom` config option.


### Tests

This adds a test that tests the option and the actual process kill (via a 
spawned test process). 

### Documentation

This adds the `taskmanager.jvm-exit-on-oom` to the `setup/config.md` docs.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/StephanEwen/incubator-flink 
exit_on_fatal_error

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/3276.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3276


commit 21c08817554e5a66186afa83158ca9c6ac975ba4
Author: Stephan Ewen 
Date:   2017-02-06T14:52:39Z

[FLINK-5718] [core] TaskManagers exit the JVM on fatal exceptions.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---