[jira] [Commented] (TRAFODION-2046) change DTM's memoryUsageChore default settings to enable memory threthold

2016-07-08 Thread Sean Broeder (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368057#comment-15368057
 ] 

Sean Broeder commented on TRAFODION-2046:
-

This change has added a new MemoryUsageException that is not handled correctly 
and is causing test 130 in the daily builds to fail.  As a short term fix I am 
submitting a pull request to back out these changes until we can better and 
more predictably handle these exceptions, otherwise transactional DDL can leave 
a sustem in an unuseable state.

>>drop table tlob130ts1;
 
*** ERROR[8448] Unable to access Hbase interface. Call to 
ExpHbaseInterface::getRowOpen returned error HBASE_OPEN_ERROR(-704). Cause:
java.io.IOException: 
org.apache.hadoop.hbase.client.transactional.MemoryUsageException: get memory 
usage exceeds 90 percent, trxId is 101365
org.apache.hadoop.hbase.client.transactional.TransactionalTable.get(TransactionalTable.java:211)
org.apache.hadoop.hbase.client.transactional.RMInterface.get(RMInterface.java:278)
org.trafodion.sql.HTableClient.startGet(HTableClient.java:1049)
org.trafodion.sql.HBaseClient.startGet(HBaseClient.java:1596)
.
 
--- SQL operation failed with errors.
 

> change DTM's memoryUsageChore default settings to enable memory threthold
> -
>
> Key: TRAFODION-2046
> URL: https://issues.apache.org/jira/browse/TRAFODION-2046
> Project: Apache Trafodion
>  Issue Type: Improvement
>Reporter: liu ming
>Assignee: liu ming
>
> DTM provide MemoryUsageChore to monitor the memory pressure in a 
> RegionServer, when memory is beyond a defined threshold, it will give a 
> warning or prevent new operations based on a flag.
> By default, the threshold is now 100%, and the flag is just warning. So that 
> means DTM coprocessor can use all RS memory and when it find memory pressure 
> , just print warning, instead of control new transaction.
> This will lead to un-predictable behavior for certain fat transaction.
> It is better to change the default setting to enable protection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-2046) change DTM's memoryUsageChore default settings to enable memory threthold

2016-06-14 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330786#comment-15330786
 ] 

ASF GitHub Bot commented on TRAFODION-2046:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-trafodion/pull/542


> change DTM's memoryUsageChore default settings to enable memory threthold
> -
>
> Key: TRAFODION-2046
> URL: https://issues.apache.org/jira/browse/TRAFODION-2046
> Project: Apache Trafodion
>  Issue Type: Improvement
>Reporter: liu ming
>Assignee: liu ming
>
> DTM provide MemoryUsageChore to monitor the memory pressure in a 
> RegionServer, when memory is beyond a defined threshold, it will give a 
> warning or prevent new operations based on a flag.
> By default, the threshold is now 100%, and the flag is just warning. So that 
> means DTM coprocessor can use all RS memory and when it find memory pressure 
> , just print warning, instead of control new transaction.
> This will lead to un-predictable behavior for certain fat transaction.
> It is better to change the default setting to enable protection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-2046) change DTM's memoryUsageChore default settings to enable memory threthold

2016-06-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328737#comment-15328737
 ] 

ASF GitHub Bot commented on TRAFODION-2046:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/542#discussion_r66894843
  
--- Diff: 
core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/coprocessor/transactional/TrxRegionEndpoint.java.tmpl
 ---
@@ -359,7 +359,7 @@ CoprocessorService, Coprocessor {
   private static final int DEFAULT_SLEEP = 60 * 1000;
   private static final int DEFAULT_MEMORY_THRESHOLD = 100; // 100% memory 
used
--- End diff --

Yes, Dave, that is a good suggestion, much better coding style.


> change DTM's memoryUsageChore default settings to enable memory threthold
> -
>
> Key: TRAFODION-2046
> URL: https://issues.apache.org/jira/browse/TRAFODION-2046
> Project: Apache Trafodion
>  Issue Type: Improvement
>Reporter: liu ming
>Assignee: liu ming
>
> DTM provide MemoryUsageChore to monitor the memory pressure in a 
> RegionServer, when memory is beyond a defined threshold, it will give a 
> warning or prevent new operations based on a flag.
> By default, the threshold is now 100%, and the flag is just warning. So that 
> means DTM coprocessor can use all RS memory and when it find memory pressure 
> , just print warning, instead of control new transaction.
> This will lead to un-predictable behavior for certain fat transaction.
> It is better to change the default setting to enable protection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-2046) change DTM's memoryUsageChore default settings to enable memory threthold

2016-06-13 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328694#comment-15328694
 ] 

ASF GitHub Bot commented on TRAFODION-2046:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/542#discussion_r66891751
  
--- Diff: 
core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/coprocessor/transactional/TrxRegionEndpoint.java.tmpl
 ---
@@ -359,7 +359,7 @@ CoprocessorService, Coprocessor {
   private static final int DEFAULT_SLEEP = 60 * 1000;
   private static final int DEFAULT_MEMORY_THRESHOLD = 100; // 100% memory 
used
--- End diff --

I see. Perhaps add a new "private static final int" for your new usage? For 
example, "private static final int DEFAULT_STARTUP_MEMORY_THRESHOLD = 90;". 
Just a stylistic nit. 


> change DTM's memoryUsageChore default settings to enable memory threthold
> -
>
> Key: TRAFODION-2046
> URL: https://issues.apache.org/jira/browse/TRAFODION-2046
> Project: Apache Trafodion
>  Issue Type: Improvement
>Reporter: liu ming
>Assignee: liu ming
>
> DTM provide MemoryUsageChore to monitor the memory pressure in a 
> RegionServer, when memory is beyond a defined threshold, it will give a 
> warning or prevent new operations based on a flag.
> By default, the threshold is now 100%, and the flag is just warning. So that 
> means DTM coprocessor can use all RS memory and when it find memory pressure 
> , just print warning, instead of control new transaction.
> This will lead to un-predictable behavior for certain fat transaction.
> It is better to change the default setting to enable protection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-2046) change DTM's memoryUsageChore default settings to enable memory threthold

2016-06-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325779#comment-15325779
 ] 

ASF GitHub Bot commented on TRAFODION-2046:
---

GitHub user traflm opened a pull request:

https://github.com/apache/incubator-trafodion/pull/542

[TRAFODION-2046]change DTM's memoryUsageChore default settings to ena…

…ble memory threthold

Change the default behavior of DTM to restrict the memory usage in Region 
Server to 90%, if above that threshold, stop all new transaction operations 
until memory released under the threshold.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/traflm/incubator-trafodion TRAFODION-2046-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-trafodion/pull/542.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #542


commit 31d0bea37b7b0f09ba84495f28a6f9044e7368d6
Author: Liu Ming 
Date:   2016-06-11T08:47:41Z

[TRAFODION-2046]change DTM's memoryUsageChore default settings to enable 
memory threthold




> change DTM's memoryUsageChore default settings to enable memory threthold
> -
>
> Key: TRAFODION-2046
> URL: https://issues.apache.org/jira/browse/TRAFODION-2046
> Project: Apache Trafodion
>  Issue Type: Improvement
>Reporter: liu ming
>Assignee: liu ming
>
> DTM provide MemoryUsageChore to monitor the memory pressure in a 
> RegionServer, when memory is beyond a defined threshold, it will give a 
> warning or prevent new operations based on a flag.
> By default, the threshold is now 100%, and the flag is just warning. So that 
> means DTM coprocessor can use all RS memory and when it find memory pressure 
> , just print warning, instead of control new transaction.
> This will lead to un-predictable behavior for certain fat transaction.
> It is better to change the default setting to enable protection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-2046) change DTM's memoryUsageChore default settings to enable memory threthold

2016-06-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325760#comment-15325760
 ] 

ASF GitHub Bot commented on TRAFODION-2046:
---

Github user traflm closed the pull request at:

https://github.com/apache/incubator-trafodion/pull/537


> change DTM's memoryUsageChore default settings to enable memory threthold
> -
>
> Key: TRAFODION-2046
> URL: https://issues.apache.org/jira/browse/TRAFODION-2046
> Project: Apache Trafodion
>  Issue Type: Improvement
>Reporter: liu ming
>Assignee: liu ming
>
> DTM provide MemoryUsageChore to monitor the memory pressure in a 
> RegionServer, when memory is beyond a defined threshold, it will give a 
> warning or prevent new operations based on a flag.
> By default, the threshold is now 100%, and the flag is just warning. So that 
> means DTM coprocessor can use all RS memory and when it find memory pressure 
> , just print warning, instead of control new transaction.
> This will lead to un-predictable behavior for certain fat transaction.
> It is better to change the default setting to enable protection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-2046) change DTM's memoryUsageChore default settings to enable memory threthold

2016-06-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TRAFODION-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323933#comment-15323933
 ] 

ASF GitHub Bot commented on TRAFODION-2046:
---

GitHub user traflm opened a pull request:

https://github.com/apache/incubator-trafodion/pull/537

[TRAFODION-2046]change DTM's memoryUsageChore default settings to ena…

…ble memory threthold

Change the default behavior of DTM to restrict the memory usage in Region 
Server to 90%, if above that threshold, stop all new transaction operations 
until memory released under the threshold.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/traflm/incubator-trafodion TRAFODION-2046

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-trafodion/pull/537.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #537


commit c849baf89221a3f943e3d66d303390d0cc453fa8
Author: Liu Ming 
Date:   2016-06-10T06:07:34Z

[TRAFODION-2046]change DTM's memoryUsageChore default settings to enable 
memory threthold




> change DTM's memoryUsageChore default settings to enable memory threthold
> -
>
> Key: TRAFODION-2046
> URL: https://issues.apache.org/jira/browse/TRAFODION-2046
> Project: Apache Trafodion
>  Issue Type: Improvement
>Reporter: liu ming
>Assignee: liu ming
>
> DTM provide MemoryUsageChore to monitor the memory pressure in a 
> RegionServer, when memory is beyond a defined threshold, it will give a 
> warning or prevent new operations based on a flag.
> By default, the threshold is now 100%, and the flag is just warning. So that 
> means DTM coprocessor can use all RS memory and when it find memory pressure 
> , just print warning, instead of control new transaction.
> This will lead to un-predictable behavior for certain fat transaction.
> It is better to change the default setting to enable protection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)