[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-20512: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, > HIVE-20512.6.patch, HIVE-20512.7.patch, HIVE-20512.8.patch, > HIVE-20512.9.patch, HIVE-20512.91.patch, HIVE-20512.92.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.92.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, > HIVE-20512.6.patch, HIVE-20512.7.patch, HIVE-20512.8.patch, > HIVE-20512.9.patch, HIVE-20512.91.patch, HIVE-20512.92.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.91.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, > HIVE-20512.6.patch, HIVE-20512.7.patch, HIVE-20512.8.patch, > HIVE-20512.9.patch, HIVE-20512.91.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: (was: HIVE-20512.10.patch) > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, > HIVE-20512.6.patch, HIVE-20512.7.patch, HIVE-20512.8.patch, HIVE-20512.9.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.10.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.10.patch, > HIVE-20512.2.patch, HIVE-20512.3.patch, HIVE-20512.4.patch, > HIVE-20512.5.patch, HIVE-20512.6.patch, HIVE-20512.7.patch, > HIVE-20512.8.patch, HIVE-20512.9.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: (was: HIVE-20512.9.patch) > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, > HIVE-20512.6.patch, HIVE-20512.7.patch, HIVE-20512.8.patch, HIVE-20512.9.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.9.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, > HIVE-20512.6.patch, HIVE-20512.7.patch, HIVE-20512.8.patch, HIVE-20512.9.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.9.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, > HIVE-20512.6.patch, HIVE-20512.7.patch, HIVE-20512.8.patch, HIVE-20512.9.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.8.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, > HIVE-20512.6.patch, HIVE-20512.7.patch, HIVE-20512.8.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.7.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, > HIVE-20512.6.patch, HIVE-20512.7.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: (was: HIVE-20512.6.patch) > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, HIVE-20512.6.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.6.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, HIVE-20512.6.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.6.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch, HIVE-20512.6.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.5.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch, HIVE-20512.5.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.4.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: (was: HIVE-20512.4.patch) > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.4.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch, HIVE-20512.4.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.3.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch, > HIVE-20512.3.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.2.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch, HIVE-20512.2.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.1.patch > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: (was: HIVE-20512.1.patch) > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20512) Improve record and memory usage logging in SparkRecordHandler
[ https://issues.apache.org/jira/browse/HIVE-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharathkrishna Guruvayoor Murali updated HIVE-20512: Attachment: HIVE-20512.1.patch Status: Patch Available (was: Open) [~stakiar] Can you please check the attached patch if this approach looks good. I tried adding unit tests, but not sure if it is needed, as I should parse the log in a hard-coded way to check the String "processed " + rowNumber + " rows: used memory =" and probably sleep() till the threshold to check if logs are printed again. What do you suggest? > Improve record and memory usage logging in SparkRecordHandler > - > > Key: HIVE-20512 > URL: https://issues.apache.org/jira/browse/HIVE-20512 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Sahil Takiar >Assignee: Bharathkrishna Guruvayoor Murali >Priority: Major > Attachments: HIVE-20512.1.patch > > > We currently log memory usage and # of records processed in Spark tasks, but > we should improve the methodology for how frequently we log this info. > Currently we use the following code: > {code:java} > private long getNextLogThreshold(long currentThreshold) { > // A very simple counter to keep track of number of rows processed by the > // reducer. It dumps > // every 1 million times, and quickly before that > if (currentThreshold >= 100) { > return currentThreshold + 100; > } > return 10 * currentThreshold; > } > {code} > The issue is that after a while, the increase by 10x factor means that you > have to process a huge # of records before this gets triggered. > A better approach would be to log this info at a given interval. This would > help in debugging tasks that are seemingly hung. -- This message was sent by Atlassian JIRA (v7.6.3#76005)