[jira] [Commented] (HIVE-13107) LLAP: Rotate GC logs periodically to prevent full disks

2016-03-22 Thread Lefty Leverenz (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207237#comment-15207237
 ] 

Lefty Leverenz commented on HIVE-13107:
---

Does this need to be documented in the wiki?  (If so, please add a TODOC2.1 
label.)  It could go in a new subsection of Hive Logging:

* [Getting Started -- Hive Logging | 
https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-HiveLogging]

> LLAP: Rotate GC logs periodically to prevent full disks
> ---
>
> Key: HIVE-13107
> URL: https://issues.apache.org/jira/browse/HIVE-13107
> Project: Hive
>  Issue Type: Improvement
>  Components: llap
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Trivial
> Fix For: 2.1.0
>
> Attachments: HIVE-13107.1.patch, HIVE-13107.2.patch
>
>
> STDOUT cannot be rotated easily, so log GC logs to a different file and 
> rotate periodically with -XX:+UseGCLogFileRotation
> NO PRECOMMIT TESTS



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13107) LLAP: Rotate GC logs periodically to prevent full disks

2016-03-21 Thread Prasanth Jayachandran (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205797#comment-15205797
 ] 

Prasanth Jayachandran commented on HIVE-13107:
--

+1

> LLAP: Rotate GC logs periodically to prevent full disks
> ---
>
> Key: HIVE-13107
> URL: https://issues.apache.org/jira/browse/HIVE-13107
> Project: Hive
>  Issue Type: Improvement
>  Components: llap
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Trivial
> Attachments: HIVE-13107.1.patch
>
>
> STDOUT cannot be rotated easily, so log GC logs to a different file and 
> rotate periodically with -XX:+UseGCLogFileRotation
> NO PRECOMMIT TESTS



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13107) LLAP: Rotate GC logs periodically to prevent full disks

2016-02-19 Thread Gopal V (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155332#comment-15155332
 ] 

Gopal V commented on HIVE-13107:


We don't have a dedicated tmpfs at this point, but the issue can happen due to 
the file length marker being updated for a file into the ext journal.

I suspect we'll have a flip-switch-launch-control sort of mode to turn logging 
off and speed up things in general.

> LLAP: Rotate GC logs periodically to prevent full disks
> ---
>
> Key: HIVE-13107
> URL: https://issues.apache.org/jira/browse/HIVE-13107
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
>
> STDOUT cannot be rotated easily, so log GC logs to a different file and 
> rotate periodically with -XX:+UseGCLogFileRotation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13107) LLAP: Rotate GC logs periodically to prevent full disks

2016-02-19 Thread Prasanth Jayachandran (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155329#comment-15155329
 ] 

Prasanth Jayachandran commented on HIVE-13107:
--

Another note to avoid writing disk (instead use /tmpfs?) as write() syscal can 
block application[1] for long time.

[1] 
https://engineering.linkedin.com/blog/2016/02/eliminating-large-jvm-gc-pauses-caused-by-background-io-traffic


> LLAP: Rotate GC logs periodically to prevent full disks
> ---
>
> Key: HIVE-13107
> URL: https://issues.apache.org/jira/browse/HIVE-13107
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
>
> STDOUT cannot be rotated easily, so log GC logs to a different file and 
> rotate periodically with -XX:+UseGCLogFileRotation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13107) LLAP: Rotate GC logs periodically to prevent full disks

2016-02-19 Thread Prasanth Jayachandran (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155327#comment-15155327
 ] 

Prasanth Jayachandran commented on HIVE-13107:
--

Also we should consider using -XX:+PrintGCDateStamps instead of 
-XX:+PrintGCTimeStamps. 

{code}
-XX:+PrintGCTimeStamps -> 0,185: [GC 66048K->53077K(251392K), 0,0977580 secs]
-XX:+PrintGCDateStamps -> 2014-01-03T12:08:38.102-0100: [GC 
66048K->53077K(251392K), 0,0959470 secs]
{code}

Easier to correlate with log lines.

> LLAP: Rotate GC logs periodically to prevent full disks
> ---
>
> Key: HIVE-13107
> URL: https://issues.apache.org/jira/browse/HIVE-13107
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Gopal V
>Assignee: Gopal V
>
> STDOUT cannot be rotated easily, so log GC logs to a different file and 
> rotate periodically with -XX:+UseGCLogFileRotation



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)