[
https://issues.apache.org/jira/browse/HIVE-19571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496832#comment-16496832
]
Sahil Takiar commented on HIVE-19571:
-------------------------------------
Attaching what I have so far. Still need to look into the following things:
* Isolation of dirs on the ptest master
* Double check .vm scripts to ensure no clashing
The hardest part is going to be testing. We'll probably have to setup a ptest
dedicated cluster for testing this, which isn't easy to do.
> Ability to run multiple pre-commit jobs on a ptest server
> ---------------------------------------------------------
>
> Key: HIVE-19571
> URL: https://issues.apache.org/jira/browse/HIVE-19571
> Project: Hive
> Issue Type: Sub-task
> Components: Testing Infrastructure
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
> Attachments: HIVE-17317.WIP.1.patch
>
>
> I've been taking a look at the Disk, Network, and CPU usage of the GCE
> instances we run ptest on, and it doesn't look like we are fully utilizing
> the machines. The resource usage is very up and down.
> During each ptest execution, there is a large chunk of time (~20 min) where
> its just the Jenkins job that is doing any work (checking out github repos,
> building code, figuring out test batches, etc.). During this time, the ptest
> nodes are mostly idle - the CPU and Disk I/O are almost zero.
> Even when ptest is running, I think some of resources are under-utilized.
> Network and disk resource spike at the beginning of the job, probably because
> ptest is distributing resources to each machine, each slave is downloading
> jars, etc. However, after that, when the actual tests run, there is almost 0
> network activity (which makes sense since tests runs on a single node). For
> disk usage, there is activity, but not nearly as high as when the setup phase
> was occuring. CPU usage fluctuates between 40-80%.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
