[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14360895#comment-14360895 ] Hudson commented on PHOENIX-39: --- SUCCESS: Integrated in Phoenix-master #614 (See [https://builds.apache.org/job/Phoenix-master/614/]) PHOENIX-39. ADding Pherf as a Phoenix module. Changed assembly and pom to build Pherf and include *.zip into distribution. (mujtaba: rev 36b88651cbeb680a7d06828bcb5d61d440341ad2) * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/util/RowCalculator.java * phoenix-pherf/src/test/java/org/apache/phoenix/pherf/ResourceTest.java * phoenix-pherf/src/test/resources/scenario/test_scenario.xml * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/DataLoadTimeSummary.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/DataLoadThreadTime.java * phoenix-pherf/cluster/pherf.sh * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/exception/PherfException.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/ResultHandler.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/workload/MultithreadedRunner.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/workload/QueryVerifier.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/impl/CSVResultHandler.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/configuration/Column.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/jmx/Stat.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/jmx/monitors/ThreadMonitor.java * phoenix-pherf/src/test/java/org/apache/phoenix/pherf/TestHBaseProps.java * phoenix-pherf/src/test/java/org/apache/phoenix/pherf/DataIngestTest.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/util/ResourceList.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/loaddata/DataLoader.java * phoenix-pherf/config/scenario/user_defined_scenario.xml * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/jmx/monitors/GarbageCollectorElapsedTimeMonitor.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/Result.java * phoenix-pherf/src/test/java/org/apache/phoenix/pherf/ResultTest.java * phoenix-pherf/src/main/assembly/standalone.xml * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/exception/PherfRuntimeException.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/file/Header.java * phoenix-pherf/src/test/java/org/apache/phoenix/pherf/RuleGeneratorTest.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/impl/ImageResultHandler.java * phoenix-pherf/README.md * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/configuration/XMLConfigParser.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/configuration/Scenario.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/jmx/monitors/FreeMemoryMonitor.java * phoenix-pherf/src/main/resources/datamodel/create_prod_test_unsalted.sql * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/workload/QueryExecutor.java * phoenix-pherf/config/datamodel/user_defined_schema.sql * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/jmx/MonitorManager.java * phoenix-pherf/src/test/java/org/apache/phoenix/pherf/BaseTestWithCluster.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/file/ResultFileDetails.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/configuration/QuerySet.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/ScenarioResult.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/ResultUtil.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/jmx/monitors/TotalMemoryMonitor.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/workload/MultithreadedDiffer.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/workload/WorkloadExecutor.java * phoenix-assembly/src/build/components/all-common-files.xml * phoenix-pherf/config/env.sh * phoenix-pherf/src/test/java/org/apache/phoenix/pherf/DataLoaderTest.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/jmx/monitors/HeapMemoryMonitor.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/jmx/monitors/NonHeapMemoryMonitor.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/DataModelResult.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/QuerySetResult.java * phoenix-pherf/src/main/assembly/cluster.xml * phoenix-pherf/src/test/java/org/apache/phoenix/pherf/ColumnTest.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/configuration/DataOverride.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/jmx/monitors/CPULoadAverageMonitor.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/result/impl/XMLResultHandler.java * phoenix-pherf/src/main/java/org/apache/phoenix/pherf/configuration/DataSequence.java * phoenix-pherf/src/test/resources/hbase-site.xml * phoenix-pherf/src/main/java/org/apache/phoenix/ph
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359432#comment-14359432 ] Mujtaba Chohan commented on PHOENIX-39: --- +1 :) > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Cody Marcel > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359360#comment-14359360 ] Cody Marcel commented on PHOENIX-39: [~jamestaylor] Sure thing. I'll start creating them. > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Cody Marcel > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359320#comment-14359320 ] James Taylor commented on PHOENIX-39: - Correct. Mujtaba, if your +1, please commit to master and 4.0 branches. Cody - would you mind filing those follow up JIRAs (or maybe you already have and I missed them)? Thanks - excellent work! > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Cody Marcel > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359307#comment-14359307 ] Cody Marcel commented on PHOENIX-39: [~mujtabachohan] I think [~jamestaylor] is waiting for your +1. Is there anything outstanding you wanted to see? > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Cody Marcel > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355576#comment-14355576 ] Jesse Yates commented on PHOENIX-39: +1 overall; this is massive though, so I'd want another committer who knows the added code more closely to also +1, i.e. [~mujtabachohan]. There are probably some nits we can cleanup, but that will likely be an as-we-go kind of thing. > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Cody Marcel > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353866#comment-14353866 ] Cody Marcel commented on PHOENIX-39: Pull request sent. https://github.com/apache/phoenix/pull/41 > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Cody Marcel > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344188#comment-14344188 ] Cody Marcel commented on PHOENIX-39: Awesome thanks! Patch is in progress. > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Cody Marcel > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344082#comment-14344082 ] James Taylor commented on PHOENIX-39: - I added you as a contributor, [~cody.mar...@gmail.com]. > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Mujtaba Chohan > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344012#comment-14344012 ] Cody Marcel commented on PHOENIX-39: [~jamestaylor] Can you make me a project contributor here so I can change assignee to me? > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Mujtaba Chohan > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14344008#comment-14344008 ] Cody Marcel commented on PHOENIX-39: I just came in to add a Jira for Pherf. I guess I should use this one instead. > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Mujtaba Chohan > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315014#comment-14315014 ] Mujtaba Chohan commented on PHOENIX-39: --- Correct [~jamestaylor]. We'll close this when we add Pherf (performance/functional testing at scale) module to Phoenix. We also have YCSB patch available for JDBC compliant DBs comparison with patch available at https://github.com/brianfrankcooper/YCSB/pull/178 > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Mujtaba Chohan > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-39) Add sustained load tester that measures throughput
[ https://issues.apache.org/jira/browse/PHOENIX-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314888#comment-14314888 ] James Taylor commented on PHOENIX-39: - This sounds like Pherf, [~mujtabachohan] and [~cody.mar...@gmail.com]. > Add sustained load tester that measures throughput > -- > > Key: PHOENIX-39 > URL: https://issues.apache.org/jira/browse/PHOENIX-39 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Mujtaba Chohan > > We should add a YCSB-like [1] sustained load tester that measures throughput > over an extended time period for a fully loaded cluster using Phoenix. > Ideally, we'd want to be able to dial up/down the read/write percentages, and > control the types of queries being run (scan, aggregate, joins, array usage, > etc). Another interesting dimension is simultaneous users and on top of that > multi-tenant views. > This would be a big effort, but we can stage it and increase the knobs and > dials as we go. > [1] http://hbase.apache.org/book/apd.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
