[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16663628#comment-16663628 ] ASF GitHub Bot commented on IGNITE-9637: Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4796 > SQL: Create suite for data load benchmarks > -- > > Key: IGNITE-9637 > URL: https://issues.apache.org/jira/browse/IGNITE-9637 > Project: Ignite > Issue Type: Task > Components: yardstick >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > Fix For: 2.8 > > > We want to have benchmark suite that measures how long it takes to upload > data to server. > There is existing benchmark code in org.apache.ignite.yardstick.upload > package > 1) Improve data model: > - Add indexes to data model. Number of indexes should be configurable. > - Add missing benchmark that performs putAll > 2) Define parameters to use in benchmarks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16663620#comment-16663620 ] Vladimir Ozerov commented on IGNITE-9637: - Merged to master. > SQL: Create suite for data load benchmarks > -- > > Key: IGNITE-9637 > URL: https://issues.apache.org/jira/browse/IGNITE-9637 > Project: Ignite > Issue Type: Task > Components: yardstick >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > Fix For: 2.8 > > > We want to have benchmark suite that measures how long it takes to upload > data to server. > There is existing benchmark code in org.apache.ignite.yardstick.upload > package > 1) Improve data model: > - Add indexes to data model. Number of indexes should be configurable. > - Add missing benchmark that performs putAll > 2) Define parameters to use in benchmarks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629596#comment-16629596 ] Pavel Kuznetsov commented on IGNITE-9637: - added 1 missed benchmark (native sql insert) added thick driver to configuration. runned on aws with 2 servers and 1(2 in case of native benchmarks) clients ||Configuration ||Avg (ops/sec)|| |RELEASE-upload-native-putAll |3,862.24| |RELEASE-upload-native-put |157,256.65| |RELEASE-upload-native-streamer |2,161.86| |RELEASE-upload-native-sql-insert |195,163.52| |RELEASE-upload-jdbc-thin-copy |12,325.41| |RELEASE-upload-jdbc-thin-streaming |2,875.94| |RELEASE-upload-jdbc-thin-insert|276,875.17| |RELEASE-upload-jdbc-v2-insert |197,233.92| |RELEASE-upload-jdbc-thin-batched-insert|35,139.98| |RELEASE-upload-jdbc-v2-batched-insert |195,218.65| [~vozerov] Could you please take a look at the patch? > SQL: Create suite for data load benchmarks > -- > > Key: IGNITE-9637 > URL: https://issues.apache.org/jira/browse/IGNITE-9637 > Project: Ignite > Issue Type: Bug >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We want to have benchmark suite that measures how long it takes to upload > data to server. > There is existing benchmark code in org.apache.ignite.yardstick.upload > package > 1) Improve data model: > - Add indexes to data model. Number of indexes should be configurable. > - Add missing benchmark that performs putAll > 2) Define parameters to use in benchmarks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625916#comment-16625916 ] Taras Ledkov commented on IGNITE-9637: -- [~pkouznet], the patch is OK with me. Review status: | Default product behavior | not affects | | Binary compatibility etc. | not affects | | Tests | none. | > SQL: Create suite for data load benchmarks > -- > > Key: IGNITE-9637 > URL: https://issues.apache.org/jira/browse/IGNITE-9637 > Project: Ignite > Issue Type: Bug >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We want to have benchmark suite that measures how long it takes to upload > data to server. > There is existing benchmark code in org.apache.ignite.yardstick.upload > package > 1) Improve data model: > - Add indexes to data model. Number of indexes should be configurable. > - Add missing benchmark that performs putAll > 2) Define parameters to use in benchmarks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625741#comment-16625741 ] Pavel Kuznetsov commented on IGNITE-9637: - Thank you! done. [~vozerov] Could you please take a look at the patch? > SQL: Create suite for data load benchmarks > -- > > Key: IGNITE-9637 > URL: https://issues.apache.org/jira/browse/IGNITE-9637 > Project: Ignite > Issue Type: Bug >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We want to have benchmark suite that measures how long it takes to upload > data to server. > There is existing benchmark code in org.apache.ignite.yardstick.upload > package > 1) Improve data model: > - Add indexes to data model. Number of indexes should be configurable. > - Add missing benchmark that performs putAll > 2) Define parameters to use in benchmarks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[
https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623281#comment-16623281
]
Taras Ledkov commented on IGNITE-9637:
--
[~pkouznet], my comments:
# please review my minor changes;
# add javadoc for {{UploadBenchmarkArguments#idxCnt}};
> SQL: Create suite for data load benchmarks
> --
>
> Key: IGNITE-9637
> URL: https://issues.apache.org/jira/browse/IGNITE-9637
> Project: Ignite
> Issue Type: Bug
>Reporter: Pavel Kuznetsov
>Assignee: Pavel Kuznetsov
>Priority: Major
>
> We want to have benchmark suite that measures how long it takes to upload
> data to server.
> There is existing benchmark code in org.apache.ignite.yardstick.upload
> package
> 1) Improve data model:
> - Add indexes to data model. Number of indexes should be configurable.
> - Add missing benchmark that performs putAll
> 2) Define parameters to use in benchmarks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622777#comment-16622777 ] Pavel Kuznetsov commented on IGNITE-9637: - ||Benchmark class |Configuration name |Load time, ms|| |NativePutAllBenchmark |upload-native-putAll |7,472.01| |NativePutBenchmark |upload-native-put |187,832.46| |NativeStreamerBenchmark|upload-native-streamer |2,265.88| |CopyBenchmark |upload-copy|12,476.99| |InsertBenchmark|upload-insert |295,748.12| |InsertBenchmark|upload-sql-streaming |2,838.30| |BatchedInsertBenchmark |upload-batched-insert |15,087.90| > SQL: Create suite for data load benchmarks > -- > > Key: IGNITE-9637 > URL: https://issues.apache.org/jira/browse/IGNITE-9637 > Project: Ignite > Issue Type: Bug >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We want to have benchmark suite that measures how long it takes to upload > data to server. > There is existing benchmark code in org.apache.ignite.yardstick.upload > package > 1) Improve data model: > - Add indexes to data model. Number of indexes should be configurable. > - Add missing benchmark that performs putAll > 2) Define parameters to use in benchmarks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621954#comment-16621954 ] Pavel Kuznetsov commented on IGNITE-9637: - [~tledkov] Could you please take a look at the patch too? > SQL: Create suite for data load benchmarks > -- > > Key: IGNITE-9637 > URL: https://issues.apache.org/jira/browse/IGNITE-9637 > Project: Ignite > Issue Type: Bug >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We want to have benchmark suite that measures how long it takes to upload > data to server. > There is existing benchmark code in org.apache.ignite.yardstick.upload > package > 1) Improve data model: > - Add indexes to data model. Number of indexes should be configurable. > - Add missing benchmark that performs putAll > 2) Define parameters to use in benchmarks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621870#comment-16621870 ] Pavel Kuznetsov commented on IGNITE-9637: - created PR on GitHub https://github.com/apache/ignite/pull/4796 > SQL: Create suite for data load benchmarks > -- > > Key: IGNITE-9637 > URL: https://issues.apache.org/jira/browse/IGNITE-9637 > Project: Ignite > Issue Type: Bug >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We want to have benchmark suite that measures how long it takes to upload > data to server. > There is existing benchmark code in org.apache.ignite.yardstick.upload > package > 1) Improve data model: > - Add indexes to data model. Number of indexes should be configurable. > - Add missing benchmark that performs putAll > 2) Define parameters to use in benchmarks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621868#comment-16621868 ] ASF GitHub Bot commented on IGNITE-9637: GitHub user pavel-kuznetsov opened a pull request: https://github.com/apache/ignite/pull/4796 IGNITE-9637: Suite for data load benchmarks You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9637 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4796.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4796 commit 6dfea795a7628c44b930b4f7cb097725e5d2260d Author: Pavel Kuznetsov Date: 2018-09-19T11:09:54Z ignite-9637: added indexes. "--idx-count" handles how many indexes there is in the data model. commit 71e93a1b32e18fb05183570875d3b6f8646cbfd9 Author: Pavel Kuznetsov Date: 2018-09-19T12:07:17Z ignite-9637: native putAll benchmark. commit 6f40922d883f38cf54abdca2bcada4a0b1fc5c55 Author: Pavel Kuznetsov Date: 2018-09-19T15:13:40Z ignite-9637: benchmark configuration draft. Created configuration for data load benchmarks run that contains one run of each benchmark with some "common" parameters. commit 27bbd1e3bc1d2440f796769d9ba1ba6da712dad7 Author: Pavel Kuznetsov Date: 2018-09-19T18:11:56Z ignite-9637: fixed streaming "ordered" option. "ordered"/"" instead of "ordered on"/"ordered off". By default ordered is now turned off. > SQL: Create suite for data load benchmarks > -- > > Key: IGNITE-9637 > URL: https://issues.apache.org/jira/browse/IGNITE-9637 > Project: Ignite > Issue Type: Bug >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We want to have benchmark suite that measures how long it takes to upload > data to server. > There is existing benchmark code in org.apache.ignite.yardstick.upload > package > 1) Improve data model: > - Add indexes to data model. Number of indexes should be configurable. > - Add missing benchmark that performs putAll > 2) Define parameters to use in benchmarks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9637) SQL: Create suite for data load benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620739#comment-16620739 ] Pavel Kuznetsov commented on IGNITE-9637: - About 2) Batched insert and putAll benchmarks have "batchSize" parameter Sql and native streamer benchmarks have : "BATCH_SIZE", "ALLOW_OVERWRITE", "PER_NODE_PARALLEL_OPERATIONS", "PER_NODE_BUFFER_SIZE", "ORDERED" parameters [~vozerov] could you please take a look at configuration file modules/yardstick/config/upload/benchmark-upload-regular.properties ? > SQL: Create suite for data load benchmarks > -- > > Key: IGNITE-9637 > URL: https://issues.apache.org/jira/browse/IGNITE-9637 > Project: Ignite > Issue Type: Bug >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We want to have benchmark suite that measures how long it takes to upload > data to server. > There is existing benchmark code in org.apache.ignite.yardstick.upload > package > 1) Improve data model: > - Add indexes to data model. Number of indexes should be configurable. > - Add missing benchmark that performs putAll > 2) Define parameters to use in benchmarks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
