Oleg, Thanks for the clarification.
My opinion is that we should leave ‘modules/yardstick/README.txt’ and ‘modules/yardstick/DEVNOTES.txt’ either unchanged or have only those instructions there that explain how to build and run benchmarks from ‘modules/yardstick’. This existing files can refer to the sources and compiled benchmarks that are in Ignite binary releases but this should be a couple of statements, no more. As for the instructions related to the sources and binaries added to Ignite binaries, preferably they need to be added to auto-generated README.txt. *Anton*, is it feasible to do? Finally, when apply the reviews notes please build and share Ignite binaries with your patch. Want to see the new project structure and final version of the instructions. Presently I can’t merge your changes due to some conflicts. *Anton*, please review modifications in the build procedures. — Denis > On Jan 31, 2017, at 3:12 AM, Oleg Ostanin <oosta...@gridgain.com> wrote: > > Hi Denis, > > Yes, we have included Ignite Yardstick source files with its pom.xml in > Ignite binary release. "Building from standalone sources" is the building > instruction for these source files. "Building from Ignite Sources" is the > instruction for building Ignite Yardstick from `modules/yardstick` in > Ignite source files. > > On Tue, Jan 31, 2017 at 3:31 AM, Denis Magda <dma...@apache.org> wrote: > >> Hi Oleg, >> >> Great progress, thanks for keep driving this! >> >> I’ve left some minor notes in GitHub’s pull-request. I have the following >> questions aside: >> >> - What is the difference between "Building from standalone sources" and >> "Building from Ignite Sources"? In my understanding, a user downloads >> Apache Ignite release that has all the sources locally. >> >> - I do remember we planned to add the benchmarks sources in a form of a >> ready to be used project with its own pom.xml (similar to examples). Did >> you put this task off? >> >> — >> Denis >> >>> On Jan 27, 2017, at 2:13 AM, Oleg Ostanin <oosta...@gridgain.com> wrote: >>> >>> Hi! >>> >>> I've changed the README.txt and DEVNOTES.txt files. Also added a simple >>> config file for quick and easy start. Please take a look at them and tell >>> me what you think. >>> >>> https://github.com/apache/ignite/pull/1471 >>> >>> On Wed, Dec 28, 2016 at 8:59 AM, Ilya Suntsov <isunt...@gridgain.com> >> wrote: >>> >>>> Denis, >>>> >>>> I think we can remove all configs except: >>>> >>>> benchmark-multicast.properties >>>> >>>> benchmark.properties >>>> >>>> ignite-base-config.xml >>>> >>>> ignite-localhost-config.xml >>>> >>>> ignite-multicast-config.xml >>>> >>>> 2016-12-28 2:49 GMT+03:00 Denis Magda <dma...@apache.org>: >>>> >>>>> I would have only those configs that are useful. Ilya Suntsov, basing >> on >>>>> your experience, please suggest which configs makes sense to include >> into >>>>> every Ignite release. >>>>> >>>>> Oleg, also please note that community decided to include not only the >>>>> benchmarking binaries but the sources as well into every Apache Ignite >>>>> release. I’ve update the ticket before. Hope you followed the >> discussion >>>> ;) >>>>> https://issues.apache.org/jira/browse/IGNITE-4212? >>>>> focusedCommentId=15765151&page=com.atlassian.jira. >>>>> plugin.system.issuetabpanels:comment-tabpanel#comment-15765151 >>>>> >>>>> — >>>>> Denis >>>>> >>>>>> On Dec 27, 2016, at 5:35 AM, Oleg Ostanin <oosta...@gridgain.com> >>>> wrote: >>>>>> >>>>>> I mean removing those configs from binary assembly, not from >>>> repository. >>>>>> >>>>>> On Tue, Dec 27, 2016 at 4:28 PM, Oleg Ostanin <oosta...@gridgain.com> >>>>> wrote: >>>>>> >>>>>>> Hello Igniters. >>>>>>> I think it would be better to remove some configuration files from >>>>>>> benchmarks/config: >>>>>>> >>>>>>> ignite-base-load-config.xml >>>>>>> ignite-cache-load-config.xml >>>>>>> ignite-failover-base-config.xml >>>>>>> ignite-failover-localhost-config.xml >>>>>>> benchmark-cache-load.properties >>>>>>> benchmark-cache-load-win.properties >>>>>>> benchmark-failover.properties >>>>>>> >>>>>>> because those configs do not relate to any of performance tests. >>>>>>> >>>>>>> On Tue, Dec 20, 2016 at 11:24 PM, Denis Magda <dma...@apache.org> >>>>> wrote: >>>>>>> >>>>>>>> Summarized the discussion updating the ticket >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212# < >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4212#> >>>>>>>> >>>>>>>> — >>>>>>>> Denis >>>>>>>> >>>>>>>>> On Dec 19, 2016, at 12:26 PM, Dmitriy Setrakyan < >>>>> dsetrak...@apache.org> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Sergey, >>>>>>>>> >>>>>>>>> I am not sure I like "extras". I am voting for "benchmarks" folder >>>>> right >>>>>>>>> under the root folder. >>>>>>>>> >>>>>>>>> D. >>>>>>>>> >>>>>>>>> On Mon, Dec 19, 2016 at 12:07 PM, Sergey Kozlov < >>>> skoz...@gridgain.com >>>>>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Formatting has cut lines: >>>>>>>>>> >>>>>>>>>> — apache_ignite_root_folder >>>>>>>>>> — bin >>>>>>>>>> — examples >>>>>>>>>> — extras >>>>>>>>>> — benchmarks >>>>>>>>>> — bin >>>>>>>>>> — src (benchmarks sources with pom.xml) >>>>>>>>>> — config >>>>>>>>>> — libs (compiled benchmarks) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Dec 19, 2016 at 11:04 PM, Sergey Kozlov < >>>>> skoz...@gridgain.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Denis, >>>>>>>>>>> >>>>>>>>>>> Mostly yes. But I look ahead and think that we may include more >>>>>>>> things in >>>>>>>>>>> future than yardstick only. It's why I suggest something like >>>> that: >>>>>>>>>>> — apache_ignite_root_folder >>>>>>>>>>> — bin >>>>>>>>>>> — examples >>>>>>>>>>> — extras >>>>>>>>>>> — benchmarks >>>>>>>>>>> — bin >>>>>>>>>>> — src (benchmarks sources with pom.xml) >>>>>>>>>>> — config >>>>>>>>>>> — libs (compiled benchmarks) >>>>>>>>>>> >>>>>>>>>>> On Mon, Dec 19, 2016 at 10:15 PM, Denis Magda <dma...@apache.org >>> >>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Well, if to refer to Dmitriy suggestion we can have the >> following >>>>>>>>>>>> structure >>>>>>>>>>>> >>>>>>>>>>>> — apache_ignite_root_folder >>>>>>>>>>>> — examples >>>>>>>>>>>> — bin >>>>>>>>>>>> — benchmarks >>>>>>>>>>>> — bin >>>>>>>>>>>> — src (benchmarks sources with pom.xml) >>>>>>>>>>>> — config >>>>>>>>>>>> — libs (compiled benchmarks) >>>>>>>>>>>> >>>>>>>>>>>> Sergey, will it cover all the use case you’ve met previously? >>>>>>>>>>>> >>>>>>>>>>>> — >>>>>>>>>>>> Denis >>>>>>>>>>>> >>>>>>>>>>>>> On Dec 19, 2016, at 9:59 AM, Sergey Kozlov < >>>> skoz...@gridgain.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Yardstick requires own scripts/configurations (/bin, /config, >>>>> /libs) >>>>>>>>>> and >>>>>>>>>>>>> creates work/logs directory under yardstick root. >>>> "libs/optional" >>>>> is >>>>>>>>>> for >>>>>>>>>>>>> optional modules but in general we can't say that for >> Yardstick. >>>>>>>> Also >>>>>>>>>> it >>>>>>>>>>>>> may break the current user understanding of "libs/optional" >>>>>>>> directory >>>>>>>>>> as >>>>>>>>>>>>> place for additonal functionality activated by copying in >>>> "libs". >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Dec 19, 2016 at 7:53 PM, Dmitriy Setrakyan < >>>>>>>>>>>> dsetrak...@apache.org> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> I would be against using libs/optional or libs/ folder for >>>>> anything >>>>>>>>>>>>>> benchmark related. I am also against adding any yardstick >>>>> libraries >>>>>>>>>>>> without >>>>>>>>>>>>>> providing code. >>>>>>>>>>>>>> >>>>>>>>>>>>>> In my view, if the community wants to include benchmarks in >>>>>>>> releases, >>>>>>>>>>>> then >>>>>>>>>>>>>> we should add a "benchmarks" folder, which provides everything >>>>>>>>>>>> benchmark >>>>>>>>>>>>>> related, from code to all the dependent libraries, and >>>>>>>> documentation >>>>>>>>>>>>>> instructions. >>>>>>>>>>>>>> >>>>>>>>>>>>>> D. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Dec 19, 2016 at 8:11 AM, Denis Magda < >>>> dma...@apache.org> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Actually, “libs/optional” is already a kind of extra for me. >>>> Why >>>>>>>> do >>>>>>>>>> we >>>>>>>>>>>>>>> need this new folder if “libs/optional” semantic works well? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is there anyone else who is concerned about “libs/optional”? >>>> If >>>>>>>>>>>> there’re >>>>>>>>>>>>>>> not, I would agree on this and get down to the >> implementation. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> — >>>>>>>>>>>>>>> Denis >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Dec 19, 2016, at 1:10 AM, Sergey Kozlov < >>>>> skoz...@gridgain.com >>>>>>>>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> What's about to introduce the new root folder called >> 'extras' >>>>>>>> with >>>>>>>>>>>>>>>> subfolder 'ignite-yardstick' and put there yardstick >>>> binaries? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 10:02 PM, Denis Magda < >>>>> dma...@apache.org >>>>>>>>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Dmitriy, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please have a look at IGNITE-4212 description ( >>>>>>>>>>>>>>> https://issues.apache.org/ >>>>>>>>>>>>>>>>> jira/browse/IGNITE-4212). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The whole purpose of the ticket is to automate benchmarks >>>>>>>>>> execution >>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>> the end user for a specific Ignite release. Now he/she >> needs >>>>> to >>>>>>>> go >>>>>>>>>>>>>>> through >>>>>>>>>>>>>>>>> a number of steps like build, configure, run strictly >>>>> following >>>>>>>>>>>>>> lengthy >>>>>>>>>>>>>>>>> Yardstick guidance. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Ideally, once a specific release is downloaded it should be >>>>>>>>>> possible >>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> run a concrete benchmark with a ready-to-use script. The >>>>> script >>>>>>>>>>>> needs >>>>>>>>>>>>>>>>> benchmarks' lib which makes sense to put under >>>> “libs/optional” >>>>>>>>>>>> folder. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If someone wants to modify the source of an existed >>>> benchmark >>>>> or >>>>>>>>>>>> add a >>>>>>>>>>>>>>> new >>>>>>>>>>>>>>>>> one then he/she needs to follow existed Yardstick guidance. >>>>> So, >>>>>>>> no >>>>>>>>>>>>>> need >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> release benchmarks’s sources as a part of Ignite release. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> — >>>>>>>>>>>>>>>>> Denis >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Dec 18, 2016, at 7:08 AM, Dmitriy Setrakyan < >>>>>>>>>>>>>> dsetrak...@apache.org> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Sun, Dec 18, 2016 at 2:53 AM, Oleg Ostanin < >>>>>>>>>>>> oosta...@gridgain.com >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Dmitriy, ignite-yardstick allows user to run plenty of >>>>> useful >>>>>>>>>>>>>>> Yardstick >>>>>>>>>>>>>>>>>>> benchmarks, which can be used to check Ignite >> performance. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> In that case, why would it be under the "libs" folder at >>>> all? >>>>>>>> Do >>>>>>>>>> we >>>>>>>>>>>>>>>>> really >>>>>>>>>>>>>>>>>> need to include benchmarks into Ignite? If yes, then I >>>> would >>>>>>>>>>>> create a >>>>>>>>>>>>>>>>>> benchmarks folder under "examples" and add all the >>>> benchmarks >>>>>>>>>>>> there. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 11:49 PM, Dmitriy Setrakyan < >>>>>>>>>>>>>>>>> dsetrak...@apache.org >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Oleg, what does ignite-yardstick module do? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Fri, Dec 16, 2016 at 12:37 AM, Oleg Ostanin < >>>>>>>>>>>>>>> oosta...@gridgain.com> >>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hello Igniters! >>>>>>>>>>>>>>>>>>>>> I'm working on ticket IGNITE-4212 "Ignite Benchmarking >>>>>>>>>>>>>>> Simplification >>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>> Automation" and I'd like to ask your opinion about >>>>>>>>>>>>>> ignite-yardstick: >>>>>>>>>>>>>>>>>>>> where >>>>>>>>>>>>>>>>>>>>> do you think is the most appropriate place to put a >>>>> compiled >>>>>>>>>>>>>>>>>>>>> ignite-yardstick module in the apache-ignite binary >>>>>>>> assembly? >>>>>>>>>> We >>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>> put >>>>>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>>> in the libs/optional along with an others optional >>>>>>>> libraries, >>>>>>>>>> or >>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>> can >>>>>>>>>>>>>>>>>>>>> create a new directory named "tools" in the root >>>> directory >>>>>>>> and >>>>>>>>>>>> put >>>>>>>>>>>>>>>>>>>>> "ignite-yardstick" in it, or we can find another >>>> solution. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Best regards >>>>>>>>>>>>>>>>>>>>> Oleg >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Sergey Kozlov >>>>>>>>>>>>>>>> GridGain Systems >>>>>>>>>>>>>>>> www.gridgain.com >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Sergey Kozlov >>>>>>>>>>>>> GridGain Systems >>>>>>>>>>>>> www.gridgain.com >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Sergey Kozlov >>>>>>>>>>> GridGain Systems >>>>>>>>>>> www.gridgain.com >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Sergey Kozlov >>>>>>>>>> GridGain Systems >>>>>>>>>> www.gridgain.com >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Ilya Suntsov >>>> >> >>
