Hi François, Thank you very much for your response with very helpful information. I realized that this command is launched as follows at https://ci.ursalabs.org/#/builders/73/builds/100/steps/3/logs/stdio
$ archery benchmark diff --output=diff.json --suite-filter=parquet-encoding-benchmark --benchmark-filter=BM_PlainDecodingBoolean --repetitions=1 WORKSPACE origin/master I will read these files and try to enhance "archery benchmark diff ... " Best Regards, Kazuaki Ishizaki Francois Saint-Jacques <fsaintjacq...@gmail.com> wrote on 2020/08/28 04:51:54: > From: Francois Saint-Jacques <fsaintjacq...@gmail.com> > To: dev <dev@arrow.apache.org> > Date: 2020/08/28 04:52 > Subject: [EXTERNAL] Re: Ursabot Benchmark framework for other languages > > Hello Kazuaki! > > I recommend you read and take a look at the benchmark sub-library [1] > of archery and how it's glued [2]. You will need to implement: > > - A runner for the framework you intend to use [3] and [4], it also > implies capturing the output into a class that implements the > "Benchmark" interface. > - Tweak the glue code such that it doesn't assume that there's only a > CppBenchmarkRunner [5] and probably some change in the command line > UX, e.g. `--language=<lang>` with default to cpp such that it doesn't > break the current workflow. > > Once you have that and it's working locally, you can ask Krizstian > help on how to integrate it with ursabot. > > François > > [1] INVALID URI REMOVED > u=https-3A__github.com_apache_arrow_tree_master_dev_archery_archery_benchmark&d=DwIFaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f- > ZCGj9Pg&m=YL3_uTOJChdZeElefIDnPIv6QJaOSuDKN7kTsLhT5PI&s=6SzHrCW51fn6u_aaWv3lVHfxjtqP69_2lzOXpfyu0Tc&e= > [2] INVALID URI REMOVED > u=https-3A__github.com_apache_arrow_blob_master_dev_archery_archery_cli.py-23L349-2DL577&d=DwIFaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f- > ZCGj9Pg&m=YL3_uTOJChdZeElefIDnPIv6QJaOSuDKN7kTsLhT5PI&s=LJsnpmWSiEzpshVC- > gKrWbgSNXlOmvi7ADX7E4qCAPw&e= > [3] INVALID URI REMOVED > u=https-3A__github.com_apache_arrow_blob_master_dev_archery_archery_benchmark_runner.py-23L133-2DL207&d=DwIFaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f- > ZCGj9Pg&m=YL3_uTOJChdZeElefIDnPIv6QJaOSuDKN7kTsLhT5PI&s=jco8YNDD7bvohax6BO66r6SKbj0rUVr35ZT4Cs3ij04&e= > [4] INVALID URI REMOVED > u=https-3A__github.com_apache_arrow_blob_master_dev_archery_archery_benchmark_google.py-23L33-2DL175&d=DwIFaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f- > ZCGj9Pg&m=YL3_uTOJChdZeElefIDnPIv6QJaOSuDKN7kTsLhT5PI&s=1JxymaI97EC1JGPPYUmWES0VOe5Y6y2rYwd1baXFa0g&e= > [5] INVALID URI REMOVED > u=https-3A__github.com_apache_arrow_blob_master_dev_archery_archery_benchmark_runner.py-23L51-2DL86&d=DwIFaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f- > ZCGj9Pg&m=YL3_uTOJChdZeElefIDnPIv6QJaOSuDKN7kTsLhT5PI&s=Ce3HMKrtkM60N9aDCQSY- > BAVvStfX0DKKF5c_CYGZJQ&e= > > On Thu, Aug 27, 2020 at 1:59 PM Kazuaki Ishizaki <ishiz...@jp.ibm.com> wrote: > > > > I am new to the Ursabot benchmark framework like > > INVALID URI REMOVED > u=https-3A__github.com_apache_arrow_pull_7940-23issuecomment-2D673183390&d=DwIFaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=b70dG_9wpCdZSkBJahHYQ4IwKMdp2hQM29f- > ZCGj9Pg&m=YL3_uTOJChdZeElefIDnPIv6QJaOSuDKN7kTsLhT5PI&s=vxLT4ZdO- > gTIqXZsLumESzBdS4XJsl9gDw3dM4cBQnc&e= . > > > > How can we run benchmark programs written in other languages (e.g. Java) > > beyond C++? If it is not supported yet, what do we need to support to run > > benchmark programs in other languages? > > > > Best Regards, > > Kazuaki Ishizaki, > > >
