>From the longest tests these two looks like an easy ~5-8 min win. What do you think?
1) test_permanent_udfs.py::test_java_udfs_hive_integration - runs for ~5 minutes consistently, I think it is not necessary to execute it with this amount of UDFS https://github.com/apache/impala/blob/master/tests/custom_cluster/test_permanent_udfs.py#L187 2) test_breakpad.py::test_abort_writes_minidump - runs for 4 minutes most probably due to the CLUSTER_WAIT_TIMEOUT_IN_SECONDS, this could be set lower for this test https://github.com/apache/impala/blob/master/tests/custom_cluster/test_breakpad.py#L155 https://github.com/apache/impala/blob/master/tests/common/impala_cluster.py#L67 On Tue, Aug 6, 2019 at 2:10 PM Tamas Mate <tm...@cloudera.com> wrote: > The source file was corrupted, that is why the sentry curiosity came up. > Updated the Gist > <https://gist.github.com/tmater/df43cb35a25e7dda65aa24ed50f3fcbe> with > the new report. > > On Mon, Aug 5, 2019 at 8:35 PM Tamas Mate <tm...@cloudera.com> wrote: > >> No, possibly the mail server rejected it without notification. Uploaded >> as a secret Gist >> <https://gist.github.com/tmater/df43cb35a25e7dda65aa24ed50f3fcbe>. >> >> On Mon, Aug 5, 2019 at 7:44 PM Tim Armstrong <tarmstr...@cloudera.com> >> wrote: >> >>> Did you forget to attach it? >>> >>> On Mon, Aug 5, 2019 at 10:32 AM Tamas Mate <tm...@cloudera.com> wrote: >>> >>> > Hi Tim, All, >>> > >>> > I made a quick log analysis on the linked job. Just blindly checked the >>> > differences between the consecutive lines to look for hotspots, so slow >>> > operations that are logging continuously remain hidden. There are some >>> > tests that running for minutes, will start checking them later. Also, >>> there >>> > is this curiosity, I haven't looked into it yet either: >>> > 21:15:09 --> Starting the Sentry Policy Server >>> > 21:21:45 Stopping Sentry >>> > 21:27:56 19/08/04 21:06:23 INFO testutil.SentryServicePinger: Sentry >>> > Service ping succeeded. >>> > 21:27:56 --> Starting Ranger Server >>> > 21:30:39 Restarting Apache Ranger Admin >>> > 21:30:39 Apache Ranger Admin Service is not running >>> > 21:34:16 Starting Apache Ranger Admin Service >>> > 21:34:16 Apache Ranger Admin Service with pid 41746 has started. >>> > >>> > Attached the report to this mail. >>> > >>> > Best regards, >>> > Tamas >>> > >>> > On Mon, Aug 5, 2019 at 8:28 AM Tim Armstrong <tarmstr...@cloudera.com> >>> > wrote: >>> > >>> >> Hi All, >>> >> Just letting you know I tweaked the precommit testing job - >>> >> https://jenkins.impala.io/job/parallel-all-tests - so that it runs >>> the >>> >> backend and frontend tests separately from the other tests. >>> >> >>> >> If you have ideas for reducing build+test runtime, please share or >>> >> implement them. The frontend authorisation tests are running way too >>> long, >>> >> but the long pole is still the other job with the Python tests. I >>> looked >>> >> at >>> >> a recent job ( >>> >> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/6942/ >>> >> < >>> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/6942/consoleFull >>> >) >>> >> and >>> >> here's the time taken per step: >>> >> >>> >> 0:00:20 Checked out repos >>> >> 0:01:04 System setup done >>> >> 0:00:27 Hadoop-lzo built >>> >> 0:00:33 python bootstrap >>> >> 0:00:47 download toolchain and cdh components >>> >> 0:29:36 build impala >>> >> 0:34:42 load data >>> >> 0:45:08 serial tests >>> >> 0:03:45 mini stress tests (all skipped or xfailed) >>> >> 0:56:16 parallel tests >>> >> 1:37:49 custom cluster tests >>> >> 0:15:55 auth tests >>> >> 0:01:48 archiving >>> >> >>> >> - Tim >>> >> >>> > >>> > >>> >> >> >> -- >> Tamas Mate >> Senior Customer Operations Engineer >> Cloudera >> > > > -- > Tamas Mate > Senior Customer Operations Engineer > Cloudera > -- Tamas Mate Senior Customer Operations Engineer Cloudera
