Hi Tim, All, I made a quick log analysis on the linked job. Just blindly checked the differences between the consecutive lines to look for hotspots, so slow operations that are logging continuously remain hidden. There are some tests that running for minutes, will start checking them later. Also, there is this curiosity, I haven't looked into it yet either: 21:15:09 --> Starting the Sentry Policy Server 21:21:45 Stopping Sentry 21:27:56 19/08/04 21:06:23 INFO testutil.SentryServicePinger: Sentry Service ping succeeded. 21:27:56 --> Starting Ranger Server 21:30:39 Restarting Apache Ranger Admin 21:30:39 Apache Ranger Admin Service is not running 21:34:16 Starting Apache Ranger Admin Service 21:34:16 Apache Ranger Admin Service with pid 41746 has started.
Attached the report to this mail. Best regards, Tamas On Mon, Aug 5, 2019 at 8:28 AM Tim Armstrong <tarmstr...@cloudera.com> wrote: > Hi All, > Just letting you know I tweaked the precommit testing job - > https://jenkins.impala.io/job/parallel-all-tests - so that it runs the > backend and frontend tests separately from the other tests. > > If you have ideas for reducing build+test runtime, please share or > implement them. The frontend authorisation tests are running way too long, > but the long pole is still the other job with the Python tests. I looked at > a recent job ( > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/6942/ > <https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/6942/consoleFull>) > and > here's the time taken per step: > > 0:00:20 Checked out repos > 0:01:04 System setup done > 0:00:27 Hadoop-lzo built > 0:00:33 python bootstrap > 0:00:47 download toolchain and cdh components > 0:29:36 build impala > 0:34:42 load data > 0:45:08 serial tests > 0:03:45 mini stress tests (all skipped or xfailed) > 0:56:16 parallel tests > 1:37:49 custom cluster tests > 0:15:55 auth tests > 0:01:48 archiving > > - Tim >
