On Sat, Jun 15, 2019 at 2:20 PM Jim Apple <apa...@jbapple.com> wrote:
> My goal is to have Impala keep up with (what I perceive to be) the most > popular version of the most popular Linux distribution, for the purpose of > easing the workflow of developers, especially new developers. > Sure, that makes sense. I use Ubuntu 18 myself, but tend to develop Impala on a remote box running el7 because the dev environment is too heavy-weight to realistically run on my laptop. > > 18.04 stopped being able to load data some time between June 9th and > https://jenkins.impala.io/job/ubuntu-18.04-from-scratch/14/ and June 12 > and > > https://jenkins.impala.io/job/ubuntu-18.04-from-scratch/16/artifact/Impala/logs_static/logs/data_loading/catalogd.ERROR/*view*/ > . > I tried reproducing the June 9 run with the same git checkouts (Impala and > Impala-LZO) as #14 today, and data loading still failed. > > What RHEL 7 components did you have in mind that are closer to Ubuntu 16.04 > than 18.04? > Stuff like libc, openssl, krb5, sasl, etc are pretty different version-wise. At least, I know when we made Kudu pass tests on Ubuntu 18, we dealt with issues mostly in those libraries, which aren't part of the toolchain (for security reasons we rely on OS-provided libs). Generally I think precommit running on something closer to the oldest supported OS is better than running on the newest, since it's more likely that new OSes are backward-compatible. Otherwise it's very easy to introduce code that uses features not available on el7, for example. > > On Wed, May 22, 2019 at 10:41 AM Todd Lipcon <t...@cloudera.com> wrote: > > > On Mon, May 20, 2019 at 8:36 PM Jim Apple <apa...@jbapple.com> wrote: > > > > > Maybe now would be a good time to implement Everblue jobs that ping > dev@ > > > when they fail. Thoughts? > > > > > > > Mixed feelings on that. We already get many test runs per day of the > > "default" config because people are running precommit builds. Adding an > > additional cron-based job to the mix that runs the same builds doesn't > seem > > like it adds much unless it tests some other config (eg Ubuntu 18 or a > > longer suite of tests). One thing I could get on board with would be > > switching the precommit builds to run just "core" tests or some other > > faster subset, and defer the exhaustive/long runs to scheduled builds or > as > > an optional precommit for particularly invasive patches. I think that > would > > increase dev quality of life substantially (I find my productivity is > often > > hampered by only getting two shots at a precommit run per work day). > > > > I'm not against adding a cron-triggered full test/build on Ubuntu 18, but > > would like to know if someone plans to sign up to triage it when it > fails. > > My experience with other Apache communities is that collective ownership > > over test triage duty (ie "email the dev list on failure" doesn't work. I > > seem to recall we had such builds back in 2010 or so on Hadoop and they > > just always got ignored. In various "day job" teams I've seen this work > via > > a prescriptive rotation ("all team members take a triage/build-cop > shift") > > but that's not really compatbile with the nature of Apache projects being > > volunteer communities. > > > > So, I think I'll put the question back to you: as a committer you can > spend > > your time as you like. If you think an Ubuntu 18 job running on a > schedule > > would be useful and willing to sign up to triage failures, sounds great > to > > me :) Personally I don't develop on Ubuntu 18 and in my day job it's not > a > > particularly important deployment platform, so I personally don't think > > I'll spend much time triaging that build. > > > > Todd > > > > > > > > > > On Mon, May 20, 2019 at 9:09 AM Todd Lipcon <t...@cloudera.com> wrote: > > > > > > > Adding a build-only job for 18.04 makes sense to me. A full test run > on > > > > every precommit seems a bit expensive but doing one once a week or > > > > something like that might be a good idea to prevent runtime > > regressions. > > > > > > > > As for switching the precommit from 16.04 to 18.04, I'd lean towards > > > > keeping to 16.04 due to it being closer in terms of component > versions > > to > > > > common enterprise distros like RHEL 7. > > > > > > > > -Todd > > > > > > > > On Sun, May 19, 2019 at 5:03 PM Jim Apple <jbap...@apache.org> > wrote: > > > > > > > > > HEAD now passes on Ubuntu 18.04: > > > > > > > > > > https://jenkins.impala.io/job/ubuntu-18.04-from-scratch/ > > > > > > > > > > Thanks to the community members who have made this happen! > > > > > > > > > > Should we add Ubuntu 18.04 to our pre-merge Jenkins job, replace > > 16.04 > > > > with > > > > > 18.04 in our pre-merge Jenkins job, or neither? > > > > > > > > > > I propose adding 18.04 for now (ans so running both 16.04 and 18.04 > > on > > > > > merge) and removing 16.04 when it starts to become inconvenient. > > > > > > > > > > > > > > > > > -- > > > > Todd Lipcon > > > > Software Engineer, Cloudera > > > > > > > > > > > > > -- > > Todd Lipcon > > Software Engineer, Cloudera > > > -- Todd Lipcon Software Engineer, Cloudera
