Re: New PMC member: Laszlo Gaal

[Sailesh Mukil]

Congratulations, Laszlo!

On Mon, Jan 24, 2022 at 8:02 AM Laszlo Gaal 
wrote:

> Thank you all very much!
> It is great, and truly rewarding to belong to such a superb community.
>
>   - Laszlo
>
> On Mon, Jan 24, 2022 at 9:56 AM Gabor Kaszab 
> wrote:
>
> > Congrats, Laci!
> >
> > On Mon, Jan 24, 2022 at 9:04 AM Dinesh Garg 
> wrote:
> >
> > > Well deserved , long overdue . Congratulations Laszlo
> > >
> > > On Mon, Jan 24, 2022 at 12:26 PM skyyws  wrote:
> > >
> > > > Congratulations Laszlo!
> > > >
> > > >
> > > > | |
> > > > skyyws
> > > > |
> > > > |
> > > > sky...@163.com
> > > > |
> > > > 签名由网易邮箱大师定制
> > > >
> > > >
> > > > On 01/22/2022 03:46，Zoltán Borók-Nagy wrote：
> > > > The Project Management Committee (PMC) for Apache Impala has invited
> > > Laszlo
> > > > Gaal to become a PMC member and we are pleased to announce that they
> > have
> > > > accepted.
> > > >
> > > > Congratulations and welcome, Laszlo!
> > > >
> > > --
> > > Sent from Gmail Mobile
> > >
> >
>

Re: Welcome Paul Rogers as a Committer

[Sailesh Mukil]

Congratulations, Paul!

On Tue, Feb 5, 2019 at 4:34 PM Quanlong Huang 
wrote:

> Congratulations, Paul!
>
> On Wed, Feb 6, 2019 at 7:53 AM Zoram Thanga  wrote:
>
> > Congratulations, Paul!
> >
> > On Tue, Feb 5, 2019 at 12:54 PM Vihang Karajgaonkar  >
> > wrote:
> >
> > > Congratulations Paul!
> > >
> > > On Tue, Feb 5, 2019 at 11:46 AM Bharathkrishna Guruvayoor Murali <
> > > bhar...@cloudera.com> wrote:
> > >
> > > > Congratulations Paul!
> > > >
> > > > On Tue, Feb 5, 2019 at 11:28 AM Andrew Sherman <
> asher...@cloudera.com>
> > > > wrote:
> > > >
> > > > > Well done Paul!
> > > > >
> > > > > On Tue, Feb 5, 2019 at 11:16 AM Fredy Wijaya  >
> > > > wrote:
> > > > >
> > > > > > Congratulations, Paul!
> > > > > >
> > > > > > On Tue, Feb 5, 2019 at 12:58 PM Yongjun Zhang <
> yzh...@cloudera.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Congratulations Paul! Super well done!
> > > > > > >
> > > > > > > --Yongjun
> > > > > > >
> > > > > > > On Mon, Feb 4, 2019 at 6:25 PM Philip Zeyliger <
> > > phi...@cloudera.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > The Project Management Committee (PMC) for Apache Impala has
> > > > invited
> > > > > > Paul
> > > > > > > > Rogers to become a committer and we are pleased to announce
> > that
> > > > they
> > > > > > > have
> > > > > > > > accepted.
> > > > > > > > Congratulations and welcome, Paul!
> > > > > > > >
> > > > > > > > On behalf of the PMC,
> > > > > > > >
> > > > > > > > -- Philip
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> > Zoram Thanga
> > Cloudera Inc.
> > 395 Page Mill Road
> > Palo Alto, CA 94306
> >
>

Re: expr-benchmark

[Sailesh Mukil]

The benchmarks weren't being actively maintained last I saw (6 months back
or so), and most of them aren't run regularly as well. So a lot of the
dependent code may have changed causing them to break. So, there's no
guarantee that they will work.

On Sat, Sep 8, 2018 at 10:08 AM Jim Apple  wrote:

> I can't get exp-benchmark working near HEAD. Has anyone else seen and
> solved this? We don't run benchmarks in pre-commit testing, so I'm not
> sure if this is just me, perhaps something abut it not being hermetic,
> or if this has been broken for a while.
>
> 18/09/08 10:00:40 INFO util.JvmPauseMonitor: Starting JVM pause monitor
> Wrote minidump to
> /tmp/minidumps/expr-benchmark/11ed5026-eff4-4214-639a048a-0389c5c4.dmp
> Wrote minidump to
> /tmp/minidumps/expr-benchmark/11ed5026-eff4-4214-639a048a-0389c5c4.dmp
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00e99791, pid=30887,
> tid=0x7fb47665f8c0
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_131-b11) (build
> 1.8.0_131-b11)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.131-b11 mixed mode
> linux-amd64 compressed oops)
> # Problematic frame:
> # C  [expr-benchmark+0xa99791]
> impala::DescriptorTbl::GetSlotDescriptor(int) const+0x1
> #
> # Failed to write core dump. Core dumps have been disabled. To enable
> core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /home/jbapple/Impala/logs/hs_err_pid30887.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
>

Re: #pragma once?

[Sailesh Mukil]

An advantage of using #pragma once is potential improved compilation
speeds. However, a con is that it's non-standard and therefore, its
behavior can change at any point and can also vary across compilers,
potentially making the code even less portable.

That being said, since Kudu has been using it for a while and has had no
issues, we can do the same since the potential benefits outweigh the cons.

On Wed, Aug 1, 2018 at 11:48 AM, Tim Armstrong <
tarmstr...@cloudera.com.invalid> wrote:

> Todd brought up our include guards on a code review, asking why we don't
> use #pragma once instead: https://gerrit.cloudera.org/#/c/10988/5 . It
> sounds like Kudu has switched to it
>
> #pragma once does seem cleaner and our GCC and Clang versions are modern
> enough to support it.
>
> What do people think about switching to that as the preferred way of
> including headers only once?
>
> - Tim
>

Re: Order of error messages printed

[Sailesh Mukil]

The ErrorLogMap's key is "TErrorCode::type" which is an Enum. I don't think
we can derive any inherent meaning from the order of the errors listed in
the enum (other than OK=0, which wouldn't be in this map anyway). I'm not
aware of any clients relying on this order, and even if there were, it
would probably be incorrect to do so. So, unless anyone else chimes in with
a good reason to keep this as a map instead of an unordered_map, I'm all
for changing it.

On Tue, Jul 24, 2018 at 12:12 PM, Michael Ho 
wrote:

> Hi,
>
> While making some changes for be/src/util/error-util.cc, I stumbled upon
> the question of whether we need a map instead of an unordered map for
> ErrorLogMap (used for tracking error messages for each error code). This
> affects the order in which error messages are printed in PrintErrorMap().
> Looking through the code, it doesn't appear to me that there is any
> ordering requirement. However, it's unclear to me if any client of Impala
> makes assumption about the ordering of the output in PrintErrorMap(). So,
> sending this email to the list in case anyone knows anything.
>
> --
> Thanks,
> Michael
>

Re: impalad running as superuser in tests

[Sailesh Mukil]

On Wed, Jul 18, 2018 at 10:21 AM, Todd Lipcon 
wrote:

> On Tue, Jul 17, 2018 at 5:27 PM, Sailesh Mukil
>  > wrote:
>
> > On Tue, Jul 17, 2018 at 2:47 PM, Todd Lipcon 
> > wrote:
> >
> > > Hey folks,
> > >
> > > I'm working on a regression test for IMPALA-7311 and found something
> > > interesting. It appears that in our normal minicluster setup, impalad
> > runs
> > > as the same username as the namenode (namely, the username of the
> > > developer, in my case 'todd').
> > >
> > > This means that the NN treats impala as a superuser, and therefore
> > doesn't
> > > actually enforce permissions. So, tests about the behavior of Impala on
> > > files that it doesn't have access to are somewhat tricky to write.
> > >
> > >
> > What kind of files do you specifically mean? Something that the daemon
> > tries to access directly (Eg: keytab file, log files, etc.) ? I'm
> guessing
> > it's not this since you mentioned the NN.
> >
> > Or files that belong to a table/partition in HDFS? If it's this case, we
> > would go through Sentry before accessing files that belong to a table,
> and
> > access would be determined by Sentry on the "session user" (not the
> impalad
> > user) before Impala even tries to access HDFS. (Eg:
> > tests/authorization/test_authorization.py)
> >
>
> Right, files on HDFS. I mean that, in cases where Sentry is not enabled or
> set up, and even in some cases where it is set up but not synchronized with
> HDFS, it's possible that the user can point table metadata at files or
> directories that aren't writable to the 'impala' user on HDFS. For example,
> I can do:
>
> CREATE EXTERNAL TABLE foo (...) LOCATION '/user/todd/my-dir';
>
> and it's likely that 'my-dir' is not writable by 'impala' on a real
> cluster. Thus, if I try to insert into it, I get an error because "impala"
> does not have HDFS permissions to access this directory.
>
> Currently, the frontend does some checks here to try to produce a nice
> error. But, those checks are based on cached metadata which could be in
> accurate. In the case that it's inaccurate, the error will be thrown from
> the backend when it tries to create a file in a non-writable location.
>
> In the minicluster environment, it's impossible to test this case (actual
> permissions enforced by the NN causing an error) because the backend is
> running as an HDFS superuser. That is to say, it has full permissions
> everywhere. That's due to the special case behavior that HDFS has: it
> determines the name of the superuser to be the username that is running the
> NN. Since in the minicluster, both impala and the NN run as 'todd' in my
> case, impala acts as superuser. In a real cluster (even with security
> disabled) impala typically runs as 'impala' whereas the NN runs as 'hdfs'
> and thus impala does not have superuser privileges.
>

This makes sense, thanks for the explanation. The 'HADOOP_USER_NAME'
approach seems like a good way to go, but as Phil said, might cause issues
with other components (or not).


> -Todd
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: impalad running as superuser in tests

[Sailesh Mukil]

On Tue, Jul 17, 2018 at 2:47 PM, Todd Lipcon 
wrote:

> Hey folks,
>
> I'm working on a regression test for IMPALA-7311 and found something
> interesting. It appears that in our normal minicluster setup, impalad runs
> as the same username as the namenode (namely, the username of the
> developer, in my case 'todd').
>
> This means that the NN treats impala as a superuser, and therefore doesn't
> actually enforce permissions. So, tests about the behavior of Impala on
> files that it doesn't have access to are somewhat tricky to write.
>
>
What kind of files do you specifically mean? Something that the daemon
tries to access directly (Eg: keytab file, log files, etc.) ? I'm guessing
it's not this since you mentioned the NN.

Or files that belong to a table/partition in HDFS? If it's this case, we
would go through Sentry before accessing files that belong to a table, and
access would be determined by Sentry on the "session user" (not the impalad
user) before Impala even tries to access HDFS. (Eg:
tests/authorization/test_authorization.py)

If you're describing a different scenario than the above 2 I mentioned,
then I'd be interested to hear that, so it'll be easier to understand why
this change is necessary.

Has anyone run into this before? Should we consider running either the
> impalad or the namenode as a different spoofed username so that the
> minicluster environment is more authentic to true cluster environments? We
> can do this easily by setting the HADOOP_USER_NAME environment variable or
> system property.
>
> -Todd
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: Removing IMPALA_MINICLUSTER_PROFILE=2 from master

[Sailesh Mukil]

+1 for removing it.

I'm all for reduced complexity if not a lot of users are benefitting from
it.

On Wed, Jul 11, 2018 at 9:29 AM, Fredy Wijaya 
wrote:

> Hi all,
>
> In the master branch, we support both Hadoop 2 and Hadoop 3 via
> IMPALA_MINICLUSTER_PROFILE environment variable. When Impala transitioned
> from Hadoop 2 to Hadoop 3, we introduced IMPALA_MINICLUSTER_PROFILE as a
> way to easily switch between Hadoop 2 and Hadoop 3. It made sense at that
> time. However, I believe the IMPALA_MINICLUSTER_PROFILE=2 has outlived its
> usefulness and has grown more complex to maintain given that we need to
> introduce shims, feature flags, additional Jenkins jobs, etc. Given that we
> already have 2.x branch for Impala with Hadoop 2 support. I'm proposing
> that we remove IMPALA_MINICLUSTER_PROFILE=2 in the master branch so that we
> can focus our effort on supporting Hadoop 3 on the master branch.
>
> What do others think?
>

Re: Impala OS and Java version support

[Sailesh Mukil]

I'm in favor of the minimum OS version support proposal. From 3.x, I don't
see any reason to support much older OSes especially since it's becoming
much more burdensome to special case code for them and have those paths
tested.

I'll defer to others on the minimum Java version proposal since they may
have better insight.

On Fri, Jul 6, 2018 at 9:19 AM, Lars Volker  wrote:

> Hi All,
>
> With the automatic commit flow to the 2.x branch being disabled, I'd like
> to suggest that we deprecate support for some older operating systems on
> the 3.x branch. This would allow us to get rid of almost all the checks in
> common/config.h.in as well as some workarounds in the code. It would also
> simplify the upcoming rebase of the KRPC code (IMPALA-7006).
>
> I'd like to propose that we pick Ubuntu 14.04, CentOS 6.8, and SLES 12 as
> the minimum operating system versions on which Impala has to compile and
> pass all tests. I'd also like to deprecate support for Java 7 and require
> Java 8 for all changes going forward. To my knowledge these are the minimum
> versions that people use for development and deployment of the 3.x branch.
>
> I'm curious what others think.
>
> Cheers, Lars
>

Re: boost::scoped_ptr vs std::unique_ptr

[Sailesh Mukil]

I'm in favor.

Since the main distinction is that a unique_ptr is moveable, whereas a
scoped_ptr is not, we should just make sure that we do our due diligence
during code reviews so that we catch those cases.

Also, making a unique_ptr const disallows moving it, since the move
constructor takes a non-const unique_ptr container. It probably won't work
in all places, but we could enforce that in certain parts of the code.

On Thu, Jul 5, 2018 at 4:49 PM, Thomas Tauber-Marshall <
tmarsh...@cloudera.com.invalid> wrote:

> I'm definitely in favor of using more standard c++ to reduce both confusion
> and our reliance on boost, especially as I suspect a lot of people (eg. me)
> don't know the subtle difference between scoped_ptr and unique_ptr off the
> top of their head anyways.
>
> Fwiw, I was under the impression from talking with people in the past that
> we were already trying to make this move, and the
> PartitionedAggregationNode refactor that just went in made the switch to
> unique_ptr, though no one commented on it in the review.
>
> On Thu, Jul 5, 2018 at 4:39 PM Tim Armstrong
>  wrote:
>
> > I was just talking with Michael Ho on a review about this
> > https://gerrit.cloudera.org/#/c/10810/7/be/src/exec/scan-node.h@271
> >
> > For a while we've continued using scoped_ptr in some places because it
> > supports a smaller set of operators and implies that the pointer isn't
> > movable. See
> >
> > https://cwiki.apache.org/confluence/display/IMPALA/
> Resource+Management+Best+Practices+in+Impala
> > .
> >
> > I don't think we're consistently following this pattern and it seems to
> > cause some confusion about what the best practice is, particularly for
> > people coming from other code bases. I personally like the distinction,
> but
> > I don't feel that strongly about it.
> >
> > What do people think? Should we continue using scoped_ptr or move away
> from
> > it. There is already a JIRA to make the change but we haven't done it
> > because of the above reasons:
> > https://issues.apache.org/jira/browse/IMPALA-3444
> >
> > - Tim
> >
>

Re: Automatically rebase changes before GVO?

[Sailesh Mukil]

+1

On Mon, Jun 11, 2018 at 3:02 PM, Jim Apple  wrote:

> No objection from me.
>
> On Mon, Jun 11, 2018 at 12:06 PM, Tim Armstrong 
> wrote:
>
> > > On nit: as GVD gets more complex, it becomes harder for new people to
> > understand the messages and +Ns applied to their patches. That doesn't
> mean
> > we shouldn't do this, only that it's something to keep an eye on.
> >
> > I think this helps with that problem in net by removing the manual rebase
> > step that people have to remember to do.
> >
> >
> > On Mon, Jun 11, 2018 at 12:04 PM, Tim Armstrong  >
> > wrote:
> >
> > > I've tried my job a few times and it's working as expected. Any
> > objections
> > > to me switching over gerrit-verify-dryrun to my approach?
> > >
> > > On Thu, Jun 7, 2018 at 2:42 PM, Tim Armstrong  >
> > > wrote:
> > >
> > >> Ok, I was able to put together a test job that does the automatic
> rebase
> > >> and carries a +2 here: https://jenkins.impala.io/job/
> > >> gerrit-verify-dryrun-tarmstrong/
> > >>
> > >> The diff in job config required to get it to work is here:
> > >> https://jenkins.impala.io/job/gerrit-verify-dryrun-tarmstron
> > >> g/jobConfigHistory/showDiffFiles?timestamp1=2018-06-07_20-
> > >> 41-18×tamp2=2018-06-07_21-38-58
> > >>
> > >> I tested by creating some private drafts, adding "Impala Public
> Jenkins"
> > >> as a reviewer and testing the various scenarios.
> > >>
> > >> On Thu, Jun 7, 2018 at 2:26 PM, Jim Apple 
> wrote:
> > >>
> > >>> I agree with both of you.
> > >>>
> > >>> On nit: as GVD gets more complex, it becomes harder for new people to
> > >>> understand the messages and +Ns applied to their patches. That
> doesn't
> > >>> mean
> > >>> we shouldn't do this, only that it's something to keep an eye on.
> > >>>
> > >>> On Thu, Jun 7, 2018 at 1:28 PM, Philip Zeyliger  >
> > >>> wrote:
> > >>>
> > >>> > Seems fine, especially since we do the rebase as our submission
> > >>> strategy
> > >>> > anyway, so we're already accepting/testing something that's likely
> to
> > >>> get
> > >>> > rebased, and we may as well minimize that window.
> > >>> >
> > >>> > I'd be in favor of the bot also carrying the votes.
> > >>> >
> > >>> >
> > >>> >
> > >>> > On Thu, Jun 7, 2018 at 1:24 PM, Tim Armstrong <
> > tarmstr...@cloudera.com
> > >>> >
> > >>> > wrote:
> > >>> >
> > >>> > > One annoyance with our precommit job is the requirement to
> manually
> > >>> > rebase
> > >>> > > the change before starting the merge. Failure to do so either
> leads
> > >>> to
> > >>> > > false positives or false negatives - builds that failed because
> > they
> > >>> were
> > >>> > > missing a flaky/broken test fix and builds that succeeded despite
> > >>> > > interacting badly with a previous fix.
> > >>> > >
> > >>> > > What do people think about modifying gerrit-verify-dryrun to
> > >>> > automatically
> > >>> > > rebase the patch (by the programmatic equivalent of hitting the
> > >>> "Rebase"
> > >>> > > button) at the start of the job? The patch author would still
> have
> > to
> > >>> > carry
> > >>> > > the +2 but this might make our lives a bit easier.
> > >>> > >
> > >>> > > - Tim
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

Re: New message on your code reviews when cherry-picking your change fails.

[Sailesh Mukil]

Thanks for doing this!

On Fri, May 11, 2018 at 12:17 PM, Tim Armstrong 
wrote:

> Thanks for setting this up, this should make things a lot easier.
>
> On Fri, May 11, 2018 at 12:02 PM, Philip Zeyliger 
> wrote:
>
> > Hey folks,
> >
> > I just updated https://jenkins.impala.io/job/cherrypick-2.x-and-test/ to
> > comment on a code review the first time a change causes the
> cherry-picking
> > job to fail. A sample e-mail is forwarded below, with the message
> > highlighted by me. I've used Gerrit to send e-mail, which, frankly, was
> > easier to implement for me, but it does end up possibly sending to more
> > people than a more targeted approach.
> >
> > Hopefully this helps get failed cherrypicks noticed and handled
> > expediently.
> >
> > Please holler if you either notice this being buggy or if you hate it. I
> > figure we can try it for a week and rip it up if we don't like it.
> >
> > -- Philip
> >
> > -- Forwarded message --
> > From: Impala Public Jenkins (Code Review) 
> > Date: Fri, May 11, 2018 at 11:58 AM
> > Subject: [Impala-ASF-CR] Update version to 3.1.0-SNAPSHOT
> > To: Sailesh Mukil , impala...@cloudera.com,
> > revi...@impala.incubator.apache.org
> >
> >
> > Impala Public Jenkins *posted comments* on this change.
> >
> > View Change <http://gerrit.cloudera.org:8080/10360>
> >
> > Patch set 2:
> >
> > This change did not cherrypick successfully into branch 2.x. To resolve
> > this, please do the cherry-pick manually and submit it to Gerrit at
> > refs/for/2.x or add an exception to the branch 2.x copy of
> > bin/ignored_commits.json. Thanks, your friendly bot at
> > https://jenkins.impala.io/job/cherrypick-2.x-and-test/479/ .
> >
> >
> > To view, visit change 10360 <http://gerrit.cloudera.org:8080/10360>. To
> > unsubscribe, visit settings <http://gerrit.cloudera.org:8080/settings>.
> > Gerrit-Project: Impala-ASF
> > Gerrit-Branch: master
> > Gerrit-MessageType: comment
> > Gerrit-Change-Id: Ia2e2df73d27fa332d17fec4e9aa469ea91b14167
> > Gerrit-Change-Number: 10360
> > Gerrit-PatchSet: 2
> > Gerrit-Owner: Sailesh Mukil 
> > Gerrit-Reviewer: Impala Public Jenkins  cloudera.com
> > >
> > Gerrit-Reviewer: Michael Brown 
> > Gerrit-Reviewer: Philip Zeyliger 
> > Gerrit-Comment-Date: Fri, 11 May 2018 18:58:13 +
> > Gerrit-HasComments: No
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "impala-cr" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to impala-cr+unsubscr...@cloudera.com.
> > For more options, visit https://groups.google.com/a/
> cloudera.com/d/optout.
> >
>

Re: [ANNOUNCE] Apache Impala 3.0.0 release

[Sailesh Mukil]

My pleasure!

On Thu, May 10, 2018 at 9:16 AM, Tim Armstrong 
wrote:

> Thanks Sailesh!
>
> On Wed, May 9, 2018 at 10:09 PM, Jim Apple  wrote:
>
> > Thanks to Sailesh for managing the release, and thanks to the many, many
> > others who contributed to creating this new branch and maintaining the
> > relationship between the two branches.
> >
> > On Wed, May 9, 2018 at 1:37 PM, Sailesh Mukil 
> wrote:
> >
> > > The Apache Impala team is pleased to announce the release of Impala
> > 3.0.0.
> > >
> > > Impala is a high-performance distributed SQL engine.
> > >
> > > The release is available at: https://impala.apache.org/downloads.html
> > >
> > > Thanks,
> > >
> > > The Apache Impala team
> > >
> > >
> > >
> > >
> >
>

Re: Jenkins Upgrade

[Sailesh Mukil]

Jenkins is back up.

On Wed, May 9, 2018 at 9:47 PM, Sailesh Mukil  wrote:

> I'll be performing a Jenkins upgrade shortly, and will have it in the
> "Prepare for restart" mode. I'll send in a mail when the upgrade is done.
>

Jenkins Upgrade

[Sailesh Mukil]

I'll be performing a Jenkins upgrade shortly, and will have it in the
"Prepare for restart" mode. I'll send in a mail when the upgrade is done.

[ANNOUNCE] Apache Impala 3.0.0 release

[Sailesh Mukil]

The Apache Impala team is pleased to announce the release of Impala 3.0.0.

Impala is a high-performance distributed SQL engine.

The release is available at: https://impala.apache.org/downloads.html

Thanks,

The Apache Impala team

Re: Maintaining Impala Documentation Sets for Releases

[Sailesh Mukil]

#3 is best. I'm doing something similar for the maintaining the latest
releases for 2.x and 3.x in the "Downloads" page.

Patch: https://gerrit.cloudera.org/#/c/10333/

On Mon, May 7, 2018 at 4:18 PM, Lars Volker  wrote:

> I'm in favor of #3, too. Ideally we'd have proper pages for both releases,
> including docs, and downloads.
>
> On Mon, May 7, 2018 at 4:14 PM, Jim Apple  wrote:
>
> > I'd advocate #3, at least until we are no longer regularly making new 2.x
> > releases.
> >
> > On Mon, May 7, 2018 at 3:06 PM, Alexandra Rodoni 
> > wrote:
> >
> > > I would like to get your feedback on how to manage Impala doc site when
> > we
> > > have 2 parallel release streams, 2.x and 3.x.
> > >
> > > Currently, https://impala.apache.org/impala-docs.html has the HTML
> > version
> > > of the latest version doc set. For older versions, only the PDF
> versions
> > > are available.
> > >
> > > Options are:
> > >
> > > 1. Have the latest HTML version doc available. This can look confusing,
> > for
> > > example, when 2.13 shows as the latest, but 3.0 is listed in the Older
> > > Release section.
> > >
> > > 2. Have the highest release number as the latest and offer the HTML for
> > > that version. So, 3.0 will still be the latest release when 2.13 comes
> > out.
> > > And 2.13 docs will be only available in PDF when released.
> > >
> > > 3. Revamp the doc page and the workflow so that we can have both 2.x
> and
> > > 3.x docs in HTML.
> > >
> > > Let me know what you think is the best option.
> > >
> > > Thanks.
> > > alex rodoni
> > >
> >
>

[RESULT] Vote on Impala 3.0.0 RC1

[Sailesh Mukil]

https://lists.apache.org/thread.html/c1be7c5d3b69986c854b7f17d616c7c454cac77a9d13bb1041b371ce@%3Cdev.impala.apache.org%3E

The vote has passed with the following tally:

+1 (binding):
- Jim Apple
- Thomas Tauber-Marshall
- Sailesh Mukil
- Lars Volker
- Philip Zeyliger

-1:  None


Thank you for participating in the vote!

- Sailesh

Re: [VOTE] 3.0.0 release candidate 1

[Sailesh Mukil]

The vote has passed with the following tally:

+1 (binding):
- Jim Apple
- Thomas Tauber-Marshall
- Sailesh Mukil
- Lars Volker
- Philip Zeyliger

-1:  None


Thank you for participating in the vote!

On Thu, May 3, 2018 at 10:41 AM, Philip Zeyliger 
wrote:

> +1 (Binding)
>
> I checked the hashes and the GPG signatures. Checked the tree hash. Diffed
> the tarball with the git RC tag.
>
> I also ran tests via docker/test-with-docker successfully.
>
> -- Philip
>
>
>
> On Tue, May 1, 2018 at 4:33 PM, Sailesh Mukil 
> wrote:
>
> > This is a vote to release Impala 3.0.0
> >
> > The artifacts for testing can be downloaded from:
> > https://dist.apache.org/repos/dist/dev/impala/3.0.0/RC1/
> > Git tag: 3.0.0-rc1
> > Tree hash: d5bf1fdeb20ac332ce48c6a2b0588bcaf7ac9cb0
> >
> > Please vote +1 or -1. -1 votes should be accompanied by an explanation of
> > the reason. Only PMC members have binding votes, but other
> > community members are encouraged to cast non-binding votes. This vote
> will
> > pass if there are 3 binding +1 votes and more binding +1 votes than -1
> > votes.
> >
> > Please also note that this release is a new major release, and does not
> > override the recently released 2.12.0 as the latest release. Rather,
> 2.12.0
> > is the latest release from the 2.x branch and 3.0.0 is the first and
> latest
> > release from the 3.x branch.
> >
> > This wiki page describes how to check the release before you vote:
> > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#
> >
> > The vote will be open until 04:30 PM PST on Friday, May 4
> >
>

Re: 2.x branch divergence, its implications and what you can do.

[Sailesh Mukil]

Thanks Phil and Tim, they're good points.

>From the above, I can gather that although there is some extra burden on
authors, we can keep it the way it is, and add onto Phil's Jenkins email
notification change to explicitly notify authors when cherry-picks to 2.x
fail, since failures going unnoticed is 80% of the problem.

A review bot warning is also a good idea (but not necessarily a must have),
especially for when people forget to add "not for 2.x". A default comment
of "Are you sure this should go into 2.x" would help.

I don't think we're at a point where a majority of commits shouldn't go
into '2.x' from 'master'. We can switch to explicit cherry-picking when
that time comes. Of course if people feel differently, they should speak up.

- Sailesh

On Thu, May 3, 2018 at 12:53 PM, Philip Zeyliger 
wrote:

> I don't like writing "Cherry-picks: not for 2.x" in commit messages and
> then subsequently doing the cherry-pick. That's sending a confusing message
> in the commit message. We can re-word the sigils if we want to say
> something about "don't cherry-pick automatically but I'll do it myself" but
> that just seems to delay the inevitable, and, all things equal, I'd rather
> the ordering be consistent.
>
> Is the problem the "unnoticed" part of your original e-mail? I'm starting
> to make it possible for jenkins.impala.io to send e-mails, so it'll be
> possible for it to send e-mails to the committer of the first broken
> change, nudging them along. Would that help?
>
> Should we stop automatically cherry-picking everything? Now that 3.0 and
> 2.12 are (nearly/already) released, should we switch to explicit
> cherry-picking?
>
> I'd also be quite happy to start experimenting with review bots that
> comment on things like "is this a clean cherry-pick" on reviews
> automatically.
>
> -- Philip
>
>
> On Thu, May 3, 2018 at 12:35 PM, Tim Armstrong 
> wrote:
>
> > Maybe we should just start with posting back a warning to the code review
> > if the change doesn't cleanly apply?
> >
> > On Thu, May 3, 2018 at 12:34 PM, Tim Armstrong 
> > wrote:
> >
> > > It would be good to encourage people to proactively do cherry-picks
> but I
> > > think it needs a bit more thought before automating it or adding more
> > > barriers.
> > >
> > > That could potentially cause hold-ups if there's a dependency on a
> > > previous patch. E.g. if I'm merging two patches B and C that depend on
> > each
> > > other and B gets merged, then the state while B is being tested is:
> > >
> > > Master
> > >  B->A->...
> > >
> > > 2.x
> > >  A->...
> > >
> > > and B is in the process of being tested, then what you really want is
> for
> > > a new patch to cleanly apply on top of B, but I think your proposed
> check
> > > is that it applies on top of A.
> > >
> > > It's also introduces the possibility of more mistakes if we encourage
> > > people to do the cherry-pick to 2.x before precommit tests for master
> > have
> > > passed, since they may be cherry-picking a version that doesn't pass
> > those
> > > tests. People can of course repeat the cherry-picking process each time
> > > they push out a new revision but it does introduce more risk of
> > > accidentally diverging your master and 2.x commits versus just doing
> the
> > > conflict resolution once.
> > >
> > > On Thu, May 3, 2018 at 12:17 PM, Lars Volker  wrote:
> > >
> > >> Should we add a check to the master GVO that makes sure that a change
> > >> either applies to 2.x, or has a "not for 2.x" label?
> > >>
> > >> On Thu, May 3, 2018 at 12:12 PM, Sailesh Mukil 
> > >> wrote:
> > >>
> > >> > It's been a while since we started maintaining 2 active branches,
> > master
> > >> > and 2.x, and the divergence between them has grown quite a bit.
> > >> >
> > >> > The implications of this (as Michael Brown brought to my attention)
> > are
> > >> > that a patch that goes into 'master' may not cleanly apply to '2.x',
> > >> which
> > >> > would fail the cherrypick-2.x-and-test(
> > >> > https://jenkins.impala.io/job/cherrypick-2.x-and-test/) job, and if
> > >> > unnoticed, can hold up all future cherry-picks from 'master' to
> '2.x&

Re: 2.x branch divergence, its implications and what you can do.

[Sailesh Mukil]

On Thu, May 3, 2018 at 12:17 PM, Lars Volker  wrote:

> Should we add a check to the master GVO that makes sure that a change
> either applies to 2.x, or has a "not for 2.x" label?
>
>
I like this idea. Let me open a JIRA for it.


> On Thu, May 3, 2018 at 12:12 PM, Sailesh Mukil 
> wrote:
>
> > It's been a while since we started maintaining 2 active branches, master
> > and 2.x, and the divergence between them has grown quite a bit.
> >
> > The implications of this (as Michael Brown brought to my attention) are
> > that a patch that goes into 'master' may not cleanly apply to '2.x',
> which
> > would fail the cherrypick-2.x-and-test(
> > https://jenkins.impala.io/job/cherrypick-2.x-and-test/) job, and if
> > unnoticed, can hold up all future cherry-picks from 'master' to '2.x'
> until
> > this is resolved.
> >
> > If this does happen, the original author of the patch needs to manually
> > cherry-pick the patch from 'master' to '2.x', resolve the conflicts and
> > push a change for review to the '2.x' branch with the same Change-Id.
> >
> > To avoid these types of failures, I advise authors to locally cherry pick
> > their patch to '2.x' to see if it applies cleanly before running GVO.
> >
> > Lastly, if a patch is meant only for 'master', and not for '2.x', please
> > remember to add the "Cherry-picks: not for 2.x" line to your commit
> > message. There have been a few patches in which this was not included,
> > which unfortunately requires manual work to resolve later on.
> >
> > Thank you for your time!
> >
> > - Sailesh
> >
>

Re: 2.x branch divergence, its implications and what you can do.

[Sailesh Mukil]

That's one way to go about it. To elaborate:
Add "Cherry-picks: not for 2.x" in the original patch that goes into
'master', manually cherry-pick to '2.x', resolve conflicts and push.
However, this way, if there are multiple patches being merged, we could end
up with a different order of commits in '2.x' than in 'master'.

Another way would be to anticipate this failure, let GVO for that patch
complete in master, and do the manual cherry-pick+resolve+push after. This
way, the order of commits in '2.x' would match the order of commits in
'master'.

On Thu, May 3, 2018 at 12:16 PM, Jim Apple  wrote:

> >
> > To avoid these types of failures, I advise authors to locally cherry pick
> > their patch to '2.x' to see if it applies cleanly before running GVO.
> >
>
> And thus commit to 2.x a patch that has not passed GVO?
>
> Just making sure I understand the framework.
>

2.x branch divergence, its implications and what you can do.

[Sailesh Mukil]

It's been a while since we started maintaining 2 active branches, master
and 2.x, and the divergence between them has grown quite a bit.

The implications of this (as Michael Brown brought to my attention) are
that a patch that goes into 'master' may not cleanly apply to '2.x', which
would fail the cherrypick-2.x-and-test(
https://jenkins.impala.io/job/cherrypick-2.x-and-test/) job, and if
unnoticed, can hold up all future cherry-picks from 'master' to '2.x' until
this is resolved.

If this does happen, the original author of the patch needs to manually
cherry-pick the patch from 'master' to '2.x', resolve the conflicts and
push a change for review to the '2.x' branch with the same Change-Id.

To avoid these types of failures, I advise authors to locally cherry pick
their patch to '2.x' to see if it applies cleanly before running GVO.

Lastly, if a patch is meant only for 'master', and not for '2.x', please
remember to add the "Cherry-picks: not for 2.x" line to your commit
message. There have been a few patches in which this was not included,
which unfortunately requires manual work to resolve later on.

Thank you for your time!

- Sailesh

Re: [VOTE] 3.0.0 release candidate 1

[Sailesh Mukil]

+1 (binding)

- Validated the sha512 file
- Validated the GPG signature
- Ran the release test job:
https://jenkins.impala.io/view/Utility/job/release-test-ub1604/12/

On Wed, May 2, 2018 at 9:17 PM, Jim Apple  wrote:

> +1, binding
>
> I looked at the output of the release testing job that Sailesh appears to
> have run on jenkins.impala.io.
>
> On Tue, May 1, 2018 at 4:34 PM Sailesh Mukil  wrote:
>
> > This is a vote to release Impala 3.0.0
> >
> > The artifacts for testing can be downloaded from:
> > https://dist.apache.org/repos/dist/dev/impala/3.0.0/RC1/
> > Git tag: 3.0.0-rc1
> > Tree hash: d5bf1fdeb20ac332ce48c6a2b0588bcaf7ac9cb0
> >
> > Please vote +1 or -1. -1 votes should be accompanied by an explanation of
> > the reason. Only PMC members have binding votes, but other
> > community members are encouraged to cast non-binding votes. This vote
> will
> > pass if there are 3 binding +1 votes and more binding +1 votes than -1
> > votes.
> >
> > Please also note that this release is a new major release, and does not
> > override the recently released 2.12.0 as the latest release. Rather,
> 2.12.0
> > is the latest release from the 2.x branch and 3.0.0 is the first and
> latest
> > release from the 3.x branch.
> >
> > This wiki page describes how to check the release before you vote:
> > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#
> >
> > The vote will be open until 04:30 PM PST on Friday, May 4
> >
>

[VOTE] 3.0.0 release candidate 1

[Sailesh Mukil]

This is a vote to release Impala 3.0.0

The artifacts for testing can be downloaded from:
https://dist.apache.org/repos/dist/dev/impala/3.0.0/RC1/
Git tag: 3.0.0-rc1
Tree hash: d5bf1fdeb20ac332ce48c6a2b0588bcaf7ac9cb0

Please vote +1 or -1. -1 votes should be accompanied by an explanation of
the reason. Only PMC members have binding votes, but other
community members are encouraged to cast non-binding votes. This vote will
pass if there are 3 binding +1 votes and more binding +1 votes than -1
votes.

Please also note that this release is a new major release, and does not
override the recently released 2.12.0 as the latest release. Rather, 2.12.0
is the latest release from the 2.x branch and 3.0.0 is the first and latest
release from the 3.x branch.

This wiki page describes how to check the release before you vote:
https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#

The vote will be open until 04:30 PM PST on Friday, May 4

Re: Re: Re: [DISCUSS] Apache Impala 2.12.0 and 3.0 releases

[Sailesh Mukil]

An update:

Impala 2.12.0 has been released.

All remaining tasks for Impala 3.0.0 have been completed. The release is
now in preparation and a vote will commence soon.

On Fri, Apr 13, 2018 at 11:51 AM, Sailesh Mukil 
wrote:

> Thank you all.
>
> I've triaged all the remaining open JIRAs for Impala 2.12.0 and Impala 3.0
> and we have 2 more breaking changes that need to go in for Impala 3.0:
> https://issues.apache.org/jira/browse/IMPALA-6340
> https://issues.apache.org/jira/browse/IMPALA-5893
>
> There are also a couple of patches that are very close to completion that
> will make it in:
> https://issues.apache.org/jira/browse/IMPALA-6790
> https://issues.apache.org/jira/browse/IMPALA-6793
>
> And lastly, there are a few docs JIRAs that are being worked on for Impala
> 3.0 and Impala 2.12.0.
>
> IMPALA-6340 is probably the most important change at this point, since
> DECIMAL_V2 is enabled by default in Impala 3.0 and this is a blocker bug
> that needs to be fixed to allow the feature to be used reliably.
>
> All the above JIRAs are being worked on as top priorities and we should
> have them completed soon. I will start the remaining formalities as soon as
> those changes are in.
>
> Full list of open 2.12.0 JIRAs:
> https://issues.apache.org/jira/browse/IMPALA-6793?jql=
> project%20%3D%20IMPALA%20AND%20status%20in%20(Open%2C%20%
> 22In%20Progress%22%2C%20Reopened)%20AND%20%22Target%20Version%22%20%3D%20%
> 22Impala%202.12.0%22
>
> Full list of open 3.0 JIRAs:
> https://issues.apache.org/jira/browse/IMPALA-6793?jql=
> project%20%3D%20IMPALA%20AND%20status%20in%20(Open%2C%20%
> 22In%20Progress%22%2C%20Reopened)%20AND%20%22Target%20Version%22%20%3D%20%
> 22Impala%203.0%22
>
> Thanks to all the authors and reviewers of the above JIRAs for working on
> them in a timely manner.
>
> - Sailesh
>
> On Tue, Apr 10, 2018 at 11:28 AM, Tim Armstrong 
> wrote:
>
>> We also just merged a small breaking change to pave the way for Jinchul
>> Kim's patch: https://gerrit.cloudera.org/#/c/9957/ (otherwise we might
>> have
>> to wait for the next major release to merge that).
>>
>> On Mon, Apr 2, 2018 at 8:33 PM, Quanlong Huang 
>> wrote:
>>
>> > Thanks, Tim and Jim! Then I have no other suggestions.+1
>> >
>> > At 2018-04-03 10:57:58, "Jim Apple"  wrote:
>> > >Additionally, because this patch doesn't break any backwards
>> > compatibility,
>> > >it's fine to merge it for 2.13 and 3.1 when those releases are cut. And
>> > >since there are no minimum waiting period between releases, we could
>> cut
>> > >those right after that patch lands if we like.
>> > >
>> > >On Mon, Apr 2, 2018 at 4:05 PM, Tim Armstrong > >
>> > >wrote:
>> > >
>> > >> I don't think we should hold the release for the ORC scanner patch -
>> > it's a
>> > >> great feature, but there are many other equally important features
>> and
>> > >> improvements that have been waiting for months to be released.
>> > >>
>> > >> Generally, there are always going to be big changes that are "nearly
>> > there"
>> > >> and I think we're best off releasing frequently rather than trying to
>> > hold
>> > >> releases for specific features. This is especially true for large
>> > features
>> > >> where it sometimes take a bit of bake time to flush out all of the
>> > issues
>> > >> with the initial patch.
>> > >>
>> > >> - Tim
>> > >>
>> > >> On Mon, Apr 2, 2018 at 3:49 PM, Quanlong Huang <
>> huang_quanl...@126.com>
>> > >> wrote:
>> > >>
>> > >> > Hi friends,
>> > >> >
>> > >> >
>> > >> > Is it possible to wait until IMPALA-5717 (
>> > https://gerrit.cloudera.org/#
>> > >> > /c/9134/) is merged? We're eager to upgrade our Impala cluster with
>> > the
>> > >> > latest ORC-support. I've heard from the community that many other
>> > >> companies
>> > >> > are waiting for it too. As it's an experimental feature so far, we
>> can
>> > >> get
>> > >> > more feedback on this release.
>> > >> >
>> > >> >
>> > >> > As Tim said, this patch is closed to be merged. Could you wait
>> until
>> > it's
>> > >> > merged?
>> > >> >
>> > >> >
>> > >> > Thanks,
>> > >> > Quanlong
>> > >> >
>> > >> > At 2018-04-03 05:56:11, "Jim Apple"  wrote:
>> > >> > >agreed.
>> > >> > >
>> > >> > >On Mon, Apr 2, 2018 at 2:50 PM, Philip Zeyliger <
>> phi...@cloudera.com
>> > >
>> > >> > wrote:
>> > >> > >
>> > >> > >> Sounds like a good plan to me!
>> > >> > >>
>> > >> > >> -- Philip
>> > >> > >>
>> > >> >
>> > >>
>> >
>>
>
>

Re: [ANNOUNCE] Apache Impala 2.12.0 release

[Sailesh Mukil]

My pleasure!

On Mon, Apr 30, 2018 at 11:19 AM, Jim Apple  wrote:

> -user, -announce
>
> Thanks for driving this, Sailesh!
>
> On Mon, Apr 30, 2018 at 10:03 AM, Sailesh Mukil 
> wrote:
>
> > The Apache Impala team is pleased to announce the release of Impala
> 2.12.0.
> >
> > Impala is a high-performance distributed SQL engine.
> >
> > The release is available at: https://impala.apache.org/downloads.html
> >
> > Thanks,
> >
> > The Apache Impala team
> >
> >
> >
>

[ANNOUNCE] Apache Impala 2.12.0 release

[Sailesh Mukil]

The Apache Impala team is pleased to announce the release of Impala 2.12.0.

Impala is a high-performance distributed SQL engine.

The release is available at: https://impala.apache.org/downloads.html

Thanks,

The Apache Impala team

Re: Re: [VOTE] 2.12.0 release candidate 1

[Sailesh Mukil]

Thanks for catching that, Quanlong. As Jim mentioned, this is a crash only
on a debug build, and shouldn't show up on a release build. (That DCHECK
might be incorrect)

I've already published the 2.12.0 artifacts to the release page on the main
Apache distribution repo, so we won't be able to include this fix as part
of this release. We will track it as a known issue for 2.12.0 debug builds
and treat it as a blocker for the immediate next release.

On Wed, Apr 25, 2018 at 9:13 AM, Jim Apple  wrote:

> This is a DCHECK_EQ, so it won't fire if you compile with -release.
>
> On Wed, Apr 25, 2018 at 6:49 AM, Quanlong Huang 
> wrote:
>
> > -1 (non-binding)
> >
> > Not mean to block you, but when I played around on this version I hit a
> > crash (IMPALA-6927).
> > I'm going to upgrade our production Impala cluster with this version. But
> > I have concerns if users can crash the impalad when they just clicking on
> > the webpage...
> >
> >
> > Sorry for my late vote.
> >
> >
> > At 2018-04-21 06:58:01, "Jim Apple"  wrote:
> > >+1: I trust the results of the Jenkins job.
> > >
> > >On Fri, Apr 20, 2018 at 3:45 PM, Sailesh Mukil 
> > wrote:
> > >
> > >> +1 (binding)
> > >>
> > >> - Validated the sha512 file
> > >> - Validated the GPG signature
> > >> - Ran the release test job:
> > >> https://jenkins.impala.io/view/Utility/job/release-test-ub1604/10/
> > >>
> > >> Also, thanks Lars for bringing up IMPALA-4744. I'm not fixing it for
> > this
> > >> release, since a fix for that would require a new RC, which would
> > restart
> > >> the entire voting process. Since we know that it's not a regression
> and
> > >> only a minor cosmetic drawback. I've updated the release documentation
> > to
> > >> ensure that we have it fixed for every upcoming release henceforth.
> > >>
> > >> On Fri, Apr 20, 2018 at 1:55 PM, Thomas Tauber-Marshall <
> > >> tmarsh...@cloudera.com> wrote:
> > >>
> > >> > +1 (binding)
> > >> >
> > >> > Ran the release test job:
> > >> > https://jenkins.impala.io/job/release-test-ub1604/9/
> > >> >
> > >> >
> > >> > On Fri, Apr 20, 2018 at 11:58 AM Lars Volker 
> wrote:
> > >> >
> > >> > > +1 (binding)
> > >> > >
> > >> > > * Validated that the sha512 file has been fixed
> > >> > > * Validated the GPG signature
> > >> > > * Ran the release test job on jenkins.impala.io, result is here
> > >> > > <https://jenkins.impala.io/view/Utility/job/release-test-
> ub1604/8/>
> > >> > > * Unpacked and compiled the tarball on my development machine,
> made
> > >> sure
> > >> > > the daemon starts, and ran some queries.
> > >> > >
> > >> > > I noticed that the build identifies itself as "impalad version
> > >> > > 2.12.0-RELEASE DEBUG (build Could not obtain git hash)".
> > >> > > This is tracked in IMPALA-4744
> > >> > > <https://issues.apache.org/jira/browse/IMPALA-4744> and we're
> > released
> > >> > > with
> > >> > > this before. Should we require a subsequent RC, we can manually
> > change
> > >> > > GIT_HASH in bin/save-version.sh.
> > >> > >
> > >> > > Thanks for creating the RC, Sailesh!
> > >> > >
> > >> > >
> > >> > > On Thu, Apr 19, 2018 at 4:57 PM, Sailesh Mukil <
> > sail...@cloudera.com>
> > >> > > wrote:
> > >> > >
> > >> > > > I've already updated that file to not have an absolute path.
> Could
> > >> you
> > >> > > > download that and try again?
> > >> > > >
> > >> > > > On Thu, Apr 19, 2018, 4:15 PM Jim Apple 
> > >> wrote:
> > >> > > >
> > >> > > > > -1:
> > >> > > > >
> > >> > > > > /tmp/foo:$ sha512sum --check apache-impala-2.12.0.tar.gz.
> sha512
> > >> > > > > sha512sum: /tmp/apache-impala-2.12.0.tar.gz: No such file or
> > >> > directory
> > >> > > > > /tmp/apache-impala-2.12.0.tar.gz: FAILED open or read
> > >> > 

[RESULT] Vote on Impala 2.12.0 RC 1

[Sailesh Mukil]

https://lists.apache.org/thread.html/bdbb9d74834a5db53ee6565958044f3af1d4160f45cf0024393a4fe2@%3Cdev.impala.apache.org%3E

The vote has passed with the following tally:

+1 (binding):
* Lars Volker
* Thomas Tauber-Marshall
* Jim Apple
* Sailesh Mukil

-1 (binding) - None
(There was one which was changed to a +1 in the same vote)

0 - None

+1 (non-binding):
* 张启凡

Thank you everyone for testing and participating in the vote.

- Sailesh

Re: [VOTE] 2.12.0 release candidate 1

[Sailesh Mukil]

+1 (binding)

- Validated the sha512 file
- Validated the GPG signature
- Ran the release test job:
https://jenkins.impala.io/view/Utility/job/release-test-ub1604/10/

Also, thanks Lars for bringing up IMPALA-4744. I'm not fixing it for this
release, since a fix for that would require a new RC, which would restart
the entire voting process. Since we know that it's not a regression and
only a minor cosmetic drawback. I've updated the release documentation to
ensure that we have it fixed for every upcoming release henceforth.

On Fri, Apr 20, 2018 at 1:55 PM, Thomas Tauber-Marshall <
tmarsh...@cloudera.com> wrote:

> +1 (binding)
>
> Ran the release test job:
> https://jenkins.impala.io/job/release-test-ub1604/9/
>
>
> On Fri, Apr 20, 2018 at 11:58 AM Lars Volker  wrote:
>
> > +1 (binding)
> >
> > * Validated that the sha512 file has been fixed
> > * Validated the GPG signature
> > * Ran the release test job on jenkins.impala.io, result is here
> > <https://jenkins.impala.io/view/Utility/job/release-test-ub1604/8/>
> > * Unpacked and compiled the tarball on my development machine, made sure
> > the daemon starts, and ran some queries.
> >
> > I noticed that the build identifies itself as "impalad version
> > 2.12.0-RELEASE DEBUG (build Could not obtain git hash)".
> > This is tracked in IMPALA-4744
> > <https://issues.apache.org/jira/browse/IMPALA-4744> and we're released
> > with
> > this before. Should we require a subsequent RC, we can manually change
> > GIT_HASH in bin/save-version.sh.
> >
> > Thanks for creating the RC, Sailesh!
> >
> >
> > On Thu, Apr 19, 2018 at 4:57 PM, Sailesh Mukil 
> > wrote:
> >
> > > I've already updated that file to not have an absolute path. Could you
> > > download that and try again?
> > >
> > > On Thu, Apr 19, 2018, 4:15 PM Jim Apple  wrote:
> > >
> > > > -1:
> > > >
> > > > /tmp/foo:$ sha512sum --check apache-impala-2.12.0.tar.gz.sha512
> > > > sha512sum: /tmp/apache-impala-2.12.0.tar.gz: No such file or
> directory
> > > > /tmp/apache-impala-2.12.0.tar.gz: FAILED open or read
> > > > sha512sum: WARNING: 1 listed file could not be read
> > > >
> > > >
> > > > On Thu, Apr 19, 2018 at 2:40 PM, Sailesh Mukil  >
> > > > wrote:
> > > >
> > > > > This is a vote to release Impala 2.12.0
> > > > >
> > > > > The artifacts for testing can be downloaded from:
> > > > > https://dist.apache.org/repos/dist/dev/impala/2.12.0/RC1/
> > > > > Git tag: 2.12.0-rc1
> > > > > Tree hash: 07f18b8539a044c506bb3ff5a118f3b7b5ea1cf7
> > > > >
> > > > > Please vote +1 or -1. -1 votes should be accompanied by an
> > explanation
> > > of
> > > > > the reason. Only PMC members have binding votes, but other
> > > > > community members are encouraged to cast non-binding votes. This
> vote
> > > > will
> > > > > pass if there are 3 binding +1 votes and more binding +1 votes than
> > -1
> > > > > votes.
> > > > >
> > > > > This wiki page describes how to check the release before you vote:
> > > > > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#
> > > > >
> > > > > The vote will be open until 3:00PM PST on Tuesday, April 24
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] 2.12.0 release candidate 1

[Sailesh Mukil]

I've already updated that file to not have an absolute path. Could you
download that and try again?

On Thu, Apr 19, 2018, 4:15 PM Jim Apple  wrote:

> -1:
>
> /tmp/foo:$ sha512sum --check apache-impala-2.12.0.tar.gz.sha512
> sha512sum: /tmp/apache-impala-2.12.0.tar.gz: No such file or directory
> /tmp/apache-impala-2.12.0.tar.gz: FAILED open or read
> sha512sum: WARNING: 1 listed file could not be read
>
>
> On Thu, Apr 19, 2018 at 2:40 PM, Sailesh Mukil 
> wrote:
>
> > This is a vote to release Impala 2.12.0
> >
> > The artifacts for testing can be downloaded from:
> > https://dist.apache.org/repos/dist/dev/impala/2.12.0/RC1/
> > Git tag: 2.12.0-rc1
> > Tree hash: 07f18b8539a044c506bb3ff5a118f3b7b5ea1cf7
> >
> > Please vote +1 or -1. -1 votes should be accompanied by an explanation of
> > the reason. Only PMC members have binding votes, but other
> > community members are encouraged to cast non-binding votes. This vote
> will
> > pass if there are 3 binding +1 votes and more binding +1 votes than -1
> > votes.
> >
> > This wiki page describes how to check the release before you vote:
> > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#
> >
> > The vote will be open until 3:00PM PST on Tuesday, April 24
> >
>

[VOTE] 2.12.0 release candidate 1

[Sailesh Mukil]

This is a vote to release Impala 2.12.0

The artifacts for testing can be downloaded from:
https://dist.apache.org/repos/dist/dev/impala/2.12.0/RC1/
Git tag: 2.12.0-rc1
Tree hash: 07f18b8539a044c506bb3ff5a118f3b7b5ea1cf7

Please vote +1 or -1. -1 votes should be accompanied by an explanation of
the reason. Only PMC members have binding votes, but other
community members are encouraged to cast non-binding votes. This vote will
pass if there are 3 binding +1 votes and more binding +1 votes than -1
votes.

This wiki page describes how to check the release before you vote:
https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#

The vote will be open until 3:00PM PST on Tuesday, April 24

Re: JIRA necromancy

[Sailesh Mukil]

Yes to this.

On Tue, Apr 17, 2018 at 11:26 AM, Alexander Behm 
wrote:

> I agree. Reopening can be very confusing.
>
> On Tue, Apr 17, 2018 at 11:20 AM, Jim Apple  wrote:
>
> > I'm convinced.
> >
> > On Tue, Apr 17, 2018 at 10:29 AM, Tim Armstrong  >
> > wrote:
> > > I noticed that there's been a trend recently towards reopening old
> issues
> > > instead of filing new issues. Not trying to pick on anyone but it seems
> > > like its worth having a discussion about best practices.
> > >
> > > Personally I think reopening JIRAs is often a bad thing for a several
> > > reasons:
> > >
> > > * We don't tend to properly triage the issue to determine if it is
> > actually
> > > has same root cause as the old one. E.g. the same test fails for two
> > > completely different reasons.
> > > * People are tempted to skimp on including diagnostic information.
> > > * It gets confusing trying to figure out which version the issue was
> > fixed
> > > in, particularly if the new thing turns out to be a separate issue.
> > > * The target version, fix version, priority, etc is wrong
> > > * It automatically ends up on the plate of whoever last fixed it,
> rather
> > > than whoever currently has bandwidth. This is particularly bad for
> anyone
> > > who has fixed or tried to fix a lot of flaky tests over the last year
> or
> > > two (e.g. me).
> > >
> > > I'd prefer if we opened new issues by default unless we're really
> > confident
> > > that it's the same issue. It's much easier to mark issues as duplicates
> > > than it is to separate out two distinct issues tracked by one JIRA.
> Even
> > if
> > > we're pretty sure it's the same thing, I think we should think
> carefully
> > > before re-opening issues from previous releases.
> > >
> > > Anyway, this is just my opinion. Do others agree or disagree?
> > >
> > > - Tim
> >
>

Re: Impala FE compilation error (related to sentry)

[Sailesh Mukil]

Great, thanks!

On Mon, Apr 16, 2018 at 4:00 PM, Fredy Wijaya  wrote:

> I filed a ticket here: https://issues.apache.org/jira/browse/IMPALA-6862.
> I'll work on a fix.
>
> *Fredy Wijaya* | Software Engineer
> e. fwij...@cloudera.com
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------
>
> On Mon, Apr 16, 2018 at 5:59 PM, Sailesh Mukil 
> wrote:
>
> > Thanks Fredy.
> >
> > Is there a fix out already?
> >
> > On Mon, Apr 16, 2018 at 3:53 PM, Fredy Wijaya 
> > wrote:
> >
> > > In Sentry 1.5.1, BitFieldAction is an abstract class:
> > > https://github.com/apache/sentry/blob/release-1.5.1/
> > > sentry-core/sentry-core-common/src/main/java/org/
> > > apache/sentry/core/common/BitFieldAction.java#L22.
> > > Sentry removed the abstract requirement in Sentry 2.0.0:
> > > https://github.com/apache/sentry/blob/master/sentry-
> > > core/sentry-core-common/src/main/java/org/apache/sentry/
> > > core/common/BitFieldAction.java#L23
> > >
> > > To support both versions of Sentry, we need to fix
> > > https://github.com/apache/impala/blob/master/fe/src/
> > > main/java/org/apache/impala/authorization/Privilege.java#L72
> > > .
> > >
> > > *Fredy Wijaya* | Software Engineer
> > > e. fwij...@cloudera.com
> > > cloudera.com <https://www.cloudera.com>
> > >
> > > [image: Cloudera] <https://www.cloudera.com/>
> > >
> > > [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> > > Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
> > Cloudera
> > > on LinkedIn] <https://www.linkedin.com/company/cloudera>
> > > --
> > >
> > > On Mon, Apr 16, 2018 at 5:10 PM, Sailesh Mukil 
> > > wrote:
> > >
> > > > I get this error when trying to the build the Impala frontend. Anyone
> > > know
> > > > how I can get past this?
> > > >
> > > > [ERROR] COMPILATION ERROR :
> > > >
> > > > [ERROR]
> > > > /home/systest/incubator-impala/fe/src/main/java/org/
> > > > apache/impala/authorization/Privilege.java:[72,25]
> > > > org.apache.sentry.core.common.BitFieldAction is abstract; cannot be
> > > > instantiated
> > > >
> > > > [INFO] BUILD FAILURE
> > > >
> > > > [ERROR] Failed to execute goal
> > > > org.apache.maven.plugins:maven-compiler-plugin:3.3:compile
> > > > (default-compile) on project impala-frontend: Compilation failure
> > > >
> > > > [ERROR]
> > > > /home/systest/incubator-impala/fe/src/main/java/org/
> > > > apache/impala/authorization/Privilege.java:[72,25]
> > > > org.apache.sentry.core.common.BitFieldAction is abstract; cannot be
> > > > instantiated
> > > >
> > > > [ERROR] -> [Help 1]
> > > >
> > > > [ERROR]
> > > >
> > > > [ERROR] To see the full stack trace of the errors, re-run Maven with
> > the
> > > -e
> > > > switch.
> > > >
> > > > [ERROR] Re-run Maven using the -X switch to enable full debug
> logging.
> > > >
> > > > [ERROR]
> > > >
> > > > [ERROR] For more information about the errors and possible solutions,
> > > > please read the following articles:
> > > >
> > > > [ERROR] [Help 1]
> > > > http://cwiki.apache.org/confluence/display/MAVEN/
> MojoFailureException
> > > >
> > >
> >
>

Re: Impala FE compilation error (related to sentry)

[Sailesh Mukil]

Thanks Fredy.

Is there a fix out already?

On Mon, Apr 16, 2018 at 3:53 PM, Fredy Wijaya  wrote:

> In Sentry 1.5.1, BitFieldAction is an abstract class:
> https://github.com/apache/sentry/blob/release-1.5.1/
> sentry-core/sentry-core-common/src/main/java/org/
> apache/sentry/core/common/BitFieldAction.java#L22.
> Sentry removed the abstract requirement in Sentry 2.0.0:
> https://github.com/apache/sentry/blob/master/sentry-
> core/sentry-core-common/src/main/java/org/apache/sentry/
> core/common/BitFieldAction.java#L23
>
> To support both versions of Sentry, we need to fix
> https://github.com/apache/impala/blob/master/fe/src/
> main/java/org/apache/impala/authorization/Privilege.java#L72
> .
>
> *Fredy Wijaya* | Software Engineer
> e. fwij...@cloudera.com
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> --
>
> On Mon, Apr 16, 2018 at 5:10 PM, Sailesh Mukil 
> wrote:
>
> > I get this error when trying to the build the Impala frontend. Anyone
> know
> > how I can get past this?
> >
> > [ERROR] COMPILATION ERROR :
> >
> > [ERROR]
> > /home/systest/incubator-impala/fe/src/main/java/org/
> > apache/impala/authorization/Privilege.java:[72,25]
> > org.apache.sentry.core.common.BitFieldAction is abstract; cannot be
> > instantiated
> >
> > [INFO] BUILD FAILURE
> >
> > [ERROR] Failed to execute goal
> > org.apache.maven.plugins:maven-compiler-plugin:3.3:compile
> > (default-compile) on project impala-frontend: Compilation failure
> >
> > [ERROR]
> > /home/systest/incubator-impala/fe/src/main/java/org/
> > apache/impala/authorization/Privilege.java:[72,25]
> > org.apache.sentry.core.common.BitFieldAction is abstract; cannot be
> > instantiated
> >
> > [ERROR] -> [Help 1]
> >
> > [ERROR]
> >
> > [ERROR] To see the full stack trace of the errors, re-run Maven with the
> -e
> > switch.
> >
> > [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> >
> > [ERROR]
> >
> > [ERROR] For more information about the errors and possible solutions,
> > please read the following articles:
> >
> > [ERROR] [Help 1]
> > http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> >
>

Impala FE compilation error (related to sentry)

[Sailesh Mukil]

I get this error when trying to the build the Impala frontend. Anyone know
how I can get past this?

[ERROR] COMPILATION ERROR :

[ERROR]
/home/systest/incubator-impala/fe/src/main/java/org/apache/impala/authorization/Privilege.java:[72,25]
org.apache.sentry.core.common.BitFieldAction is abstract; cannot be
instantiated

[INFO] BUILD FAILURE

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.3:compile
(default-compile) on project impala-frontend: Compilation failure

[ERROR]
/home/systest/incubator-impala/fe/src/main/java/org/apache/impala/authorization/Privilege.java:[72,25]
org.apache.sentry.core.common.BitFieldAction is abstract; cannot be
instantiated

[ERROR] -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions,
please read the following articles:

[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Re: Re: Re: [DISCUSS] Apache Impala 2.12.0 and 3.0 releases

[Sailesh Mukil]

Thank you all.

I've triaged all the remaining open JIRAs for Impala 2.12.0 and Impala 3.0
and we have 2 more breaking changes that need to go in for Impala 3.0:
https://issues.apache.org/jira/browse/IMPALA-6340
https://issues.apache.org/jira/browse/IMPALA-5893

There are also a couple of patches that are very close to completion that
will make it in:
https://issues.apache.org/jira/browse/IMPALA-6790
https://issues.apache.org/jira/browse/IMPALA-6793

And lastly, there are a few docs JIRAs that are being worked on for Impala
3.0 and Impala 2.12.0.

IMPALA-6340 is probably the most important change at this point, since
DECIMAL_V2 is enabled by default in Impala 3.0 and this is a blocker bug
that needs to be fixed to allow the feature to be used reliably.

All the above JIRAs are being worked on as top priorities and we should
have them completed soon. I will start the remaining formalities as soon as
those changes are in.

Full list of open 2.12.0 JIRAs:
https://issues.apache.org/jira/browse/IMPALA-6793?jql=project%20%3D%20IMPALA%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20%22Target%20Version%22%20%3D%20%22Impala%202.12.0%22

Full list of open 3.0 JIRAs:
https://issues.apache.org/jira/browse/IMPALA-6793?jql=project%20%3D%20IMPALA%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20%22Target%20Version%22%20%3D%20%22Impala%203.0%22

Thanks to all the authors and reviewers of the above JIRAs for working on
them in a timely manner.

- Sailesh

On Tue, Apr 10, 2018 at 11:28 AM, Tim Armstrong 
wrote:

> We also just merged a small breaking change to pave the way for Jinchul
> Kim's patch: https://gerrit.cloudera.org/#/c/9957/ (otherwise we might
> have
> to wait for the next major release to merge that).
>
> On Mon, Apr 2, 2018 at 8:33 PM, Quanlong Huang 
> wrote:
>
> > Thanks, Tim and Jim! Then I have no other suggestions.+1
> >
> > At 2018-04-03 10:57:58, "Jim Apple"  wrote:
> > >Additionally, because this patch doesn't break any backwards
> > compatibility,
> > >it's fine to merge it for 2.13 and 3.1 when those releases are cut. And
> > >since there are no minimum waiting period between releases, we could cut
> > >those right after that patch lands if we like.
> > >
> > >On Mon, Apr 2, 2018 at 4:05 PM, Tim Armstrong 
> > >wrote:
> > >
> > >> I don't think we should hold the release for the ORC scanner patch -
> > it's a
> > >> great feature, but there are many other equally important features and
> > >> improvements that have been waiting for months to be released.
> > >>
> > >> Generally, there are always going to be big changes that are "nearly
> > there"
> > >> and I think we're best off releasing frequently rather than trying to
> > hold
> > >> releases for specific features. This is especially true for large
> > features
> > >> where it sometimes take a bit of bake time to flush out all of the
> > issues
> > >> with the initial patch.
> > >>
> > >> - Tim
> > >>
> > >> On Mon, Apr 2, 2018 at 3:49 PM, Quanlong Huang <
> huang_quanl...@126.com>
> > >> wrote:
> > >>
> > >> > Hi friends,
> > >> >
> > >> >
> > >> > Is it possible to wait until IMPALA-5717 (
> > https://gerrit.cloudera.org/#
> > >> > /c/9134/) is merged? We're eager to upgrade our Impala cluster with
> > the
> > >> > latest ORC-support. I've heard from the community that many other
> > >> companies
> > >> > are waiting for it too. As it's an experimental feature so far, we
> can
> > >> get
> > >> > more feedback on this release.
> > >> >
> > >> >
> > >> > As Tim said, this patch is closed to be merged. Could you wait until
> > it's
> > >> > merged?
> > >> >
> > >> >
> > >> > Thanks,
> > >> > Quanlong
> > >> >
> > >> > At 2018-04-03 05:56:11, "Jim Apple"  wrote:
> > >> > >agreed.
> > >> > >
> > >> > >On Mon, Apr 2, 2018 at 2:50 PM, Philip Zeyliger <
> phi...@cloudera.com
> > >
> > >> > wrote:
> > >> > >
> > >> > >> Sounds like a good plan to me!
> > >> > >>
> > >> > >> -- Philip
> > >> > >>
> > >> >
> > >>
> >
>

Re: Should we allow attachments?

[Sailesh Mukil]

Do a lot of dev lists allow for attachments? In my experience, I've not
found that to be the case.

On Sat, Apr 7, 2018 at 8:00 PM, Philip Zeyliger  wrote:

> I'm in favor of attachments. We can always change our minds if it turns out
> to be unpleasant.
>
> P
>
> On Sat, Apr 7, 2018 at 6:41 PM Jim Apple  wrote:
>
> > Attachments sometimes (always?) don't make it to dev@. Maybe we could
> fix
> > that by filing a ticket like this one:
> >
> > https://issues.apache.org/jira/browse/INFRA-7933
> >
> > But it could always be abused, I suspect. That said, I think it would be
> > worth the risk.
> >
> > Thoughts?
> >
>

Re: Error when build with bootstrap_development.sh

[Sailesh Mukil]

Have you tried deleting the gen-cpp directory and rebuilding with
"./buildall" ?

On Wed, Apr 4, 2018 at 11:33 AM, Yang Su  wrote:

> Dear all,
>
> I try to build impala on a Ubuntu box: 16.04.1-Ubuntu x86_64.
>
> I ran bin/bootstrap_development.sh. It failed with the following error
> message:
>
> [ 17%] Building CXX object be/src/kudu/rpc/CMakeFiles/
> protoc-gen-krpc.dir/protoc-gen-krpc.cc.o
> [ 17%] Building CXX object be/src/kudu/rpc/CMakeFiles/
> protoc-gen-krpc.dir/__/__/common/kudu_version.cc.o
> In file included from /home/ysu/Impala/be/src/common/kudu_version.cc:20:0:
> /home/ysu/Impala/be/src/common/status.h:28:50: fatal error:
> gen-cpp/Status_types.h: No such file or directory
>  #include "gen-cpp/Status_types.h"  // for TStatus
>   ^
> compilation terminated.
> be/src/kudu/rpc/CMakeFiles/protoc-gen-krpc.dir/build.make:86: recipe for
> target 
> 'be/src/kudu/rpc/CMakeFiles/protoc-gen-krpc.dir/__/__/common/kudu_version.cc.o'
> failed
> make[2]: *** 
> [be/src/kudu/rpc/CMakeFiles/protoc-gen-krpc.dir/__/__/common/kudu_version.cc.o]
> Error 1
> CMakeFiles/Makefile2:3261: recipe for target 
> 'be/src/kudu/rpc/CMakeFiles/protoc-gen-krpc.dir/all'
> failed
> make[1]: *** [be/src/kudu/rpc/CMakeFiles/protoc-gen-krpc.dir/all] Error 2
> Makefile:94: recipe for target 'all' failed
> make: *** [all] Error 2
> Error in /home/ysu/Impala/bin/make_impala.sh at line 178: ${MAKE_CMD}
> ${MAKE_ARGS}
>
>
> It seems that gen-cpp directory is not properly populated. Has anyone
> encountered the same problem and how did you solve it?
>
> Thank you and best regards,
> Yang
>

[DISCUSS] Apache Impala 2.12.0 and 3.0 releases

[Sailesh Mukil]

Folks,

It's been about 3 months since our last release and there have been a good
number of bug fixes, features added and performance improvements since then.

I propose that we have a 2.12.0 release soon and a 3.0 release in parallel.
I, as a PMC of Apache Impala volunteer to mange both these releases.

Why have 2 releases in parallel?

We've been maintaining 2 branches in parallel for a while now. A 2.x branch
which works with Hadoop 2.0 and friends, and a master branch, which now
works with Hadoop 3.0 and friends.
We've also made sure to not push any incompatible changes into our 2.x
branch, but only to the master branch, and we have a good number of these
incompatible changes on master now that justifies having a 3.0 release.

There have been 351 commits on master since the last release, Apache Impala
2.11.0. I've gone through them and outlined the commits that are either
considered breaking changes or support a breaking change:

IMPALA-4277: Switch to using Hadoop 3 and friends by default.

IMPALA-6510: [DOCS] Remove refresh_after_connect

IMPALA-6647: Add CREATE fine-grained privilege

IMPALA-6230, IMPALA-6468: Fix the output type of round() and related fns

IMPALA-4277: Support multiple versions of Hadoop ecosystem

IMPALA-6415: [DOCS] fix invalid 

IMPALA-6643: Add REFRESH fine-grained privilege

IMPALA-5191, IMPALA-6415: [DOCS] Document breaking change of alias and
ordinal substitution

IMPALA-3916: [DOCS] Reserved keywords updated for Impala 3.0

IMPALA-6240: [DOCS] Document PARQUET_ARRAY_RESOLUTION query option

[DOCS] Removed the obsolete Llama options files

IMPALA-6077: remove Parquet BIT_PACKED def level support

IMPALA-3271: organise and warn on removed startup flags

IMPALA-5037: Default PARQUET_ARRAY_RESOLUTION=THREE_LEVEL

IMPALA-5293: Turn insert clustering on by default

IMPALA-4924 addendum: Change result type to decimal in a TPCH query

IMPALA-3916: Reserve SQL:2016 reserved words   (Disabled on 2.x with
IMPALA-3916: Change reserved_words_version to 2.11.0)

IMPALA-4319: remove some deprecated query options

IMPALA-4924: Enable Decimal V2 by default

IMPALA-5191: Standardize column alias behavior

IMPALA-3998: Remove refresh_after_connect option from shell

The 3.0 release would come off the master branch and the 2.12.0 release
would come off the 2.x branch.

Please speak up and let the community know if anyone has any objections to
this.

P.S: This is not a VOTE thread but only a DISCUSS thread.

Thanks,
Sailesh

Re: Help with fixing clang-diagnostic-over-aligned

[Sailesh Mukil]

Thanks for the explanation Todd and Dan! I had introduced a new
CACHELINE_ALIGNED object into the class which caused this problem. Making
the parent classes CACHELINE_ALIGNED too fixed the problem.

On Wed, Feb 14, 2018 at 12:58 PM, Todd Lipcon  wrote:

> This probably means that QueryExecMgr (or one of its descendent members) is
> annotated with __attribute__((aligned(64)), perhaps through something like
> a CACHELINE_ALIGNED macro.
>
> The issue is that, as the warning says, normal libc malloc doesn't make any
> hard guarantees about the alignment of allocations, and the way that
> 'operator new' is implemented by default only gets the size and not the
> desired allocation. I believe C++17 is adding support for an 'operator new'
> overload that also passes the desired alignment, so that it can call down
> into posix_memalign and ensure the appropriate allocation.
>
> The "correct" fix prior to C++17 is probably to do something like overload
> 'operator new' for the type, or to use posix_memalign manually to allocate
> aligned memory, and then placement-new it into that memory.
>
> Another reasonable fix would probably be to add a CHECK in the constructor
> that 'this' is aligned as desired so that, if you ever run on an allocator
> that isn't guaranteeing what you think it is, you'll know immediately
> rather than getting a surprising perf or atomicity regression due to
> cross-cacheline loads/stores.
>
> -Todd
>
>
> On Wed, Feb 14, 2018 at 12:33 PM, Sailesh Mukil 
> wrote:
>
> > Does anyone have experience with fixing the following type of
> > clang-tidy warning?
> >
> >
> > /home/ubuntu/Impala/be/src/runtime/exec-env.cc:159:21: warning: type
> > 'impala::QueryExecMgr' requires 64 bytes of alignment and the default
> > allocator only guarantees 16 bytes [clang-diagnostic-over-aligned]
> > query_exec_mgr_(new QueryExecMgr()),
> >
> >
> > /home/ubuntu/Impala/be/src/service/impalad-main.cc:80:49: warning:
> > type 'impala::ImpalaServer' requires 64 bytes of alignment and the
> > default allocator only guarantees 16 bytes
> > [clang-diagnostic-over-aligned]
> >   boost::shared_ptr impala_server(new
> > ImpalaServer(&exec_env));
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Help with fixing clang-diagnostic-over-aligned

[Sailesh Mukil]

Does anyone have experience with fixing the following type of
clang-tidy warning?


/home/ubuntu/Impala/be/src/runtime/exec-env.cc:159:21: warning: type
'impala::QueryExecMgr' requires 64 bytes of alignment and the default
allocator only guarantees 16 bytes [clang-diagnostic-over-aligned]
query_exec_mgr_(new QueryExecMgr()),


/home/ubuntu/Impala/be/src/service/impalad-main.cc:80:49: warning:
type 'impala::ImpalaServer' requires 64 bytes of alignment and the
default allocator only guarantees 16 bytes
[clang-diagnostic-over-aligned]
  boost::shared_ptr impala_server(new ImpalaServer(&exec_env));

Re: Setting up ASF Git Bot?

[Sailesh Mukil]

I like the idea. Easier to access commits from the JIRA.

On Wed, Jan 31, 2018 at 1:36 PM, Philip Zeyliger 
wrote:

> Hi folks,
>
> Would people be interested in ASF's bot that connects JIRAs to Commits? You
> can see the "activity stream" of
> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=jira-bot for
> an
> example, and there's a sample below for convenience.
>
> Hopefully it will make it more obvious that we should resolve JIRAs once
> they're done, and make it a little easier for folks to follow the bread
> crumbs. I know many of us already cut/paste the commit link into JIRA.
>
> I've not looked into the mechanics, but I imagine it's an ASF INFRA ticket
> away.
>
> Thanks!
>
> -- Philip
>
> *Sample:*
>
> Commit 17a35f464fc283cfe970536011b1d821c4de031e in couchdb's branch
> refs/heads/COUCHDB-3287
> -
> pluggable-storage-engines
> from Paul Joseph Davis
>  name=paul.joseph.davis>
> [ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=17a35f4 ]
>
> Add storage engine test suite
>
> This allows other storage engine implementations to reuse the same exact
> test suite without having to resort to shenanigans like keeping vendored
> copies up to date.
>
> COUCHDB-3287 
>

Re: thrift-server-test

[Sailesh Mukil]

Your principal isn't getting cached for some reason. The same on my machine
looks like:

$ klist /tmp/krb5cc_impala_internal

Ticket cache: FILE:/tmp/krb5cc_impala_internal

Default principal: impala/localh...@krbtest.com


Valid starting   Expires  Service principal

01/08/2018 15:49:30  01/09/2018 15:49:30  krbtgt/krbtest@krbtest.com

renew until 01/15/2018 15:49:30

01/08/2018 15:49:30  01/09/2018 15:49:30  impala/localhost@

renew until 01/15/2018 15:49:30

01/08/2018 15:49:30  01/09/2018 15:49:30  impala/localh...@krbtest.com

renew until 01/15/2018 15:49:30

Did you check if your user has appropriate permissions on the file? Looks
like only the MiniKDC was able to write to it, and maybe your 'kinit'
couldn't and silently failed (which should be a bug if it didn't throw an
error) ?

On Mon, Jan 8, 2018 at 3:40 PM, Philip Zeyliger  wrote:

> Hi Sailiesh,
>
> Is this what you'd expect?
>
> $klist /tmp/krb5cc_impala_internal
> Ticket cache: FILE:/tmp/krb5cc_impala_internal
> Default principal: impala/localh...@krbtest.com
>
> Valid starting   Expires  Service principal
> 01/08/2018 15:39:23  01/09/2018 15:39:23  krbtgt/krbtest@krbtest.com
> renew until 01/15/2018 15:39:23
>
> Thanks!
>
>
> On Mon, Jan 8, 2018 at 12:20 PM, Sailesh Mukil 
> wrote:
>
> > Can you run the test again, and klist the contents of the credential
> cache
> > and post the error logs again? Looks like "impala/localhost" might not be
> > stored as expected in the cache on your machine.
> >
> > On Wed, Dec 13, 2017 at 2:47 PM, Philip Zeyliger 
> > wrote:
> >
> > > The KDC in this case is the "minikdc" from
> > > https://github.com/apache/impala/blob/master/be/src/
> > > kudu/security/test/mini_kdc.cc.
> > > I see evidence of it, and have been able to look at its configuration
> by,
> > > um, adding --gtest_break_on_failure. (The feature actually doesn't
> work,
> > > presumably because of an interaction with breakpad, but a temporary
> > > directory is left on my filesystem, so that's nice.)
> > >
> > > -- Philip
> > >
> > > On Tue, Dec 12, 2017 at 4:08 PM, Evo Eftimov 
> > > wrote:
> > >
> > > > Is your cluster Kerberized at all, especially the Impala daemon - it
> > > > doesn’t seem to be enrolled in the KDC at all
> > > >
> > > > You / your personal account/principal is definitely enrolled though
> > > >
> > > > And there is definetly a KDC in your environment
> > > >
> > > > -Original Message-
> > > > From: Philip Zeyliger [mailto:phi...@cloudera.com]
> > > > Sent: Tuesday, December 12, 2017 11:26 PM
> > > > To: dev@impala.apache.org
> > > > Subject: thrift-server-test
> > > >
> > > > Hi folks,
> > > >
> > > > I've been running into issues with thrift-server-test and Kerberos.
> > Below
> > > > is an excerpt of "KRB5_TRACE=/dev/stderr be/build/debug/rpc/thrift-
> > > server-test";
> > > > both SslConnectivity/1 and
> > > > SslConnectivity/2 fail the same way.
> > > >
> > > > I'm running Ubuntu16.04. I've seen this both on my host, as well as
> > > inside
> > > > of an Ubuntu 16.04 Docker container.
> > > >
> > > > Does this ring any bells?
> > > >
> > > > Thanks!
> > > >
> > > > -- Philip
> > > >
> > > >
> > > > [ RUN  ] KerberosOnAndOff/ThriftKerberizedParamsTest.
> > > SslConnectivity/2
> > > > Loading random data
> > > > Initializing database '7abf-cef9-113e-eae3/krb5kdc/principal' for
> > realm
> > > '
> > > > KRBTEST.COM',
> > > > master key name 'K/m...@krbtest.com'
> > > > [31585] 1513120922.459517: Retrieving K/m...@krbtest.com from
> > > > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0)
> > with
> > > > result: 0/Success
> > > > [31586] 1513120922.472314: Retrieving K/m...@krbtest.com from
> > > > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0)
> > with
> > > > result: 0/Success
> > > > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info):
> > > setting
> > > > up network...
> > > > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info):
> > > > listening on fd 1

Re: thrift-server-test

[Sailesh Mukil]

Can you run the test again, and klist the contents of the credential cache
and post the error logs again? Looks like "impala/localhost" might not be
stored as expected in the cache on your machine.

On Wed, Dec 13, 2017 at 2:47 PM, Philip Zeyliger 
wrote:

> The KDC in this case is the "minikdc" from
> https://github.com/apache/impala/blob/master/be/src/
> kudu/security/test/mini_kdc.cc.
> I see evidence of it, and have been able to look at its configuration by,
> um, adding --gtest_break_on_failure. (The feature actually doesn't work,
> presumably because of an interaction with breakpad, but a temporary
> directory is left on my filesystem, so that's nice.)
>
> -- Philip
>
> On Tue, Dec 12, 2017 at 4:08 PM, Evo Eftimov 
> wrote:
>
> > Is your cluster Kerberized at all, especially the Impala daemon - it
> > doesn’t seem to be enrolled in the KDC at all
> >
> > You / your personal account/principal is definitely enrolled though
> >
> > And there is definetly a KDC in your environment
> >
> > -Original Message-
> > From: Philip Zeyliger [mailto:phi...@cloudera.com]
> > Sent: Tuesday, December 12, 2017 11:26 PM
> > To: dev@impala.apache.org
> > Subject: thrift-server-test
> >
> > Hi folks,
> >
> > I've been running into issues with thrift-server-test and Kerberos. Below
> > is an excerpt of "KRB5_TRACE=/dev/stderr be/build/debug/rpc/thrift-
> server-test";
> > both SslConnectivity/1 and
> > SslConnectivity/2 fail the same way.
> >
> > I'm running Ubuntu16.04. I've seen this both on my host, as well as
> inside
> > of an Ubuntu 16.04 Docker container.
> >
> > Does this ring any bells?
> >
> > Thanks!
> >
> > -- Philip
> >
> >
> > [ RUN  ] KerberosOnAndOff/ThriftKerberizedParamsTest.
> SslConnectivity/2
> > Loading random data
> > Initializing database '7abf-cef9-113e-eae3/krb5kdc/principal' for realm
> '
> > KRBTEST.COM',
> > master key name 'K/m...@krbtest.com'
> > [31585] 1513120922.459517: Retrieving K/m...@krbtest.com from
> > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) with
> > result: 0/Success
> > [31586] 1513120922.472314: Retrieving K/m...@krbtest.com from
> > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) with
> > result: 0/Success
> > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info):
> setting
> > up network...
> > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info):
> > listening on fd 11: udp 0.0.0.0.51781 (pktinfo)
> > krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked Dec 12 15:22:02
> > philip-dev.gce.cloudera.com krb5kdc[31586](info): listening on fd 12:
> udp
> > ::.51781 (pktinfo) Dec 12 15:22:02 philip-dev.gce.cloudera.com
> > krb5kdc[31586](info): set up 2 sockets Dec 12 15:22:02
> > philip-dev.gce.cloudera.com krb5kdc[31586](info):
> > commencing operation
> > krb5kdc: starting...
> > Authenticating as principal philip/ad...@krbtest.com with password.
> > [31589] 1513120922.498913: Retrieving K/m...@krbtest.com from
> > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) with
> > result: 0/Success
> > WARNING: no policy specified for impala/localh...@krbtest.com;
> defaulting
> > to no policy Principal "impala/localh...@krbtest.com" created.
> > Authenticating as principal philip/ad...@krbtest.com with password.
> > [31590] 1513120922.508777: Retrieving K/m...@krbtest.com from
> > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) with
> > result: 0/Success
> > Entry for principal impala/localhost with kvno 2, encryption type
> > aes256-cts-hmac-sha1-96 added to keytab
> > WRFILE:7abf-cef9-113e-eae3/krb5kdc/impala_localhost.keytab.
> > Entry for principal impala/localhost with kvno 2, encryption type
> > aes128-cts-hmac-sha1-96 added to keytab
> > WRFILE:7abf-cef9-113e-eae3/krb5kdc/impala_localhost.keytab.
> > Entry for principal impala/localhost with kvno 2, encryption type
> > des3-cbc-sha1 added to keytab
> > WRFILE:7abf-cef9-113e-eae3/krb5kdc/impala_localhost.keytab.
> > Entry for principal impala/localhost with kvno 2, encryption type
> > arcfour-hmac added to keytab WRFILE:7abf-cef9-113e-eae3/
> > krb5kdc/impala_localhost.keytab.
> > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info): AS_REQ
> > (6 etypes {18 17 16 23 25 26}) 127.0.0.1: ISSUE: authtime 1513120922,
> > etypes
> > {rep=18 tkt=18 ses=18}, impala/localh...@krbtest.com for krbtgt/
> > krbtest@krbtest.com [31476] 1513120922.532304: ccselect can't find
> > appropriate cache for server principal impala@localhost [31476]
> > 1513120922.532347: Getting credentials impala/localh...@krbtest.com
> > -> impala@localhost using ccache FILE:/tmp/krb5cc_impala_internal
> > [31476] 1513120922.532382: Retrieving impala/localh...@krbtest.com ->
> > impala@localhost from FILE:/tmp/krb5cc_impala_internal with result:
> > -1765328243/Matching credential not found [31476] 1513120922.532407:
> > Retrieving impala/localh...@krbtest.com -> krbtgt/localhost@localhost
> > from FILE:/tmp/krb5cc_impala_internal with
> 

Re: build broken

[Sailesh Mukil]

On Wed, Dec 6, 2017 at 9:16 AM, Tim Armstrong 
wrote:

> I wonder if the "Rebase always" merge strategy would help with this
> scenario:
> https://gerrit-review.googlesource.com/Documentation/project-
> configuration.html#submit_type
>
>
"Rebase Always" seems like it would fix the above scenario. If the child
isn't merged, it wouldn't merge both the patches and it would fall back to
the client having to manually merge both the patches.

I recall doing a similar thing quite a while back (i.e. GVO-ing 2 patches
in one go), and the behavior was similar to "Rebase Always", so I had to
manually merge the patches after the GVO succeeded, which seems reasonable
as long as the GVO passes. Not sure if our gerrit settings changed at some
point.


> On Wed, Dec 6, 2017 at 9:04 AM, Daniel Hecht  wrote:
>
> > Sailesh, please keep that in mind with the ServicePool change -- you'll
> > need to explicitly merge part 1 first.
> >
>

Yes, I'll make sure to do that. Thanks.


> > On Wed, Dec 6, 2017 at 8:41 AM, Tim Armstrong 
> > wrote:
> >
> > > I think that's a consequence of the "cherry pick" merge strategy. It
> does
> > > seem like a flaw in our merge process. It would be nice if we could
> > > configure gerrit so that it didn't merge patches where the parent
> isn't a
> > > commit on the master branch.
> > >
> > > On Tue, Dec 5, 2017 at 9:31 PM, Sailesh Mukil 
> > > wrote:
> > >
> > > > Sorry about that. I thought they would both be merged together, but
> it
> > > > looks like that wasn't the case.
> > > >
> > > > On Tue, Dec 5, 2017 at 6:22 PM, Michael Ho 
> wrote:
> > > >
> > > > > Thanks Tim for fixing it and Jin Chul for investigating the
> problem.
> > > > Sorry
> > > > > for missing that during code review.
> > > > >
> > > > > On Tue, Dec 5, 2017 at 6:00 PM, Tim Armstrong <
> > tarmstr...@cloudera.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Thanks for investigating Jin Chul. I came to the same conclusion
> > and
> > > > > merged
> > > > > > the change.
> > > > > >
> > > > > > On Tue, Dec 5, 2017 at 5:57 PM, Jin Chul Kim 
> > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Currently Impala build is broken because the child change was
> > > merged
> > > > > > > without the merge of the parent change.
> > > > > > >
> > > > > > > (parent) https://gerrit.cloudera.org/#/c/8760/: [security]
> Make
> > > the
> > > > > > > kerberos principal configurable for Kudu servers
> > > > > > >
> > > > > > > (child) https://gerrit.cloudera.org/#/c/8761/: IMPALA-6256:
> > > > Incorrect
> > > > > > > principal will be used for internal connections if
> > > FLAGS_be_principal
> > > > > is
> > > > > > > set
> > > > > > >
> > > > > > > A workaround: cherry-pick the parent change.
> > > > > > >
> > > > > > > Best regards,
> > > > > > > Jinchul
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > Michael
> > > > >
> > > >
> > >
> >
>

Re: build broken

[Sailesh Mukil]

Sorry about that. I thought they would both be merged together, but it
looks like that wasn't the case.

On Tue, Dec 5, 2017 at 6:22 PM, Michael Ho  wrote:

> Thanks Tim for fixing it and Jin Chul for investigating the problem. Sorry
> for missing that during code review.
>
> On Tue, Dec 5, 2017 at 6:00 PM, Tim Armstrong 
> wrote:
>
> > Thanks for investigating Jin Chul. I came to the same conclusion and
> merged
> > the change.
> >
> > On Tue, Dec 5, 2017 at 5:57 PM, Jin Chul Kim  wrote:
> >
> > > Hi,
> > >
> > > Currently Impala build is broken because the child change was merged
> > > without the merge of the parent change.
> > >
> > > (parent) https://gerrit.cloudera.org/#/c/8760/: [security] Make the
> > > kerberos principal configurable for Kudu servers
> > >
> > > (child) https://gerrit.cloudera.org/#/c/8761/: IMPALA-6256: Incorrect
> > > principal will be used for internal connections if FLAGS_be_principal
> is
> > > set
> > >
> > > A workaround: cherry-pick the parent change.
> > >
> > > Best regards,
> > > Jinchul
> > >
> >
>
>
>
> --
> Thanks,
> Michael
>

Re: Parquet Data File Name

[Sailesh Mukil]

Hi Sky,

Currently I don't think it's possible to customize file names automatically
with each insert (someone can correct me if I'm wrong). As for the filename
convention, it's basically:
__data..parq

Code references:
https://github.com/apache/impala/blob/master/be/src/exec/hdfs-table-sink.cc#L229-L245

https://github.com/apache/impala/blob/master/be/src/exec/hdfs-table-sink.cc#L346-L348

- Sailesh

On Sun, Dec 3, 2017 at 11:58 PM, sky  wrote:

> Hi all,
> What is the relationship between the name of the parquet data file in
> HDFS and each time insert? What is the definition format of the name of the
> data file? Can you customize the name of the corresponding data file for
> each insert?

Re: [DISCUSS] 2.11.0 release

[Sailesh Mukil]

+1

Thanks for volunteering.

On Thu, Nov 30, 2017 at 1:02 PM, Jim Apple  wrote:

> +1
>
> On Thu, Nov 30, 2017 at 12:46 PM, Thomas Tauber-Marshall <
> tmarsh...@cloudera.com> wrote:
>
> > Folks,
> >
> > It has been over 2 months since we released Apache Impala 2.10.0 and
> there
> > have been new feature improvements and a good number of bug fixes checked
> > in since then.
> >
> > I propose that we release 2.11.0 soon and I volunteer to be its release
> > manager. Please speak up and let the community know if anyone has any
> > objections to this.
> >
> > Thanks,
> > Thomas
> >
>

Re: Protocol with re: to reverting previous patches

[Sailesh Mukil]

Yes, that's how I've usually done it.

On Tue, Nov 28, 2017 at 2:16 PM, David Knupp  wrote:

> Seems like a silly question, but I just want to make sure I do this right.
>
> IMPALA-6239 was filed because the patch for IMPALA-6068 introduced a
> regression. As a result, the patch was reverted:
> https://github.com/apache/incubator-impala/commit/d1c9510
>
> Presumably, I should now just resolve IMPALA-6239 and reopen
> IMPALA-6068with a pointer to the reverted commit?
>
> --
> David Knupp | Software Engineer
> Cloudera Inc., 433 California Street #1100. San Francisco, CA 94104
> 
>
>  glink&utm_campaign=reach>
>

Re: Impala minidump dump_breakpad_symbols

[Sailesh Mukil]

Hi Sandish,

Have you tried the steps outlined here?
https://cwiki.apache.org/confluence/display/IMPALA/Debugging+Impala+Minidumps

If it still doesn't work, as Brock mentioned, please paste portions of
errors or any other helpful output that you see.

- Sailesh



On Mon, Nov 27, 2017 at 8:23 PM, Brock Noland  wrote:

> Hi Sandish,
>
> I don't know how to help you, but I think it'd be useful if you shared a
> portion of the results file for folks to look at. Note that attachments
> don't work well so I'd just paste a portion of it inline.
>
> On Mon, Nov 27, 2017 at 9:15 PM, sanysand...@gmail.com <
> sanysand...@gmail.com> wrote:
>
> > Hi Team,
> >
> > I'm trying to get minidump_stackwalk results with dump breakpad symbols,
> > but symbols are not showing in results file, below are the steps which I
> > followed. Can someone verify or give me steps to get symbols in
> > minidump_stackwalk results ??
> >
> >
> > Steps:
> > dump_breakpad_symbols.py -b be/build/ --dump_syms
> > $IMPALA_HOME/toolchain/breakpad-1b704857f1e78a864e6942e613457e
> 55f1aecb60-p3/bin/dump_syms
> > -d /tmp/syms
> >
> > $IMPALA_HOME/toolchain/breakpad-1b704857f1e78a864e6942e61345
> > 7e55f1aecb60-p3/bin/minidump_stackwalk .dmp /tmp/syms
> > > /tmp/resolved.txt
> >
>