[Toolchain-CR] Versioning of build artifacts

[Tim Armstrong (Code Review)]

Tim Armstrong has submitted this change and it was merged.

Change subject: Versioning of build artifacts
..


Versioning of build artifacts

Previously publishing a new version of toolchain artifacts clobbered the
previous version. This is problematic when changing build options since
the new artifacts may not work in all circumstances. E.g. if an
older version of Impala depends on some particular detail of how the
older artifacts were built, we have no way of avoiding breakage.

Now a unique build ID is generated (including the git hash and the
jenkins job ID) and all build artifacts are uploaded into a directory
based on this unique id.

Also switches to building only the current artifacts by default (all
historical artifacts can be built with BUILD_HISTORICAL) to speed up
builds and reduce resource requirements.

Change-Id: Ifb8774a7bd4bcae0c135684078f5ce89a28f6bc2
---
M README.md
M buildall.sh
M functions.sh
M init.sh
4 files changed, 84 insertions(+), 32 deletions(-)

Approvals:
  Matthew Jacobs: Looks good to me, approved
  Tim Armstrong: Looks good to me, approved; Verified



-- 
To view, visit http://gerrit.cloudera.org:8080/4742
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ifb8774a7bd4bcae0c135684078f5ce89a28f6bc2
Gerrit-PatchSet: 3
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Matthew Jacobs 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Armstrong 

[Toolchain-CR] Versioning of build artifacts

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: Versioning of build artifacts
..


Patch Set 3: Code-Review+2

-- 
To view, visit http://gerrit.cloudera.org:8080/4742
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb8774a7bd4bcae0c135684078f5ce89a28f6bc2
Gerrit-PatchSet: 3
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Matthew Jacobs 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Toolchain-CR] Versioning of build artifacts

[Tim Armstrong (Code Review)]

Tim Armstrong has posted comments on this change.

Change subject: Versioning of build artifacts
..


Patch Set 3: Verified+1

-- 
To view, visit http://gerrit.cloudera.org:8080/4742
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb8774a7bd4bcae0c135684078f5ce89a28f6bc2
Gerrit-PatchSet: 3
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Matthew Jacobs 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

[Toolchain-CR] Versioning of build artifacts

[Matthew Jacobs (Code Review)]

Matthew Jacobs has posted comments on this change.

Change subject: Versioning of build artifacts
..


Patch Set 3: Code-Review+2

-- 
To view, visit http://gerrit.cloudera.org:8080/4742
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifb8774a7bd4bcae0c135684078f5ce89a28f6bc2
Gerrit-PatchSet: 3
Gerrit-Project: Toolchain
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong 
Gerrit-Reviewer: Matthew Jacobs 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-HasComments: No

Podling Report Reminder - November 2016

[johndament]

Dear podling,

This email was sent by an automated system on behalf of the Apache
Incubator PMC. It is an initial reminder to give you plenty of time to
prepare your quarterly board report.

The board meeting is scheduled for Wed, 16 November 2016, 10:30 am PDT.
The report for your podling will form a part of the Incubator PMC
report. The Incubator PMC requires your report to be submitted 2 weeks
before the board meeting, to allow sufficient time for review and
submission (Wed, November 02).

Please submit your report with sufficient time to allow the Incubator
PMC, and subsequently board members to review and digest. Again, the
very latest you should submit your report is 2 weeks prior to the board
meeting.

Thanks,

The Apache Incubator PMC

Submitting your Report

--

Your report should contain the following:

*   Your project name
*   A brief description of your project, which assumes no knowledge of
the project or necessarily of its field
*   A list of the three most important issues to address in the move
towards graduation.
*   Any issues that the Incubator PMC or ASF Board might wish/need to be
aware of
*   How has the community developed since the last report
*   How has the project developed since the last report.

This should be appended to the Incubator Wiki page at:

http://wiki.apache.org/incubator/November2016

Note: This is manually populated. You may need to wait a little before
this page is created from a template.

Mentors
---

Mentors should review reports for their project(s) and sign them off on
the Incubator wiki page. Signing off reports shows that you are
following the project - projects that are not signed may raise alarms
for the Incubator PMC.

Incubator PMC

Need to re-generate functional_kudu tables

[Lars Volker]

After this change  I needed to
re-generate the tables in functional_kudu. I did so running this command:

./bin/load-data.py -w functional-planner --table_formats=kudu/none --force

Afterwards I had to recompute stats for two of the tables to get the
PlannerTest to pass:

[localhost:21000] > compute stats functional_kudu.zipcode_incomes;
[localhost:21000] > compute stats functional_kudu.testtbl;

You might need to compute stats for the other tables, too.

Thanks Dimitris for helping me with this. Lars

Re: Cannot push new patch set to Gerrit

[Jim Apple]

OK, pushing (even without rebase) is now working for me.

On Mon, Oct 24, 2016 at 2:14 PM, Tim Armstrong  wrote:
> Yeah, I think I probably clobbered whatever commit was there before with my
> force push.
>
> On Mon, Oct 24, 2016 at 2:13 PM, Henry Robinson  wrote:
>
>> Ah, that might be something magit-push did in Emacs - I was trying to push
>> from Emacs and it clearly didn't do the right thing. I think we can delete
>> that new branch.
>>
>> On 24 October 2016 at 14:06, Jim Apple  wrote:
>>
>>> Doing a git fetch of the gerrit server, I see
>>>
>>>  * [new branch]  refs/for/master -> gerrit-asf/refs/for/master
>>>
>>> I'm not sure what's going on with that - the "master" branch is pushed
>>> to by "refs/for/master" or "refs/drafts/master", so this could be a
>>> typo of some sort.
>>>
>>> The commit 119bff3ac17f847f1a4137ea56707c72cd1119be is on
>>> refs/for/master without a "Reviewed-on" line. Tim, could you take a
>>> look? It's
>>>
>>> IMPALA-1430,IMPALA-4108: codegen all builtin aggregate functions
>>>
>>> On Mon, Oct 24, 2016 at 1:57 PM, Jim Apple  wrote:
>>> > What do you think we should force-push?
>>> >
>>> > What did we do to cause it, and how can we avoid it in the future?
>>> >
>>> > On Mon, Oct 24, 2016 at 12:40 PM, Henry Robinson 
>>> wrote:
>>> >> I also saw the same thing. The way I got around it was to try pushing
>>> to
>>> >> /drafts/ and then publishing it, which worked.
>>> >>
>>> >> We might need to force-push to gerrit/master to clean this up.
>>> >>
>>> >> On 24 October 2016 at 12:37, Lars Volker  wrote:
>>> >>
>>> >>> I tried to push another patch set to Gerrit and got the error below.
>>> This
>>> >>> worked about an hour ago and now stopped working. The local parent
>>> commit
>>> >>> is the same one as remote, I didn't rebase either. Has anyone seen
>>> this
>>> >>> before? I suspect I can rebase but then the review gets much more
>>> tedious
>>> >>> to follow.
>>> >>>
>>> >>> Thanks, Lars
>>> >>>
>>> >>> ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
>>> >>> To ssh://gerrit.cloudera.org:29418/Impala-ASF
>>> >>>  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
>>> >>> error: failed to push some refs to 'ssh://
>>> >>> l...@gerrit.cloudera.org:29418/Impala-ASF'
>>> >>> hint: Updates were rejected because a pushed branch tip is behind its
>>> >>> remote
>>> >>> hint: counterpart. Check out this branch and integrate the remote
>>> changes
>>> >>> hint: (e.g. 'git pull ...') before pushing again.
>>> >>> hint: See the 'Note about fast-forwards' in 'git push --help' for
>>> details.
>>> >>> ✗ ~/i2(i2521) ✗$
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Henry Robinson
>>> >> Software Engineer
>>> >> Cloudera
>>> >> 415-994-6679
>>>
>>
>>
>>
>> --
>> Henry Robinson
>> Software Engineer
>> Cloudera
>> 415-994-6679
>>

Re: Cannot push new patch set to Gerrit

[Tim Armstrong]

Yeah, I think I probably clobbered whatever commit was there before with my
force push.

On Mon, Oct 24, 2016 at 2:13 PM, Henry Robinson  wrote:

> Ah, that might be something magit-push did in Emacs - I was trying to push
> from Emacs and it clearly didn't do the right thing. I think we can delete
> that new branch.
>
> On 24 October 2016 at 14:06, Jim Apple  wrote:
>
>> Doing a git fetch of the gerrit server, I see
>>
>>  * [new branch]  refs/for/master -> gerrit-asf/refs/for/master
>>
>> I'm not sure what's going on with that - the "master" branch is pushed
>> to by "refs/for/master" or "refs/drafts/master", so this could be a
>> typo of some sort.
>>
>> The commit 119bff3ac17f847f1a4137ea56707c72cd1119be is on
>> refs/for/master without a "Reviewed-on" line. Tim, could you take a
>> look? It's
>>
>> IMPALA-1430,IMPALA-4108: codegen all builtin aggregate functions
>>
>> On Mon, Oct 24, 2016 at 1:57 PM, Jim Apple  wrote:
>> > What do you think we should force-push?
>> >
>> > What did we do to cause it, and how can we avoid it in the future?
>> >
>> > On Mon, Oct 24, 2016 at 12:40 PM, Henry Robinson 
>> wrote:
>> >> I also saw the same thing. The way I got around it was to try pushing
>> to
>> >> /drafts/ and then publishing it, which worked.
>> >>
>> >> We might need to force-push to gerrit/master to clean this up.
>> >>
>> >> On 24 October 2016 at 12:37, Lars Volker  wrote:
>> >>
>> >>> I tried to push another patch set to Gerrit and got the error below.
>> This
>> >>> worked about an hour ago and now stopped working. The local parent
>> commit
>> >>> is the same one as remote, I didn't rebase either. Has anyone seen
>> this
>> >>> before? I suspect I can rebase but then the review gets much more
>> tedious
>> >>> to follow.
>> >>>
>> >>> Thanks, Lars
>> >>>
>> >>> ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
>> >>> To ssh://gerrit.cloudera.org:29418/Impala-ASF
>> >>>  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
>> >>> error: failed to push some refs to 'ssh://
>> >>> l...@gerrit.cloudera.org:29418/Impala-ASF'
>> >>> hint: Updates were rejected because a pushed branch tip is behind its
>> >>> remote
>> >>> hint: counterpart. Check out this branch and integrate the remote
>> changes
>> >>> hint: (e.g. 'git pull ...') before pushing again.
>> >>> hint: See the 'Note about fast-forwards' in 'git push --help' for
>> details.
>> >>> ✗ ~/i2(i2521) ✗$
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Henry Robinson
>> >> Software Engineer
>> >> Cloudera
>> >> 415-994-6679
>>
>
>
>
> --
> Henry Robinson
> Software Engineer
> Cloudera
> 415-994-6679
>

Re: Cannot push new patch set to Gerrit

[Henry Robinson]

Ah, that might be something magit-push did in Emacs - I was trying to push
from Emacs and it clearly didn't do the right thing. I think we can delete
that new branch.

On 24 October 2016 at 14:06, Jim Apple  wrote:

> Doing a git fetch of the gerrit server, I see
>
>  * [new branch]  refs/for/master -> gerrit-asf/refs/for/master
>
> I'm not sure what's going on with that - the "master" branch is pushed
> to by "refs/for/master" or "refs/drafts/master", so this could be a
> typo of some sort.
>
> The commit 119bff3ac17f847f1a4137ea56707c72cd1119be is on
> refs/for/master without a "Reviewed-on" line. Tim, could you take a
> look? It's
>
> IMPALA-1430,IMPALA-4108: codegen all builtin aggregate functions
>
> On Mon, Oct 24, 2016 at 1:57 PM, Jim Apple  wrote:
> > What do you think we should force-push?
> >
> > What did we do to cause it, and how can we avoid it in the future?
> >
> > On Mon, Oct 24, 2016 at 12:40 PM, Henry Robinson 
> wrote:
> >> I also saw the same thing. The way I got around it was to try pushing to
> >> /drafts/ and then publishing it, which worked.
> >>
> >> We might need to force-push to gerrit/master to clean this up.
> >>
> >> On 24 October 2016 at 12:37, Lars Volker  wrote:
> >>
> >>> I tried to push another patch set to Gerrit and got the error below.
> This
> >>> worked about an hour ago and now stopped working. The local parent
> commit
> >>> is the same one as remote, I didn't rebase either. Has anyone seen this
> >>> before? I suspect I can rebase but then the review gets much more
> tedious
> >>> to follow.
> >>>
> >>> Thanks, Lars
> >>>
> >>> ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
> >>> To ssh://gerrit.cloudera.org:29418/Impala-ASF
> >>>  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
> >>> error: failed to push some refs to 'ssh://
> >>> l...@gerrit.cloudera.org:29418/Impala-ASF'
> >>> hint: Updates were rejected because a pushed branch tip is behind its
> >>> remote
> >>> hint: counterpart. Check out this branch and integrate the remote
> changes
> >>> hint: (e.g. 'git pull ...') before pushing again.
> >>> hint: See the 'Note about fast-forwards' in 'git push --help' for
> details.
> >>> ✗ ~/i2(i2521) ✗$
> >>>
> >>
> >>
> >>
> >> --
> >> Henry Robinson
> >> Software Engineer
> >> Cloudera
> >> 415-994-6679
>



-- 
Henry Robinson
Software Engineer
Cloudera
415-994-6679

Re: Cannot push new patch set to Gerrit

[Tim Armstrong]

I think that's just a consequence of me force-pushing it when trying to
work around the original problem. I deleted the "refs/for/master" branch

On Mon, Oct 24, 2016 at 2:06 PM, Jim Apple  wrote:

> Doing a git fetch of the gerrit server, I see
>
>  * [new branch]  refs/for/master -> gerrit-asf/refs/for/master
>
> I'm not sure what's going on with that - the "master" branch is pushed
> to by "refs/for/master" or "refs/drafts/master", so this could be a
> typo of some sort.
>
> The commit 119bff3ac17f847f1a4137ea56707c72cd1119be is on
> refs/for/master without a "Reviewed-on" line. Tim, could you take a
> look? It's
>
> IMPALA-1430,IMPALA-4108: codegen all builtin aggregate functions
>
> On Mon, Oct 24, 2016 at 1:57 PM, Jim Apple  wrote:
> > What do you think we should force-push?
> >
> > What did we do to cause it, and how can we avoid it in the future?
> >
> > On Mon, Oct 24, 2016 at 12:40 PM, Henry Robinson 
> wrote:
> >> I also saw the same thing. The way I got around it was to try pushing to
> >> /drafts/ and then publishing it, which worked.
> >>
> >> We might need to force-push to gerrit/master to clean this up.
> >>
> >> On 24 October 2016 at 12:37, Lars Volker  wrote:
> >>
> >>> I tried to push another patch set to Gerrit and got the error below.
> This
> >>> worked about an hour ago and now stopped working. The local parent
> commit
> >>> is the same one as remote, I didn't rebase either. Has anyone seen this
> >>> before? I suspect I can rebase but then the review gets much more
> tedious
> >>> to follow.
> >>>
> >>> Thanks, Lars
> >>>
> >>> ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
> >>> To ssh://gerrit.cloudera.org:29418/Impala-ASF
> >>>  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
> >>> error: failed to push some refs to 'ssh://
> >>> l...@gerrit.cloudera.org:29418/Impala-ASF'
> >>> hint: Updates were rejected because a pushed branch tip is behind its
> >>> remote
> >>> hint: counterpart. Check out this branch and integrate the remote
> changes
> >>> hint: (e.g. 'git pull ...') before pushing again.
> >>> hint: See the 'Note about fast-forwards' in 'git push --help' for
> details.
> >>> ✗ ~/i2(i2521) ✗$
> >>>
> >>
> >>
> >>
> >> --
> >> Henry Robinson
> >> Software Engineer
> >> Cloudera
> >> 415-994-6679
>

Re: Cannot push new patch set to Gerrit

[Jim Apple]

That's this patch, which isn't +2ed yet:

https://gerrit.cloudera.org/#/c/4655/

I checked, and that's the only patch in refs/for/master that's not in
master, though master has a few changes not in refs/for/master:

+commit 61fcb489745f3f0b3f1abbf9fbf666a29a6363de
+commit 0fbb5b7e71e55346c8b97ec143854dba0088f124
+commit ff6b450ad380ce840e18875a89d9cf98058277a3
+commit 51268c053ffe41dc1aa9f1b250878113d4225258
+commit 48085274fa8ae57453477db21dae0e53eae6b766
+commit e39f1676e1c9269fb6549e8c319adf1a7ea9d445
+commit d1d88aaccd03c461c4ce6224f765afe94b7c073b
+commit 8d7b01faea6362af675a2a335b462fad3e0caa03
+commit 8a49ceaae532163f17836b1050b639329424ee5c
+commit 041fa6d946e1cbe309593c4a5515bef88e06cdb4
+commit f8d48b8582b9e460c2e0e3dbb4881636f179ae73
+commit 9ef9512e5b58a75e075538b3b94ac551363609e5
+commit 2fa1633e409e9018c012df81f291a0d7566e

On Mon, Oct 24, 2016 at 2:06 PM, Jim Apple  wrote:
> Doing a git fetch of the gerrit server, I see
>
>  * [new branch]  refs/for/master -> gerrit-asf/refs/for/master
>
> I'm not sure what's going on with that - the "master" branch is pushed
> to by "refs/for/master" or "refs/drafts/master", so this could be a
> typo of some sort.
>
> The commit 119bff3ac17f847f1a4137ea56707c72cd1119be is on
> refs/for/master without a "Reviewed-on" line. Tim, could you take a
> look? It's
>
> IMPALA-1430,IMPALA-4108: codegen all builtin aggregate functions
>
> On Mon, Oct 24, 2016 at 1:57 PM, Jim Apple  wrote:
>> What do you think we should force-push?
>>
>> What did we do to cause it, and how can we avoid it in the future?
>>
>> On Mon, Oct 24, 2016 at 12:40 PM, Henry Robinson  wrote:
>>> I also saw the same thing. The way I got around it was to try pushing to
>>> /drafts/ and then publishing it, which worked.
>>>
>>> We might need to force-push to gerrit/master to clean this up.
>>>
>>> On 24 October 2016 at 12:37, Lars Volker  wrote:
>>>
 I tried to push another patch set to Gerrit and got the error below. This
 worked about an hour ago and now stopped working. The local parent commit
 is the same one as remote, I didn't rebase either. Has anyone seen this
 before? I suspect I can rebase but then the review gets much more tedious
 to follow.

 Thanks, Lars

 ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
 To ssh://gerrit.cloudera.org:29418/Impala-ASF
  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
 error: failed to push some refs to 'ssh://
 l...@gerrit.cloudera.org:29418/Impala-ASF'
 hint: Updates were rejected because a pushed branch tip is behind its
 remote
 hint: counterpart. Check out this branch and integrate the remote changes
 hint: (e.g. 'git pull ...') before pushing again.
 hint: See the 'Note about fast-forwards' in 'git push --help' for details.
 ✗ ~/i2(i2521) ✗$

>>>
>>>
>>>
>>> --
>>> Henry Robinson
>>> Software Engineer
>>> Cloudera
>>> 415-994-6679

Re: Cannot push new patch set to Gerrit

[Jim Apple]

Doing a git fetch of the gerrit server, I see

 * [new branch]  refs/for/master -> gerrit-asf/refs/for/master

I'm not sure what's going on with that - the "master" branch is pushed
to by "refs/for/master" or "refs/drafts/master", so this could be a
typo of some sort.

The commit 119bff3ac17f847f1a4137ea56707c72cd1119be is on
refs/for/master without a "Reviewed-on" line. Tim, could you take a
look? It's

IMPALA-1430,IMPALA-4108: codegen all builtin aggregate functions

On Mon, Oct 24, 2016 at 1:57 PM, Jim Apple  wrote:
> What do you think we should force-push?
>
> What did we do to cause it, and how can we avoid it in the future?
>
> On Mon, Oct 24, 2016 at 12:40 PM, Henry Robinson  wrote:
>> I also saw the same thing. The way I got around it was to try pushing to
>> /drafts/ and then publishing it, which worked.
>>
>> We might need to force-push to gerrit/master to clean this up.
>>
>> On 24 October 2016 at 12:37, Lars Volker  wrote:
>>
>>> I tried to push another patch set to Gerrit and got the error below. This
>>> worked about an hour ago and now stopped working. The local parent commit
>>> is the same one as remote, I didn't rebase either. Has anyone seen this
>>> before? I suspect I can rebase but then the review gets much more tedious
>>> to follow.
>>>
>>> Thanks, Lars
>>>
>>> ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
>>> To ssh://gerrit.cloudera.org:29418/Impala-ASF
>>>  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
>>> error: failed to push some refs to 'ssh://
>>> l...@gerrit.cloudera.org:29418/Impala-ASF'
>>> hint: Updates were rejected because a pushed branch tip is behind its
>>> remote
>>> hint: counterpart. Check out this branch and integrate the remote changes
>>> hint: (e.g. 'git pull ...') before pushing again.
>>> hint: See the 'Note about fast-forwards' in 'git push --help' for details.
>>> ✗ ~/i2(i2521) ✗$
>>>
>>
>>
>>
>> --
>> Henry Robinson
>> Software Engineer
>> Cloudera
>> 415-994-6679

Re: Cannot push new patch set to Gerrit

[Jim Apple]

What do you think we should force-push?

What did we do to cause it, and how can we avoid it in the future?

On Mon, Oct 24, 2016 at 12:40 PM, Henry Robinson  wrote:
> I also saw the same thing. The way I got around it was to try pushing to
> /drafts/ and then publishing it, which worked.
>
> We might need to force-push to gerrit/master to clean this up.
>
> On 24 October 2016 at 12:37, Lars Volker  wrote:
>
>> I tried to push another patch set to Gerrit and got the error below. This
>> worked about an hour ago and now stopped working. The local parent commit
>> is the same one as remote, I didn't rebase either. Has anyone seen this
>> before? I suspect I can rebase but then the review gets much more tedious
>> to follow.
>>
>> Thanks, Lars
>>
>> ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
>> To ssh://gerrit.cloudera.org:29418/Impala-ASF
>>  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
>> error: failed to push some refs to 'ssh://
>> l...@gerrit.cloudera.org:29418/Impala-ASF'
>> hint: Updates were rejected because a pushed branch tip is behind its
>> remote
>> hint: counterpart. Check out this branch and integrate the remote changes
>> hint: (e.g. 'git pull ...') before pushing again.
>> hint: See the 'Note about fast-forwards' in 'git push --help' for details.
>> ✗ ~/i2(i2521) ✗$
>>
>
>
>
> --
> Henry Robinson
> Software Engineer
> Cloudera
> 415-994-6679

Re: Cannot push new patch set to Gerrit

[Henry Robinson]

I also saw the same thing. The way I got around it was to try pushing to
/drafts/ and then publishing it, which worked.

We might need to force-push to gerrit/master to clean this up.

On 24 October 2016 at 12:37, Lars Volker  wrote:

> I tried to push another patch set to Gerrit and got the error below. This
> worked about an hour ago and now stopped working. The local parent commit
> is the same one as remote, I didn't rebase either. Has anyone seen this
> before? I suspect I can rebase but then the review gets much more tedious
> to follow.
>
> Thanks, Lars
>
> ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
> To ssh://gerrit.cloudera.org:29418/Impala-ASF
>  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
> error: failed to push some refs to 'ssh://
> l...@gerrit.cloudera.org:29418/Impala-ASF'
> hint: Updates were rejected because a pushed branch tip is behind its
> remote
> hint: counterpart. Check out this branch and integrate the remote changes
> hint: (e.g. 'git pull ...') before pushing again.
> hint: See the 'Note about fast-forwards' in 'git push --help' for details.
> ✗ ~/i2(i2521) ✗$
>



-- 
Henry Robinson
Software Engineer
Cloudera
415-994-6679

Cannot push new patch set to Gerrit

[Lars Volker]

I tried to push another patch set to Gerrit and got the error below. This
worked about an hour ago and now stopped working. The local parent commit
is the same one as remote, I didn't rebase either. Has anyone seen this
before? I suspect I can rebase but then the review gets much more tedious
to follow.

Thanks, Lars

✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
To ssh://gerrit.cloudera.org:29418/Impala-ASF
 ! [rejected]HEAD -> refs/for/master (non-fast-forward)
error: failed to push some refs to 'ssh://
l...@gerrit.cloudera.org:29418/Impala-ASF'
hint: Updates were rejected because a pushed branch tip is behind its remote
hint: counterpart. Check out this branch and integrate the remote changes
hint: (e.g. 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
✗ ~/i2(i2521) ✗$

Re: crash in PHJ on master

[Tim Armstrong]

Seems like a problem that we're not doing any testing of these code paths.
I'm sure there are other bugs in there.

On Mon, Oct 24, 2016 at 8:50 AM, Marcel Kornacker 
wrote:

> Apparently that's the cause. I'll file a jira.
>
> On Mon, Oct 24, 2016 at 8:48 AM, Alex Behm  wrote:
>
> > Maybe you are using an "unusual" log level?
> >
> > On Mon, Oct 24, 2016 at 8:35 AM, Marcel Kornacker 
> > wrote:
> >
> > > I'm seeing a crash in PartitionedHashJoinNode, is anyone experiencing
> the
> > > same or know what's going on?
> > >
> > > Here is the stack:
> > > C  [libExec.so+0x53706c]  std::unique_ptr > > std::default_delete >::get() const+0x18
> > > C  [libExec.so+0x54d6bc]
> > >  impala::PartitionedHashJoinNode::NodeDebugString() const+0x296
> > > C  [libExec.so+0x54d0bd]
> > >  impala::PartitionedHashJoinNode::UpdateState(impala::
> > > PartitionedHashJoinNode::HashJoinState)+0x37f
> > > C  [libExec.so+0x53eeb5]
> > >  impala::PartitionedHashJoinNode::Open(impala::RuntimeState*)+0x793
> > > C  [libExec.so+0x512fde]
> > >  impala::PartitionedAggregationNode::Open(impala::RuntimeState*)+0x4b4
> > > C  [libRuntime.so+0x53ff0e]
> > >  impala::PlanFragmentExecutor::OpenInternal()+0x3ae
> > > C  [libRuntime.so+0x53fa33]  impala::PlanFragmentExecutor::
> Open()+0x229
> > > C  [libService.so+0x30e7a8]
> > >  impala::FragmentMgr::FragmentExecState::Exec()+0x66
> > > C  [libService.so+0x31c0c8]
> > >  impala::FragmentMgr::FragmentThread(impala::TUniqueId)+0x78
> > > C  [libService.so+0x33863e]  boost::_mfi::mf1 impala::FragmentMgr,
> > > impala::TUniqueId>::operator()(impala::FragmentMgr*,
> impala::TUniqueId)
> > > const+0x84
> > > C  [libService.so+0x336c6b]  void
> > > boost::_bi::list2,
> > > boost::_bi::value >::operator() > mf1 > > impala::FragmentMgr, impala::TUniqueId>,
> > > boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 > > impala::FragmentMgr, impala::TUniqueId>&, boost::_bi::list0&, int)+0x7d
> > > C  [libService.so+0x335557]  boost::_bi::bind_t > > boost::_mfi::mf1,
> > > boost::_bi::list2,
> > > boost::_bi::value > >::operator()()+0x3b
> > > C  [libService.so+0x33288c]
> > >  boost::detail::function::void_function_obj_invoker0 > > bi::bind_t > > boost::_mfi::mf1,
> > > boost::_bi::list2,
> > > boost::_bi::value > >,
> > > void>::invoke(boost::detail::function::function_buffer&)+0x23
> > > C  [libUtil.so+0x2e75f0]  boost::function0::operator()()
> > const+0x52
> > > C  [libUtil.so+0x2e6b97]  impala::Thread::SuperviseThread(std::string
> > > const&, std::string const&, boost::function,
> > > impala::Promise*)+0x2c5
> > > C  [libUtil.so+0x2ee48e]  void
> > > boost::_bi::list4,
> > > boost::_bi::value, boost::_bi::value > > ()()> >, boost::_bi::value*> >::operator() > > (*)(std::string const&, std::string const&, boost::function,
> > > impala::Promise*), boost::_bi::list0>(boost::_bi::type,
> void
> > > (*&)(std::string const&, std::string const&, boost::function ()()>,
> > > impala::Promise*), boost::_bi::list0&, int)+0xb2
> > > C  [libUtil.so+0x2ee3d1]  boost::_bi::bind_t > > const&, std::string const&, boost::function,
> > > impala::Promise*), boost::_bi::list4 > value,
> > > boost::_bi::value, boost::_bi::value > > ()()> >, boost::_bi::value*> >
> > >::operator()()+0x3b
> > > C  [libUtil.so+0x2ee32c]
> > >  boost::detail::thread_data > (*)(std::string
> > > const&, std::string const&, boost::function,
> > > impala::Promise*), boost::_bi::list4 > value,
> > > boost::_bi::value, boost::_bi::value > > ()()> >, boost::_bi::value*> > > >::run()+0x1e
> > >
> > > I'm at git hash ff6b450  (IMPALA-4285/IMPALA-4286: Fixes for Parquet
> > > scanner with MT_DOP > 0), ie, the latest on master.
> > >
> >
>

Re: Bulk JIRA email, or how to find good newbie tasks

[Marcel Kornacker]

I guess those can happen concurrently (ie, we need to review everything
labelled 'ramp-up').

On Mon, Oct 24, 2016 at 9:39 AM, Jim Apple  wrote:

> Why do we need to fix the "ramp*" bugs BEFORE we label "newbie" bugs?
>
> On Sat, Oct 22, 2016 at 1:12 PM, Marcel Kornacker 
> wrote:
> > Before you do that we need to sit down and reclassify what's currently
> > labeled 'ramp-up', some of those tasks aren't really suitable as that
> > (because they don't have an educational component).
> >
> > On Fri, Oct 21, 2016 at 10:53 AM, Jim Apple 
> wrote:
> >> SGTM. I will pick some of the PMC members and ask them to individually
> >> classify all of the ramp* bugs in a single component by marking the
> >> ones good for newbies as "newbie" and removing the "ramp*" tags from
> >> "newbie" issues.
> >>
> >> I hope this will produce a nice set of issues for newbies to choose
> >> from to help us entice new developers, testers, doc writers, and so
> >> on.
> >>
> >> On Fri, Oct 21, 2016 at 9:32 AM, Marcel Kornacker 
> wrote:
> >>> I agree, we have enough rules regarding jira labeling already.
> >>>
> >>> A "newbie" label to indicate tasks that someone without any/much
> >>> exposure to the codebase should be able to handle seems like a good
> >>> idea.
> >>>
> >>> "Ramp-up" really means something else (good learning experience, but
> >>> could be moderately complex and require some understanding of the
> >>> codebase).
> >>>
> >>> On Thu, Oct 20, 2016 at 10:32 PM, Henry Robinson 
> wrote:
>  I still think that adding more complexity to the JIRA state space is
> only
>  going to increase the cognitive burden of managing JIRAs, something I
> spend
>  quite a lot of my life doing already. It would need to be a pretty
> big win
>  for it to be worth it to me.
> 
>  Happy to go with the consensus, but would prefer to avoid more labels
> where
>  possible.
> 
>  On 20 October 2016 at 16:44, Jim Apple  wrote:
> 
> > Ah, I was only suggesting this "non-newbie" as a tool for committers
> > to use to help us find good newbie bugs, not as a tool for newbies.
> >
> > It will probably take a while to triage everything, and some people
> > will not get their triaging work done in a timely fashion, and new
> > bugs will show up and need triaging, and old bugs will (hopefully) be
> > fixed by newbies, requiring us to go and find new bugs to label
> > "newbie"
> >
> > On Thu, Oct 20, 2016 at 4:40 PM, Henry Robinson 
> > wrote:
> > > Right. Which 'newbie' is going to have that degree of JIRA nuance?
> > >
> > > I also feel it's better to show new contributors what they *could*
> do,
> > than
> > > to carefully mark things that they *should not* do.
> > >
> > > On 20 October 2016 at 16:28, Jim Apple 
> wrote:
> > >
> > >> So you're suggesting that there be no label for "this is triaged
> and
> > >> not suitable for newbies"?
> > >>
> > >> On Thu, Oct 20, 2016 at 4:14 PM, Henry Robinson <
> he...@cloudera.com>
> > >> wrote:
> > >> > I would strongly prefer not adding "non-newbie". It seems to
> have
> > limited
> > >> > use, and is another way to increase the state space of JIRA
> labels,
> > >> > components, etc, etc.
> > >> >
> > >> > Let's just find some good first-patch candidates and tag them.
> > >> >
> > >> > On 20 October 2016 at 16:03, Jim Apple 
> wrote:
> > >> >
> > >> >> Since I have heard no objection to these tag names, I'm picking
> > >> >> "newbie" and "non-newbie". I am going to start emailing some
> PMC
> > >> >> members to ask them to categorize issues.
> > >> >>
> > >> >> On Tue, Oct 18, 2016 at 9:57 AM, Jim Apple <
> jbap...@cloudera.com>
> > >> wrote:
> > >> >> > How shall we distinguish between the following three classes
> of
> > >> issues:
> > >> >> >
> > >> >> > 1. Un-triaged ramp-up issues
> > >> >> > 2. ramp-up issues that are not for newbies
> > >> >> > 3. newbie issues
> > >> >> >
> > >> >> > We could add two tags, "newbie" and "non-newbie". We could
> call the
> > >> >> > second tag something other than "non-newbie", like
> "second-patch"
> > or
> > >> >> > "sophomore".
> > >> >> >
> > >> >> > Thoughts?
> > >> >> >
> > >> >> > On Mon, Oct 17, 2016 at 10:27 PM, Marcel Kornacker <
> > >> mar...@cloudera.com>
> > >> >> wrote:
> > >> >> >> Please don't add that comment. :)
> > >> >> >>
> > >> >> >> What's currently labelled ramp-up is often not a good
> newbie task
> > >> (and
> > >> >> >> maybe not even a good ramp-up task). The best way to
> identify
> > newbie
> > >> >> >> tasks is for a few senior engineers to sift through the
> ramp-up
> > tasks
> > >> >> >> and pick out maybe a few dozen that truly qualify as newbie
> tasks.
> > >> >> >>
> > >> >> >> I'm happy to help out with that when I get back.
> > >

Re: Bulk JIRA email, or how to find good newbie tasks

[Jim Apple]

Why do we need to fix the "ramp*" bugs BEFORE we label "newbie" bugs?

On Sat, Oct 22, 2016 at 1:12 PM, Marcel Kornacker  wrote:
> Before you do that we need to sit down and reclassify what's currently
> labeled 'ramp-up', some of those tasks aren't really suitable as that
> (because they don't have an educational component).
>
> On Fri, Oct 21, 2016 at 10:53 AM, Jim Apple  wrote:
>> SGTM. I will pick some of the PMC members and ask them to individually
>> classify all of the ramp* bugs in a single component by marking the
>> ones good for newbies as "newbie" and removing the "ramp*" tags from
>> "newbie" issues.
>>
>> I hope this will produce a nice set of issues for newbies to choose
>> from to help us entice new developers, testers, doc writers, and so
>> on.
>>
>> On Fri, Oct 21, 2016 at 9:32 AM, Marcel Kornacker  
>> wrote:
>>> I agree, we have enough rules regarding jira labeling already.
>>>
>>> A "newbie" label to indicate tasks that someone without any/much
>>> exposure to the codebase should be able to handle seems like a good
>>> idea.
>>>
>>> "Ramp-up" really means something else (good learning experience, but
>>> could be moderately complex and require some understanding of the
>>> codebase).
>>>
>>> On Thu, Oct 20, 2016 at 10:32 PM, Henry Robinson  wrote:
 I still think that adding more complexity to the JIRA state space is only
 going to increase the cognitive burden of managing JIRAs, something I spend
 quite a lot of my life doing already. It would need to be a pretty big win
 for it to be worth it to me.

 Happy to go with the consensus, but would prefer to avoid more labels where
 possible.

 On 20 October 2016 at 16:44, Jim Apple  wrote:

> Ah, I was only suggesting this "non-newbie" as a tool for committers
> to use to help us find good newbie bugs, not as a tool for newbies.
>
> It will probably take a while to triage everything, and some people
> will not get their triaging work done in a timely fashion, and new
> bugs will show up and need triaging, and old bugs will (hopefully) be
> fixed by newbies, requiring us to go and find new bugs to label
> "newbie"
>
> On Thu, Oct 20, 2016 at 4:40 PM, Henry Robinson 
> wrote:
> > Right. Which 'newbie' is going to have that degree of JIRA nuance?
> >
> > I also feel it's better to show new contributors what they *could* do,
> than
> > to carefully mark things that they *should not* do.
> >
> > On 20 October 2016 at 16:28, Jim Apple  wrote:
> >
> >> So you're suggesting that there be no label for "this is triaged and
> >> not suitable for newbies"?
> >>
> >> On Thu, Oct 20, 2016 at 4:14 PM, Henry Robinson 
> >> wrote:
> >> > I would strongly prefer not adding "non-newbie". It seems to have
> limited
> >> > use, and is another way to increase the state space of JIRA labels,
> >> > components, etc, etc.
> >> >
> >> > Let's just find some good first-patch candidates and tag them.
> >> >
> >> > On 20 October 2016 at 16:03, Jim Apple  wrote:
> >> >
> >> >> Since I have heard no objection to these tag names, I'm picking
> >> >> "newbie" and "non-newbie". I am going to start emailing some PMC
> >> >> members to ask them to categorize issues.
> >> >>
> >> >> On Tue, Oct 18, 2016 at 9:57 AM, Jim Apple 
> >> wrote:
> >> >> > How shall we distinguish between the following three classes of
> >> issues:
> >> >> >
> >> >> > 1. Un-triaged ramp-up issues
> >> >> > 2. ramp-up issues that are not for newbies
> >> >> > 3. newbie issues
> >> >> >
> >> >> > We could add two tags, "newbie" and "non-newbie". We could call 
> >> >> > the
> >> >> > second tag something other than "non-newbie", like "second-patch"
> or
> >> >> > "sophomore".
> >> >> >
> >> >> > Thoughts?
> >> >> >
> >> >> > On Mon, Oct 17, 2016 at 10:27 PM, Marcel Kornacker <
> >> mar...@cloudera.com>
> >> >> wrote:
> >> >> >> Please don't add that comment. :)
> >> >> >>
> >> >> >> What's currently labelled ramp-up is often not a good newbie task
> >> (and
> >> >> >> maybe not even a good ramp-up task). The best way to identify
> newbie
> >> >> >> tasks is for a few senior engineers to sift through the ramp-up
> tasks
> >> >> >> and pick out maybe a few dozen that truly qualify as newbie 
> >> >> >> tasks.
> >> >> >>
> >> >> >> I'm happy to help out with that when I get back.
> >> >> >>
> >> >> >> On Mon, Oct 17, 2016 at 6:50 PM, Jim Apple 
> >> >> wrote:
> >> >> >>> The Impala JIRA has 129 tasks that have no assignee, are still
> open,
> >> >> >>> and are labelled ramp* (i.e. ramp-up, ramp-up-introductory,
> etc.).
> >> >> >>>
> >> >> >>> I'd like to find which of those tasks are good tasks for someone
> who
> >> >> >>> is making their first Impal

Re: crash in PHJ on master

[Marcel Kornacker]

Apparently that's the cause. I'll file a jira.

On Mon, Oct 24, 2016 at 8:48 AM, Alex Behm  wrote:

> Maybe you are using an "unusual" log level?
>
> On Mon, Oct 24, 2016 at 8:35 AM, Marcel Kornacker 
> wrote:
>
> > I'm seeing a crash in PartitionedHashJoinNode, is anyone experiencing the
> > same or know what's going on?
> >
> > Here is the stack:
> > C  [libExec.so+0x53706c]  std::unique_ptr > std::default_delete >::get() const+0x18
> > C  [libExec.so+0x54d6bc]
> >  impala::PartitionedHashJoinNode::NodeDebugString() const+0x296
> > C  [libExec.so+0x54d0bd]
> >  impala::PartitionedHashJoinNode::UpdateState(impala::
> > PartitionedHashJoinNode::HashJoinState)+0x37f
> > C  [libExec.so+0x53eeb5]
> >  impala::PartitionedHashJoinNode::Open(impala::RuntimeState*)+0x793
> > C  [libExec.so+0x512fde]
> >  impala::PartitionedAggregationNode::Open(impala::RuntimeState*)+0x4b4
> > C  [libRuntime.so+0x53ff0e]
> >  impala::PlanFragmentExecutor::OpenInternal()+0x3ae
> > C  [libRuntime.so+0x53fa33]  impala::PlanFragmentExecutor::Open()+0x229
> > C  [libService.so+0x30e7a8]
> >  impala::FragmentMgr::FragmentExecState::Exec()+0x66
> > C  [libService.so+0x31c0c8]
> >  impala::FragmentMgr::FragmentThread(impala::TUniqueId)+0x78
> > C  [libService.so+0x33863e]  boost::_mfi::mf1 > impala::TUniqueId>::operator()(impala::FragmentMgr*, impala::TUniqueId)
> > const+0x84
> > C  [libService.so+0x336c6b]  void
> > boost::_bi::list2,
> > boost::_bi::value >::operator() mf1 > impala::FragmentMgr, impala::TUniqueId>,
> > boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 > impala::FragmentMgr, impala::TUniqueId>&, boost::_bi::list0&, int)+0x7d
> > C  [libService.so+0x335557]  boost::_bi::bind_t > boost::_mfi::mf1,
> > boost::_bi::list2,
> > boost::_bi::value > >::operator()()+0x3b
> > C  [libService.so+0x33288c]
> >  boost::detail::function::void_function_obj_invoker0 > bi::bind_t > boost::_mfi::mf1,
> > boost::_bi::list2,
> > boost::_bi::value > >,
> > void>::invoke(boost::detail::function::function_buffer&)+0x23
> > C  [libUtil.so+0x2e75f0]  boost::function0::operator()()
> const+0x52
> > C  [libUtil.so+0x2e6b97]  impala::Thread::SuperviseThread(std::string
> > const&, std::string const&, boost::function,
> > impala::Promise*)+0x2c5
> > C  [libUtil.so+0x2ee48e]  void
> > boost::_bi::list4,
> > boost::_bi::value, boost::_bi::value > ()()> >, boost::_bi::value*> >::operator() > (*)(std::string const&, std::string const&, boost::function,
> > impala::Promise*), boost::_bi::list0>(boost::_bi::type, void
> > (*&)(std::string const&, std::string const&, boost::function,
> > impala::Promise*), boost::_bi::list0&, int)+0xb2
> > C  [libUtil.so+0x2ee3d1]  boost::_bi::bind_t > const&, std::string const&, boost::function,
> > impala::Promise*), boost::_bi::list4 value,
> > boost::_bi::value, boost::_bi::value > ()()> >, boost::_bi::value*> >
> >::operator()()+0x3b
> > C  [libUtil.so+0x2ee32c]
> >  boost::detail::thread_data (*)(std::string
> > const&, std::string const&, boost::function,
> > impala::Promise*), boost::_bi::list4 value,
> > boost::_bi::value, boost::_bi::value > ()()> >, boost::_bi::value*> > > >::run()+0x1e
> >
> > I'm at git hash ff6b450  (IMPALA-4285/IMPALA-4286: Fixes for Parquet
> > scanner with MT_DOP > 0), ie, the latest on master.
> >
>

Re: crash in PHJ on master

[Alex Behm]

Maybe you are using an "unusual" log level?

On Mon, Oct 24, 2016 at 8:35 AM, Marcel Kornacker 
wrote:

> I'm seeing a crash in PartitionedHashJoinNode, is anyone experiencing the
> same or know what's going on?
>
> Here is the stack:
> C  [libExec.so+0x53706c]  std::unique_ptr std::default_delete >::get() const+0x18
> C  [libExec.so+0x54d6bc]
>  impala::PartitionedHashJoinNode::NodeDebugString() const+0x296
> C  [libExec.so+0x54d0bd]
>  impala::PartitionedHashJoinNode::UpdateState(impala::
> PartitionedHashJoinNode::HashJoinState)+0x37f
> C  [libExec.so+0x53eeb5]
>  impala::PartitionedHashJoinNode::Open(impala::RuntimeState*)+0x793
> C  [libExec.so+0x512fde]
>  impala::PartitionedAggregationNode::Open(impala::RuntimeState*)+0x4b4
> C  [libRuntime.so+0x53ff0e]
>  impala::PlanFragmentExecutor::OpenInternal()+0x3ae
> C  [libRuntime.so+0x53fa33]  impala::PlanFragmentExecutor::Open()+0x229
> C  [libService.so+0x30e7a8]
>  impala::FragmentMgr::FragmentExecState::Exec()+0x66
> C  [libService.so+0x31c0c8]
>  impala::FragmentMgr::FragmentThread(impala::TUniqueId)+0x78
> C  [libService.so+0x33863e]  boost::_mfi::mf1 impala::TUniqueId>::operator()(impala::FragmentMgr*, impala::TUniqueId)
> const+0x84
> C  [libService.so+0x336c6b]  void
> boost::_bi::list2,
> boost::_bi::value >::operator() impala::FragmentMgr, impala::TUniqueId>,
> boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 impala::FragmentMgr, impala::TUniqueId>&, boost::_bi::list0&, int)+0x7d
> C  [libService.so+0x335557]  boost::_bi::bind_t boost::_mfi::mf1,
> boost::_bi::list2,
> boost::_bi::value > >::operator()()+0x3b
> C  [libService.so+0x33288c]
>  boost::detail::function::void_function_obj_invoker0 bi::bind_t boost::_mfi::mf1,
> boost::_bi::list2,
> boost::_bi::value > >,
> void>::invoke(boost::detail::function::function_buffer&)+0x23
> C  [libUtil.so+0x2e75f0]  boost::function0::operator()() const+0x52
> C  [libUtil.so+0x2e6b97]  impala::Thread::SuperviseThread(std::string
> const&, std::string const&, boost::function,
> impala::Promise*)+0x2c5
> C  [libUtil.so+0x2ee48e]  void
> boost::_bi::list4,
> boost::_bi::value, boost::_bi::value ()()> >, boost::_bi::value*> >::operator() (*)(std::string const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list0>(boost::_bi::type, void
> (*&)(std::string const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list0&, int)+0xb2
> C  [libUtil.so+0x2ee3d1]  boost::_bi::bind_t const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list4,
> boost::_bi::value, boost::_bi::value ()()> >, boost::_bi::value*> > >::operator()()+0x3b
> C  [libUtil.so+0x2ee32c]
>  boost::detail::thread_data const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list4,
> boost::_bi::value, boost::_bi::value ()()> >, boost::_bi::value*> > > >::run()+0x1e
>
> I'm at git hash ff6b450  (IMPALA-4285/IMPALA-4286: Fixes for Parquet
> scanner with MT_DOP > 0), ie, the latest on master.
>

Re: crash in PHJ on master

[Jim Apple]

I have a Jenkins build that passed all of the e2e tests (core
exploration strategy, debug build) with no crashes.

On Mon, Oct 24, 2016 at 8:35 AM, Marcel Kornacker  wrote:
> I'm seeing a crash in PartitionedHashJoinNode, is anyone experiencing the
> same or know what's going on?
>
> Here is the stack:
> C  [libExec.so+0x53706c]  std::unique_ptr std::default_delete >::get() const+0x18
> C  [libExec.so+0x54d6bc]
>  impala::PartitionedHashJoinNode::NodeDebugString() const+0x296
> C  [libExec.so+0x54d0bd]
>  
> impala::PartitionedHashJoinNode::UpdateState(impala::PartitionedHashJoinNode::HashJoinState)+0x37f
> C  [libExec.so+0x53eeb5]
>  impala::PartitionedHashJoinNode::Open(impala::RuntimeState*)+0x793
> C  [libExec.so+0x512fde]
>  impala::PartitionedAggregationNode::Open(impala::RuntimeState*)+0x4b4
> C  [libRuntime.so+0x53ff0e]
>  impala::PlanFragmentExecutor::OpenInternal()+0x3ae
> C  [libRuntime.so+0x53fa33]  impala::PlanFragmentExecutor::Open()+0x229
> C  [libService.so+0x30e7a8]
>  impala::FragmentMgr::FragmentExecState::Exec()+0x66
> C  [libService.so+0x31c0c8]
>  impala::FragmentMgr::FragmentThread(impala::TUniqueId)+0x78
> C  [libService.so+0x33863e]  boost::_mfi::mf1 impala::TUniqueId>::operator()(impala::FragmentMgr*, impala::TUniqueId)
> const+0x84
> C  [libService.so+0x336c6b]  void
> boost::_bi::list2,
> boost::_bi::value >::operator() impala::FragmentMgr, impala::TUniqueId>,
> boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 impala::FragmentMgr, impala::TUniqueId>&, boost::_bi::list0&, int)+0x7d
> C  [libService.so+0x335557]  boost::_bi::bind_t boost::_mfi::mf1,
> boost::_bi::list2,
> boost::_bi::value > >::operator()()+0x3b
> C  [libService.so+0x33288c]
>  boost::detail::function::void_function_obj_invoker0 boost::_mfi::mf1,
> boost::_bi::list2,
> boost::_bi::value > >,
> void>::invoke(boost::detail::function::function_buffer&)+0x23
> C  [libUtil.so+0x2e75f0]  boost::function0::operator()() const+0x52
> C  [libUtil.so+0x2e6b97]  impala::Thread::SuperviseThread(std::string
> const&, std::string const&, boost::function,
> impala::Promise*)+0x2c5
> C  [libUtil.so+0x2ee48e]  void
> boost::_bi::list4,
> boost::_bi::value, boost::_bi::value ()()> >, boost::_bi::value*> >::operator() (*)(std::string const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list0>(boost::_bi::type, void
> (*&)(std::string const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list0&, int)+0xb2
> C  [libUtil.so+0x2ee3d1]  boost::_bi::bind_t const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list4,
> boost::_bi::value, boost::_bi::value ()()> >, boost::_bi::value*> > >::operator()()+0x3b
> C  [libUtil.so+0x2ee32c]
>  boost::detail::thread_data const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list4,
> boost::_bi::value, boost::_bi::value ()()> >, boost::_bi::value*> > > >::run()+0x1e
>
> I'm at git hash ff6b450  (IMPALA-4285/IMPALA-4286: Fixes for Parquet
> scanner with MT_DOP > 0), ie, the latest on master.

crash in PHJ on master

[Marcel Kornacker]

I'm seeing a crash in PartitionedHashJoinNode, is anyone experiencing the
same or know what's going on?

Here is the stack:
C  [libExec.so+0x53706c]  std::unique_ptr >::get() const+0x18
C  [libExec.so+0x54d6bc]
 impala::PartitionedHashJoinNode::NodeDebugString() const+0x296
C  [libExec.so+0x54d0bd]
 
impala::PartitionedHashJoinNode::UpdateState(impala::PartitionedHashJoinNode::HashJoinState)+0x37f
C  [libExec.so+0x53eeb5]
 impala::PartitionedHashJoinNode::Open(impala::RuntimeState*)+0x793
C  [libExec.so+0x512fde]
 impala::PartitionedAggregationNode::Open(impala::RuntimeState*)+0x4b4
C  [libRuntime.so+0x53ff0e]
 impala::PlanFragmentExecutor::OpenInternal()+0x3ae
C  [libRuntime.so+0x53fa33]  impala::PlanFragmentExecutor::Open()+0x229
C  [libService.so+0x30e7a8]
 impala::FragmentMgr::FragmentExecState::Exec()+0x66
C  [libService.so+0x31c0c8]
 impala::FragmentMgr::FragmentThread(impala::TUniqueId)+0x78
C  [libService.so+0x33863e]  boost::_mfi::mf1::operator()(impala::FragmentMgr*, impala::TUniqueId)
const+0x84
C  [libService.so+0x336c6b]  void
boost::_bi::list2,
boost::_bi::value >::operator(),
boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1&, boost::_bi::list0&, int)+0x7d
C  [libService.so+0x335557]  boost::_bi::bind_t,
boost::_bi::list2,
boost::_bi::value > >::operator()()+0x3b
C  [libService.so+0x33288c]
 boost::detail::function::void_function_obj_invoker0,
boost::_bi::list2,
boost::_bi::value > >,
void>::invoke(boost::detail::function::function_buffer&)+0x23
C  [libUtil.so+0x2e75f0]  boost::function0::operator()() const+0x52
C  [libUtil.so+0x2e6b97]  impala::Thread::SuperviseThread(std::string
const&, std::string const&, boost::function,
impala::Promise*)+0x2c5
C  [libUtil.so+0x2ee48e]  void
boost::_bi::list4,
boost::_bi::value, boost::_bi::value >, boost::_bi::value*> >::operator(),
impala::Promise*), boost::_bi::list0>(boost::_bi::type, void
(*&)(std::string const&, std::string const&, boost::function,
impala::Promise*), boost::_bi::list0&, int)+0xb2
C  [libUtil.so+0x2ee3d1]  boost::_bi::bind_t,
impala::Promise*), boost::_bi::list4,
boost::_bi::value, boost::_bi::value >, boost::_bi::value*> > >::operator()()+0x3b
C  [libUtil.so+0x2ee32c]
 boost::detail::thread_data,
impala::Promise*), boost::_bi::list4,
boost::_bi::value, boost::_bi::value >, boost::_bi::value*> > > >::run()+0x1e

I'm at git hash ff6b450  (IMPALA-4285/IMPALA-4286: Fixes for Parquet
scanner with MT_DOP > 0), ie, the latest on master.