[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-12 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/12045 )

Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..

IMPALA-7926: Fix flakiness in test_reconnect

test_reconnect launches a shell that connects to one impalad in the
minicluster then reconnects to a different impalad while checking that
the impalad's open session metric changes accordingly.

To do this, the test gets the number of open sessions at the start of
the test and then expects that the number of sessions will have
increased by 1 on the impalad that the shell is currently connected
to.

This can be a problem if there is a session left over from another
test that is still active when test_reconnect starts but exits while
it's running.

test_reconnect is already marked to run serially, so there shouldn't
be any other sessions open while it runs anyways. The solution is to
wait at the start of the test until any sessions left over from other
tests have exited.

Testing:
- Ran the test in an environment where the timing was previously
  causing it to fail almost deterministically and it now passes.

Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Reviewed-on: http://gerrit.cloudera.org:8080/12045
Reviewed-by: Impala Public Jenkins 
Tested-by: Impala Public Jenkins 
---
M tests/shell/test_shell_interactive.py
1 file changed, 17 insertions(+), 15 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Thomas Marshall

[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-12 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12045 )

Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..


Patch Set 3: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Thu, 13 Dec 2018 01:08:50 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-12 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12045 )

Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..


Patch Set 3: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Wed, 12 Dec 2018 21:10:37 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-12 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12045 )

Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3559/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Wed, 12 Dec 2018 21:10:38 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-12 Thread Michael Brown (Code Review) 
Michael Brown has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12045 )

Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..


Patch Set 2: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Michael Brown 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Wed, 12 Dec 2018 20:59:06 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-12 Thread Lars Volker (Code Review) 
Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12045 )

Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..


Patch Set 2: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Wed, 12 Dec 2018 18:47:44 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-10 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12045 )

Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/1567/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Mon, 10 Dec 2018 20:28:40 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-10 Thread Thomas Marshall (Code Review) 
Thomas Marshall has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12045 )

Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py@235
PS1, Line 235: beeswax_port=21001, be_port=22001)
> The default value for wait_for_metric_value() is 10 seconds. Will that be e
Yes, testing locally it always takes <1s


http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py@242
PS1, Line 242: # Connect to localhost:21000 (default)
> If you change get_num_open_sessions to call wait_for_metric_value(), you ca
Done


http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py@253
PS1, Line 253: assert "[%s:21001] default>" % hostname in 
p.get_result().stdout
> while you're here remove this and change get_num_open_sessions to call wait
Done



--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Mon, 10 Dec 2018 19:58:34 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-10 Thread Thomas Marshall (Code Review) 
Hello Lars Volker,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/12045

to look at the new patch set (#2).

Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..

IMPALA-7926: Fix flakiness in test_reconnect

test_reconnect launches a shell that connects to one impalad in the
minicluster then reconnects to a different impalad while checking that
the impalad's open session metric changes accordingly.

To do this, the test gets the number of open sessions at the start of
the test and then expects that the number of sessions will have
increased by 1 on the impalad that the shell is currently connected
to.

This can be a problem if there is a session left over from another
test that is still active when test_reconnect starts but exits while
it's running.

test_reconnect is already marked to run serially, so there shouldn't
be any other sessions open while it runs anyways. The solution is to
wait at the start of the test until any sessions left over from other
tests have exited.

Testing:
- Ran the test in an environment where the timing was previously
  causing it to fail almost deterministically and it now passes.

Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
---
M tests/shell/test_shell_interactive.py
1 file changed, 17 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/12045/2
--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Lars Volker

[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-06 Thread Lars Volker (Code Review) 
Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/12045 )

Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py
File tests/shell/test_shell_interactive.py:

http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py@235
PS1, Line 235: initial_impala_service.wait_for_metric_value(
The default value for wait_for_metric_value() is 10 seconds. Will that be 
enough?


http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py@242
PS1, Line 242: assert get_num_open_sessions(initial_impala_service) == 0
If you change get_num_open_sessions to call wait_for_metric_value(), you can 
just assert here without waiting above.


http://gerrit.cloudera.org:8080/#/c/12045/1/tests/shell/test_shell_interactive.py@253
PS1, Line 253: sleep(5)
while you're here remove this and change get_num_open_sessions to call 
wait_for_metric_value instead of get_metric_value()?



--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Lars Volker 
Gerrit-Comment-Date: Thu, 06 Dec 2018 23:48:52 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-7926: Fix flakiness in test reconnect

2018-12-06 Thread Thomas Marshall (Code Review) 
Thomas Marshall has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/12045


Change subject: IMPALA-7926: Fix flakiness in test_reconnect
..

IMPALA-7926: Fix flakiness in test_reconnect

test_reconnect launches a shell that connects to one impalad in the
minicluster then reconnects to a different impalad while checking that
the impalad's open session metric changes accordingly.

To do this, the test gets the number of open sessions at the start of
the test and then expects that the number of sessions will have
increased by 1 on the impalad that the shell is currently connected
to.

This can be a problem if there is a session left over from another
test that is still active when test_reconnect starts but exits while
it's running.

test_reconnect is already marked to run serially, so there shouldn't
be any other sessions open while it runs anyways. The solution is to
wait at the start of the test until any sessions left over from other
tests have exited.

Testing:
- Ran the test in an environment where the timing was previously
  causing it to fail almost deterministically and it now passes.

Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
---
M tests/shell/test_shell_interactive.py
1 file changed, 11 insertions(+), 5 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/12045/1
--
To view, visit http://gerrit.cloudera.org:8080/12045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3017ca3bf7b4e33440cffb80e9a48a63bec14434
Gerrit-Change-Number: 12045
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Marshall