[
https://issues.apache.org/jira/browse/IMPALA-14790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18075730#comment-18075730
]
ASF subversion and git services commented on IMPALA-14790:
----------------------------------------------------------
Commit 6c3324ee70090440a03d427167d4c708731c59dd in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6c3324ee7 ]
IMPALA-14790: Avoid bug in psutil 7.2.2
psutil 7.2.2 added a mode to Process.wait() to use pidfd_open() + poll()
(https://github.com/giampaolo/psutil/issues/2705), but that can trigger
another bug that doesn't have a released fix
(https://github.com/giampaolo/psutil/issues/2715).
Revert to psutil 7.2.1 to avoid this enhancement. It's the only thing
that changed on Linux according to
https://psutil.readthedocs.io/latest/changelog.html.
Change-Id: Ic63ff19200550ef1ebc0af873a7dbca9955b8929
Reviewed-on: http://gerrit.cloudera.org:8080/24224
Reviewed-by: Joe McDonnell <[email protected]>
Tested-by: Michael Smith <[email protected]>
> Various custom cluster tests fail in start-impala-cluster.py with errors in
> psutil
> ----------------------------------------------------------------------------------
>
> Key: IMPALA-14790
> URL: https://issues.apache.org/jira/browse/IMPALA-14790
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Assignee: Michael Smith
> Priority: Critical
> Fix For: Impala 5.0.0
>
>
> An upstream Jenkins nightly job hit this issue:
> {noformat}
> custom_cluster/test_custom_statestore.py:178: in
> test_subscriber_fail_on_startup_register_failure
> assert False, "cluster startup should have failed"
> E AssertionError: cluster startup should have failed
> E assert False
> self =
> <tests.custom_cluster.test_custom_statestore.TestCustomStatestore object at
> 0x7fb72d71f010>During handling of the above exception, another exception
> occurred:
> custom_cluster/test_custom_statestore.py:180: in
> test_subscriber_fail_on_startup_register_failure
> self._stop_impala_cluster()
> self =
> <tests.custom_cluster.test_custom_statestore.TestCustomStatestore object at
> 0x7fb72d71f010>
> common/custom_cluster_test_suite.py:499: in _stop_impala_cluster
> check_call([os.path.join(IMPALA_HOME, 'bin/start-impala-cluster.py'),
> '--kill_only'])
> cls = <class
> 'tests.custom_cluster.test_custom_statestore.TestCustomStatestore'>
> ../toolchain/toolchain-packages-gcc10.4.0/python-3.11.14/lib/python3.11/subprocess.py:413:
> in check_call
> raise CalledProcessError(retcode, cmd)
> E subprocess.CalledProcessError: Command
> '['/home/ubuntu/Impala/bin/start-impala-cluster.py', '--kill_only']' returned
> non-zero exit status 1.
> cmd = ['/home/ubuntu/Impala/bin/start-impala-cluster.py',
> '--kill_only']
> kwargs = {}
> popenargs = (['/home/ubuntu/Impala/bin/start-impala-cluster.py',
> '--kill_only'],)
> retcode = 1{noformat}
> There is additional output pointing to psutil:
> {noformat}
> Traceback (most recent call last):
> File "/home/ubuntu/Impala/bin/start-impala-cluster.py", line 1230, in
> <module>
> cluster_ops.kill_all_daemons(force=options.force_kill)
> File "/home/ubuntu/Impala/bin/start-impala-cluster.py", line 790, in
> kill_all_daemons
> kill_matching_processes(["catalogd", "impalad", "statestored",
> "admissiond"], force)
> File "/home/ubuntu/Impala/bin/start-impala-cluster.py", line 307, in
> kill_matching_processes
> process.wait(KILL_TIMEOUT_IN_SECONDS)
> File
> "/home/ubuntu/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.11/site-packages/psutil/__init__.py",
> line 1373, in wait
> self._exitcode = self._proc.wait(timeout)
> ^^^^^^^^^^^^^^^^^^^^^^^^
> File
> "/home/ubuntu/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.11/site-packages/psutil/_pslinux.py",
> line 1593, in wrapper
> return fun(self, *args, **kwargs)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^
> File
> "/home/ubuntu/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.11/site-packages/psutil/_pslinux.py",
> line 1846, in wait
> return _psposix.wait_pid(self.pid, timeout)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File
> "/home/ubuntu/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.11/site-packages/psutil/_psposix.py",
> line 304, in wait_pid
> return wait_pid_pidfd_open(pid, timeout)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File
> "/home/ubuntu/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.11/site-packages/psutil/_psposix.py",
> line 182, in wait_pid_pidfd_open
> pidfd = os.pidfd_open(pid, 0)
> ^^^^^^^^^^^^^^^^^^^^^
> OSError: [Errno 22] Invalid argument{noformat}
> This may be due to our recent upgrade of psutil as part of the Python 3.11
> change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
