Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package python-jupyter-server for openSUSE:Factory checked in at 2022-07-26 19:44:17 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-jupyter-server (Old) and /work/SRC/openSUSE:Factory/.python-jupyter-server.new.1533 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-jupyter-server" Tue Jul 26 19:44:17 2022 rev:25 rq:990956 version:1.18.1 Changes: -------- --- /work/SRC/openSUSE:Factory/python-jupyter-server/python-jupyter-server.changes 2022-06-16 18:21:28.960188371 +0200 +++ /work/SRC/openSUSE:Factory/.python-jupyter-server.new.1533/python-jupyter-server.changes 2022-07-26 19:44:42.753133556 +0200 @@ -1,0 +2,20 @@ +Mon Jul 25 00:12:10 UTC 2022 - Arun Persaud <a...@gmx.de> + +- update to version 1.18.1: + * Bugs fixed + + Notify ChannelQueue that the response router thread is finishing + #896 (@CiprianAnton) + + Make ChannelQueue.get_msg true async #892 (@CiprianAnton) + +- changes from version 1.18.0: + * Enhancements made + + Show import error when faiing to load an extension #878 (@minrk) + * Bugs fixed + + Fix gateway kernel shutdown #874 (@kevin-bates) + * Maintenance and upkeep improvements + + suppress tornado deprecation warnings #882 (@minrk) + + Normalize os_path #886 (@martinRenou) + + Fix lint #867 (@blink1073) + + Fix sphinx 5.0 support #865 (@blink1073) + +------------------------------------------------------------------- Old: ---- jupyter_server-1.17.1.tar.gz New: ---- jupyter_server-1.18.1.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-jupyter-server.spec ++++++ --- /var/tmp/diff_new_pack.qirGis/_old 2022-07-26 19:44:43.129075463 +0200 +++ /var/tmp/diff_new_pack.qirGis/_new 2022-07-26 19:44:43.133074845 +0200 @@ -31,13 +31,13 @@ %bcond_with libalternatives %endif Name: python-jupyter-server%{psuffix} -Version: 1.17.1 +Version: 1.18.1 Release: 0 Summary: The backend to Jupyter web applications License: BSD-3-Clause Group: Development/Languages/Python URL: https://jupyter-server.readthedocs.io -Source: https://files.pythonhosted.org/packages/source/j/jupyter-server/jupyter_server-%{version}.tar.gz +Source: https://files.pythonhosted.org/packages/source/j/jupyter_server/jupyter_server-%{version}.tar.gz BuildRequires: %{python_module base >= 3.7} BuildRequires: %{python_module jupyter_packaging} BuildRequires: %{python_module setuptools} ++++++ jupyter_server-1.17.1.tar.gz -> jupyter_server-1.18.1.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/CHANGELOG.md new/jupyter_server-1.18.1/CHANGELOG.md --- old/jupyter_server-1.17.1/CHANGELOG.md 2022-06-07 19:07:21.000000000 +0200 +++ new/jupyter_server-1.18.1/CHANGELOG.md 2022-07-05 22:22:43.000000000 +0200 @@ -4,6 +4,58 @@ <!-- <START NEW CHANGELOG ENTRY> --> +## 1.18.1 + +([Full Changelog](https://github.com/jupyter-server/jupyter_server/compare/v1.18.0...65d779a12b6e4123de0767de6547e6cdf2d5e7e9)) + +### Bugs fixed + +- Notify ChannelQueue that the response router thread is finishing [#896](https://github.com/jupyter-server/jupyter_server/pull/896) ([@CiprianAnton](https://github.com/CiprianAnton)) +- Make ChannelQueue.get_msg true async [#892](https://github.com/jupyter-server/jupyter_server/pull/892) ([@CiprianAnton](https://github.com/CiprianAnton)) + +### Contributors to this release + +([GitHub contributors page for this release](https://github.com/jupyter-server/jupyter_server/graphs/contributors?from=2022-06-23&to=2022-07-05&type=c)) + +[@blink1073](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Ablink1073+updated%3A2022-06-23..2022-07-05&type=Issues) | [@Carreau](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3ACarreau+updated%3A2022-06-23..2022-07-05&type=Issues) | [@codecov-commenter](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Acodecov-commenter+updated%3A2022-06-23..2022-07-05&type=Issues) | [@davidbrochart](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Adavidbrochart+updated%3A2022-06-23..2022-07-05&type=Issues) | [@echarles](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Aecharles+updated%3A2022-06-23..2022-07-05&type=Issues) | [@kevin-bates](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Akevin-bates+updated%3A2022-06-23..2022-07-05&type=Issues) | [@meeseeksmachine](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_se rver+involves%3Ameeseeksmachine+updated%3A2022-06-23..2022-07-05&type=Issues) + +<!-- <END NEW CHANGELOG ENTRY> --> + +## 1.18.0 + +([Full Changelog](https://github.com/jupyter-server/jupyter_server/compare/v1.17.1...a625db91efc4927c4b6fed4bf67ab995e479c36d)) + +### Enhancements made + +- Show import error when faiing to load an extension [#878](https://github.com/jupyter-server/jupyter_server/pull/878) ([@minrk](https://github.com/minrk)) + +### Bugs fixed + +- Fix gateway kernel shutdown [#874](https://github.com/jupyter-server/jupyter_server/pull/874) ([@kevin-bates](https://github.com/kevin-bates)) + +### Maintenance and upkeep improvements + +- suppress tornado deprecation warnings [#882](https://github.com/jupyter-server/jupyter_server/pull/882) ([@minrk](https://github.com/minrk)) +- Normalize os_path [#886](https://github.com/jupyter-server/jupyter_server/pull/886) ([@martinRenou](https://github.com/martinRenou)) +- Fix lint [#867](https://github.com/jupyter-server/jupyter_server/pull/867) ([@blink1073](https://github.com/blink1073)) +- Fix sphinx 5.0 support [#865](https://github.com/jupyter-server/jupyter_server/pull/865) ([@blink1073](https://github.com/blink1073)) + +### Documentation improvements + +- Add changelog entry for 1.17.1 [#871](https://github.com/jupyter-server/jupyter_server/pull/871) ([@blink1073](https://github.com/blink1073)) + +### Contributors to this release + +([GitHub contributors page for this release](https://github.com/jupyter-server/jupyter_server/graphs/contributors?from=2022-06-07&to=2022-06-23&type=c)) + +[@blink1073](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Ablink1073+updated%3A2022-06-07..2022-06-23&type=Issues) | [@codecov-commenter](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Acodecov-commenter+updated%3A2022-06-07..2022-06-23&type=Issues) | [@davidbrochart](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Adavidbrochart+updated%3A2022-06-07..2022-06-23&type=Issues) | [@kevin-bates](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Akevin-bates+updated%3A2022-06-07..2022-06-23&type=Issues) | [@meeseeksmachine](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Ameeseeksmachine+updated%3A2022-06-07..2022-06-23&type=Issues) + +## 1.17.1 + +([Full Changelog](https://github.com/jupyter-server/jupyter_server/compare/v1.17.0...v1.17.1) + +- Address security advisory [GHSA-q874-g24w-4q9g](https://github.com/jupyter-server/jupyter_server/security/advisories/GHSA-q874-g24w-4q9g). + ## 1.17.0 ([Full Changelog](https://github.com/jupyter-server/jupyter_server/compare/v1.16.0...2b296099777d50aa86f67faf94d5cbfde906b169)) @@ -28,8 +80,6 @@ [@blink1073](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Ablink1073+updated%3A2022-03-29..2022-04-27&type=Issues) | [@codecov-commenter](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Acodecov-commenter+updated%3A2022-03-29..2022-04-27&type=Issues) | [@davidbrochart](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Adavidbrochart+updated%3A2022-03-29..2022-04-27&type=Issues) | [@echarles](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Aecharles+updated%3A2022-03-29..2022-04-27&type=Issues) | [@kevin-bates](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Akevin-bates+updated%3A2022-03-29..2022-04-27&type=Issues) | [@meeseeksdev](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3Ameeseeksdev+updated%3A2022-03-29..2022-04-27&type=Issues) | [@meeseeksmachine](https://github.com/search?q=repo%3Ajupyter-server%2Fju pyter_server+involves%3Ameeseeksmachine+updated%3A2022-03-29..2022-04-27&type=Issues) | [@Wh1isper](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3AWh1isper+updated%3A2022-03-29..2022-04-27&type=Issues) | [@Zsailer](https://github.com/search?q=repo%3Ajupyter-server%2Fjupyter_server+involves%3AZsailer+updated%3A2022-03-29..2022-04-27&type=Issues) -<!-- <END NEW CHANGELOG ENTRY> --> - ## 1.16.0 ([Full Changelog](https://github.com/jupyter-server/jupyter_server/compare/v1.15.6...d32b887ae2c3b77fe3ae67ba79c3d3c6713c0d8a)) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/PKG-INFO new/jupyter_server-1.18.1/PKG-INFO --- old/jupyter_server-1.17.1/PKG-INFO 2022-06-07 19:16:36.606193500 +0200 +++ new/jupyter_server-1.18.1/PKG-INFO 2022-07-05 22:23:12.652353500 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 2.1 Name: jupyter_server -Version: 1.17.1 +Version: 1.18.1 Summary: The backend???i.e. core services, APIs, and REST endpoints???to Jupyter web applications. Home-page: https://jupyter-server.readthedocs.io Author: Jupyter Development Team diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/docs/source/conf.py new/jupyter_server-1.18.1/docs/source/conf.py --- old/jupyter_server-1.17.1/docs/source/conf.py 2022-06-07 19:16:17.000000000 +0200 +++ new/jupyter_server-1.18.1/docs/source/conf.py 2022-07-05 22:22:58.000000000 +0200 @@ -107,7 +107,7 @@ # |version| and |release|, also used in various other places throughout the # built documents. # -__version__ = "1.17.1" +__version__ = "1.18.1" # The short X.Y version. version_parsed = parse_version(__version__) version = f"{version_parsed.major}.{version_parsed.minor}" @@ -117,7 +117,7 @@ # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. -language = None +language = "en" # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/jupyter_server/_version.py new/jupyter_server-1.18.1/jupyter_server/_version.py --- old/jupyter_server-1.17.1/jupyter_server/_version.py 2022-06-07 19:16:17.000000000 +0200 +++ new/jupyter_server-1.18.1/jupyter_server/_version.py 2022-07-05 22:22:58.000000000 +0200 @@ -2,5 +2,5 @@ store the current version info of the server. """ -version_info = (1, 17, 1, "", "") +version_info = (1, 18, 1, "", "") __version__ = ".".join(map(str, version_info[:3])) + "".join(version_info[3:]) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/jupyter_server/extension/manager.py new/jupyter_server-1.18.1/jupyter_server/extension/manager.py --- old/jupyter_server-1.17.1/jupyter_server/extension/manager.py 2022-06-07 19:07:21.000000000 +0200 +++ new/jupyter_server-1.18.1/jupyter_server/extension/manager.py 2022-07-05 22:22:43.000000000 +0200 @@ -175,10 +175,10 @@ self._extension_points = {} try: self._module, self._metadata = get_metadata(name) - except ImportError: + except ImportError as e: raise ExtensionModuleNotFound( - "The module '{name}' could not be found. Are you " - "sure the extension is installed?".format(name=name) + "The module '{name}' could not be found ({e}). Are you " + "sure the extension is installed?".format(name=name, e=e) ) # Create extension point interfaces for each extension path. for m in self._metadata: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/jupyter_server/gateway/managers.py new/jupyter_server-1.18.1/jupyter_server/gateway/managers.py --- old/jupyter_server-1.17.1/jupyter_server/gateway/managers.py 2022-06-07 19:07:21.000000000 +0200 +++ new/jupyter_server-1.18.1/jupyter_server/gateway/managers.py 2022-07-05 22:22:43.000000000 +0200 @@ -1,11 +1,13 @@ # Copyright (c) Jupyter Development Team. # Distributed under the terms of the Modified BSD License. +import asyncio import datetime import json import os from logging import Logger -from queue import Queue +from queue import Empty, Queue from threading import Thread +from time import monotonic from typing import Any, Dict, Optional import websocket @@ -92,7 +94,7 @@ The uuid of the kernel. """ model = None - km = self.get_kernel(kernel_id) + km = self.get_kernel(str(kernel_id)) if km: model = km.kernel return model @@ -166,13 +168,14 @@ async def shutdown_all(self, now=False): """Shutdown all kernels.""" - for kernel_id in self._kernels: + kids = list(self._kernels) + for kernel_id in kids: km = self.get_kernel(kernel_id) await km.shutdown_kernel(now=now) self.remove_kernel(kernel_id) async def cull_kernels(self): - """Override cull_kernels so we can be sure their state is current.""" + """Override cull_kernels, so we can be sure their state is current.""" await self.list_kernels() await super().cull_kernels() @@ -295,7 +298,7 @@ kernel_manager = Instance("jupyter_server.gateway.managers.GatewayMappingKernelManager") async def kernel_culled(self, kernel_id): - """Checks if the kernel is still considered alive and returns true if its not found.""" + """Checks if the kernel is still considered alive and returns true if it's not found.""" kernel = None try: km = self.kernel_manager.get_kernel(kernel_id) @@ -387,7 +390,7 @@ if isinstance(self.parent, AsyncMappingKernelManager): # Update connections only if there's a mapping kernel manager parent for # this kernel manager. The current kernel manager instance may not have - # an parent instance if, say, a server extension is using another application + # a parent instance if, say, a server extension is using another application # (e.g., papermill) that uses a KernelManager instance directly. self.parent._kernel_connections[self.kernel_id] = int(model["connections"]) @@ -448,8 +451,14 @@ if self.has_kernel: self.log.debug("Request shutdown kernel at: %s", self.kernel_url) - response = await gateway_request(self.kernel_url, method="DELETE") - self.log.debug("Shutdown kernel response: %d %s", response.code, response.reason) + try: + response = await gateway_request(self.kernel_url, method="DELETE") + self.log.debug("Shutdown kernel response: %d %s", response.code, response.reason) + except web.HTTPError as error: + if error.status_code == 404: + self.log.debug("Shutdown kernel response: kernel not found (ignored)") + else: + raise async def restart_kernel(self, **kw): """Restarts a kernel via HTTP.""" @@ -489,16 +498,35 @@ class ChannelQueue(Queue): channel_name: Optional[str] = None + response_router_finished: bool def __init__(self, channel_name: str, channel_socket: websocket.WebSocket, log: Logger): super().__init__() self.channel_name = channel_name self.channel_socket = channel_socket self.log = log + self.response_router_finished = False + + async def _async_get(self, timeout=None): + if timeout is None: + timeout = float("inf") + elif timeout < 0: + raise ValueError("'timeout' must be a non-negative number") + end_time = monotonic() + timeout + + while True: + try: + return self.get(block=False) + except Empty: + if self.response_router_finished: + raise RuntimeError("Response router had finished") + if monotonic() > end_time: + raise + await asyncio.sleep(0) async def get_msg(self, *args: Any, **kwargs: Any) -> dict: timeout = kwargs.get("timeout", 1) - msg = self.get(timeout=timeout) + msg = await self._async_get(timeout=timeout) self.log.debug( "Received message on channel: {}, msg_id: {}, msg_type: {}".format( self.channel_name, msg["msg_id"], msg["msg_type"] if msg else "null" @@ -518,7 +546,7 @@ @staticmethod def serialize_datetime(dt): - if isinstance(dt, (datetime.datetime)): + if isinstance(dt, datetime.datetime): return dt.timestamp() return None @@ -573,19 +601,21 @@ # flag for whether execute requests should be allowed to call raw_input: allow_stdin = False - _channels_stopped = False - _channel_queues: Optional[dict] = {} + _channels_stopped: bool + _channel_queues: Optional[Dict[str, ChannelQueue]] _control_channel: Optional[ChannelQueue] _hb_channel: Optional[ChannelQueue] _stdin_channel: Optional[ChannelQueue] _iopub_channel: Optional[ChannelQueue] _shell_channel: Optional[ChannelQueue] - def __init__(self, **kwargs): + def __init__(self, kernel_id, **kwargs): super().__init__(**kwargs) - self.kernel_id = kwargs["kernel_id"] + self.kernel_id = kernel_id self.channel_socket: Optional[websocket.WebSocket] = None self.response_router: Optional[Thread] = None + self._channels_stopped = False + self._channel_queues = {} # -------------------------------------------------------------------------- # Channel management methods @@ -595,7 +625,7 @@ """Starts the channels for this kernel. For this class, we establish a websocket connection to the destination - and setup the channel-based queues on which applicable messages will + and set up the channel-based queues on which applicable messages will be posted. """ @@ -606,10 +636,11 @@ "channels", ) # Gather cert info in case where ssl is desired... - ssl_options = {} - ssl_options["ca_certs"] = GatewayClient.instance().ca_certs - ssl_options["certfile"] = GatewayClient.instance().client_cert - ssl_options["keyfile"] = GatewayClient.instance().client_key + ssl_options = { + "ca_certs": GatewayClient.instance().ca_certs, + "certfile": GatewayClient.instance().client_cert, + "keyfile": GatewayClient.instance().client_key, + } self.channel_socket = websocket.create_connection( ws_url, @@ -617,13 +648,14 @@ enable_multithread=True, sslopt=ssl_options, ) - self.response_router = Thread(target=self._route_responses) - self.response_router.start() await ensure_async( super().start_channels(shell=shell, iopub=iopub, stdin=stdin, hb=hb, control=control) ) + self.response_router = Thread(target=self._route_responses) + self.response_router.start() + def stop_channels(self): """Stops all the running channels for this kernel. @@ -720,12 +752,17 @@ self._channel_queues[channel].put_nowait(response_message) except websocket.WebSocketConnectionClosedException: - pass # websocket closure most likely due to shutdown + pass # websocket closure most likely due to shut down except BaseException as be: if not self._channels_stopped: self.log.warning(f"Unexpected exception encountered ({be})") + # Notify channel queues that this thread had finished and no more messages are being received + assert self._channel_queues is not None + for channel_queue in self._channel_queues.values(): + channel_queue.response_router_finished = True + self.log.debug("Response router thread exiting...") diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/jupyter_server/utils.py new/jupyter_server-1.18.1/jupyter_server/utils.py --- old/jupyter_server-1.17.1/jupyter_server/utils.py 2022-06-07 19:02:17.000000000 +0200 +++ new/jupyter_server-1.18.1/jupyter_server/utils.py 2022-07-05 22:22:43.000000000 +0200 @@ -111,7 +111,7 @@ parts = path.strip("/").split("/") parts = [p for p in parts if p != ""] # remove duplicate splits path = os.path.join(root, *parts) - return path + return os.path.normpath(path) def to_api_path(os_path, root=""): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/jupyter_server.egg-info/PKG-INFO new/jupyter_server-1.18.1/jupyter_server.egg-info/PKG-INFO --- old/jupyter_server-1.17.1/jupyter_server.egg-info/PKG-INFO 2022-06-07 19:16:36.000000000 +0200 +++ new/jupyter_server-1.18.1/jupyter_server.egg-info/PKG-INFO 2022-07-05 22:23:12.000000000 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 2.1 Name: jupyter-server -Version: 1.17.1 +Version: 1.18.1 Summary: The backend???i.e. core services, APIs, and REST endpoints???to Jupyter web applications. Home-page: https://jupyter-server.readthedocs.io Author: Jupyter Development Team diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/pyproject.toml new/jupyter_server-1.18.1/pyproject.toml --- old/jupyter_server-1.17.1/pyproject.toml 2022-06-07 19:16:17.000000000 +0200 +++ new/jupyter_server-1.18.1/pyproject.toml 2022-07-05 22:22:58.000000000 +0200 @@ -18,7 +18,9 @@ # timeout_method = "thread" filterwarnings = [ "error", - "ignore:There is no current event loop:DeprecationWarning", + "module:make_current is deprecated:DeprecationWarning", + "module:clear_current is deprecated:DeprecationWarning", + "module:There is no current event loop:DeprecationWarning", "ignore:Passing a schema to Validator.iter_errors:DeprecationWarning", "ignore:unclosed <socket.socket:ResourceWarning", "ignore:unclosed event loop:ResourceWarning", @@ -32,7 +34,7 @@ post-version-spec = "dev" [tool.tbump.version] -current = "1.17.1" +current = "1.18.1" regex = ''' (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+) ((?P<channel>a|b|rc|.dev)(?P<release>\d+))? diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/tests/services/contents/test_api.py new/jupyter_server-1.18.1/tests/services/contents/test_api.py --- old/jupyter_server-1.17.1/tests/services/contents/test_api.py 2022-06-07 19:07:47.000000000 +0200 +++ new/jupyter_server-1.18.1/tests/services/contents/test_api.py 2022-07-05 22:22:43.000000000 +0200 @@ -230,19 +230,20 @@ ) assert expected_http_error(e, 400) + @pytest.mark.skipif(sys.platform == "win32", reason="Disabled retrieving hidden files on Windows") async def test_get_404_hidden(jp_fetch, contents, contents_dir): # Create text files - hidden_dir = contents_dir / '.hidden' + hidden_dir = contents_dir / ".hidden" hidden_dir.mkdir(parents=True, exist_ok=True) - txt = f"visible text file in hidden dir" - txtname = hidden_dir.joinpath(f"visible.txt") + txt = "visible text file in hidden dir" + txtname = hidden_dir.joinpath("visible.txt") txtname.write_text(txt, encoding="utf-8") - txt2 = f"hidden text file" - txtname2 = contents_dir.joinpath(f".hidden.txt") + txt2 = "hidden text file" + txtname2 = contents_dir.joinpath(".hidden.txt") txtname2.write_text(txt2, encoding="utf-8") - + with pytest.raises(tornado.httpclient.HTTPClientError) as e: await jp_fetch( "api", @@ -261,6 +262,7 @@ ) assert expected_http_error(e, 404) + @pytest.mark.parametrize("path,name", dirs) async def test_get_binary_file_contents(jp_fetch, contents, path, name): blobname = name + ".blob" @@ -441,48 +443,38 @@ @pytest.mark.skipif(sys.platform == "win32", reason="Disabled uploading hidden files on Windows") async def test_upload_txt_hidden(jp_fetch, contents, contents_dir): with pytest.raises(tornado.httpclient.HTTPClientError) as e: - body = '??nicode t??xt' + body = "??nicode t??xt" model = { - 'content' : body, - 'format' : 'text', - 'type' : 'file', + "content": body, + "format": "text", + "type": "file", } - path = '.hidden/Upload t??st.txt' + path = ".hidden/Upload t??st.txt" await jp_fetch("api", "contents", path, method="PUT", body=json.dumps(model)) assert expected_http_error(e, 400) with pytest.raises(tornado.httpclient.HTTPClientError) as e: - body = '??nicode t??xt' - model = { - 'content' : body, - 'format' : 'text', - 'type' : 'file', - 'path': '.hidden/test.txt' - } - path = 'Upload t??st.txt' + body = "??nicode t??xt" + model = {"content": body, "format": "text", "type": "file", "path": ".hidden/test.txt"} + path = "Upload t??st.txt" await jp_fetch("api", "contents", path, method="PUT", body=json.dumps(model)) assert expected_http_error(e, 400) with pytest.raises(tornado.httpclient.HTTPClientError) as e: - body = '??nicode t??xt' + body = "??nicode t??xt" model = { - 'content' : body, - 'format' : 'text', - 'type' : 'file', + "content": body, + "format": "text", + "type": "file", } - path = '.hidden.txt' + path = ".hidden.txt" await jp_fetch("api", "contents", path, method="PUT", body=json.dumps(model)) assert expected_http_error(e, 400) with pytest.raises(tornado.httpclient.HTTPClientError) as e: - body = '??nicode t??xt' - model = { - 'content' : body, - 'format' : 'text', - 'type' : 'file', - 'path': '.hidden.txt' - } - path = 'Upload t??st.txt' + body = "??nicode t??xt" + model = {"content": body, "format": "text", "type": "file", "path": ".hidden.txt"} + path = "Upload t??st.txt" await jp_fetch("api", "contents", path, method="PUT", body=json.dumps(model)) assert expected_http_error(e, 400) @@ -581,7 +573,11 @@ @pytest.mark.skipif(sys.platform == "win32", reason="Disabled copying hidden files on Windows") -async def test_copy_put_400_hidden(jp_fetch, contents, contents_dir,): +async def test_copy_put_400_hidden( + jp_fetch, + contents, + contents_dir, +): with pytest.raises(tornado.httpclient.HTTPClientError) as e: await jp_fetch( "api", @@ -591,7 +587,7 @@ body=json.dumps({"copy_from": "new.txt"}), ) assert expected_http_error(e, 400) - + with pytest.raises(tornado.httpclient.HTTPClientError) as e: await jp_fetch( "api", @@ -601,7 +597,7 @@ body=json.dumps({"copy_from": ".hidden/new.txt"}), ) assert expected_http_error(e, 400) - + with pytest.raises(tornado.httpclient.HTTPClientError) as e: await jp_fetch( "api", @@ -611,7 +607,7 @@ body=json.dumps({"copy_from": "new.txt"}), ) assert expected_http_error(e, 400) - + with pytest.raises(tornado.httpclient.HTTPClientError) as e: await jp_fetch( "api", @@ -636,16 +632,20 @@ @pytest.mark.skipif(sys.platform == "win32", reason="Disabled copying hidden files on Windows") -async def test_copy_400_hidden(jp_fetch, contents, contents_dir,): +async def test_copy_400_hidden( + jp_fetch, + contents, + contents_dir, +): # Create text files - hidden_dir = contents_dir / '.hidden' + hidden_dir = contents_dir / ".hidden" hidden_dir.mkdir(parents=True, exist_ok=True) - txt = f"visible text file in hidden dir" - txtname = hidden_dir.joinpath(f"new.txt") + txt = "visible text file in hidden dir" + txtname = hidden_dir.joinpath("new.txt") txtname.write_text(txt, encoding="utf-8") - paths = ['new.txt', '.hidden.txt'] + paths = ["new.txt", ".hidden.txt"] for name in paths: txt = f"{name} text file" txtname = contents_dir.joinpath(f"{name}.txt") @@ -660,7 +660,7 @@ body=json.dumps({"copy_from": "new.txt"}), ) assert expected_http_error(e, 400) - + with pytest.raises(tornado.httpclient.HTTPClientError) as e: await jp_fetch( "api", @@ -689,7 +689,7 @@ method="POST", body=json.dumps({"copy_from": ".hidden.txt"}), ) - assert expected_http_error(e, 400) + assert expected_http_error(e, 400) @pytest.mark.parametrize("path,name", dirs) @@ -729,20 +729,22 @@ await jp_fetch("api", "contents", "?? b", method="GET") assert expected_http_error(e, 404) + @pytest.mark.skipif(sys.platform == "win32", reason="Disabled deleting hidden dirs on Windows") async def test_delete_hidden_dir(jp_fetch, contents): with pytest.raises(tornado.httpclient.HTTPClientError) as e: await jp_fetch("api", "contents", ".hidden", method="DELETE") assert expected_http_error(e, 400) + @pytest.mark.skipif(sys.platform == "win32", reason="Disabled deleting hidden dirs on Windows") async def test_delete_hidden_file(jp_fetch, contents): - #Test deleting file in a hidden directory + # Test deleting file in a hidden directory with pytest.raises(tornado.httpclient.HTTPClientError) as e: await jp_fetch("api", "contents", ".hidden/test.txt", method="DELETE") assert expected_http_error(e, 400) - #Test deleting a hidden file + # Test deleting a hidden file with pytest.raises(tornado.httpclient.HTTPClientError) as e: await jp_fetch("api", "contents", ".hidden.txt", method="DELETE") assert expected_http_error(e, 400) @@ -778,11 +780,12 @@ assert "z.ipynb" in nbnames assert "a.ipynb" not in nbnames + @pytest.mark.skipif(sys.platform == "win32", reason="Disabled copying hidden files on Windows") async def test_rename_400_hidden(jp_fetch, jp_base_url, contents, contents_dir): with pytest.raises(tornado.httpclient.HTTPClientError) as e: - old_path = '.hidden/old.txt' - new_path = 'new.txt' + old_path = ".hidden/old.txt" + new_path = "new.txt" # Rename the file r = await jp_fetch( "api", @@ -792,10 +795,10 @@ body=json.dumps({"path": new_path}), ) assert expected_http_error(e, 400) - + with pytest.raises(tornado.httpclient.HTTPClientError) as e: - old_path = 'old.txt' - new_path = '.hidden/new.txt' + old_path = "old.txt" + new_path = ".hidden/new.txt" # Rename the file r = await jp_fetch( "api", @@ -805,10 +808,10 @@ body=json.dumps({"path": new_path}), ) assert expected_http_error(e, 400) - + with pytest.raises(tornado.httpclient.HTTPClientError) as e: - old_path = '.hidden.txt' - new_path = 'new.txt' + old_path = ".hidden.txt" + new_path = "new.txt" # Rename the file r = await jp_fetch( "api", @@ -820,8 +823,8 @@ assert expected_http_error(e, 400) with pytest.raises(tornado.httpclient.HTTPClientError) as e: - old_path = 'old.txt' - new_path = '.hidden.txt' + old_path = "old.txt" + new_path = ".hidden.txt" # Rename the file r = await jp_fetch( "api", diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/tests/services/contents/test_manager.py new/jupyter_server-1.18.1/tests/services/contents/test_manager.py --- old/jupyter_server-1.17.1/tests/services/contents/test_manager.py 2022-06-07 19:07:47.000000000 +0200 +++ new/jupyter_server-1.18.1/tests/services/contents/test_manager.py 2022-07-05 22:22:43.000000000 +0200 @@ -259,80 +259,81 @@ except HTTPError as e: assert e.status_code == 403 -@pytest.mark.skipif(sys.platform.startswith('win'), reason="Can't test hidden files on Windows") + +@pytest.mark.skipif(sys.platform.startswith("win"), reason="Can't test hidden files on Windows") async def test_400(jp_file_contents_manager_class, tmp_path): - #Test Delete behavior - #Test delete of file in hidden directory + # Test Delete behavior + # Test delete of file in hidden directory with pytest.raises(HTTPError) as excinfo: td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) - hidden_dir = '.hidden' - file_in_hidden_path = os.path.join(hidden_dir,'visible.txt') + hidden_dir = ".hidden" + file_in_hidden_path = os.path.join(hidden_dir, "visible.txt") _make_dir(cm, hidden_dir) model = await ensure_async(cm.new(path=file_in_hidden_path)) - os_path = cm._get_os_path(model['path']) + os_path = cm._get_os_path(model["path"]) try: result = await ensure_async(cm.delete_file(os_path)) except HTTPError as e: assert e.status_code == 400 - #Test delete hidden file in visible directory + # Test delete hidden file in visible directory with pytest.raises(HTTPError) as excinfo: td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) - hidden_dir = 'visible' - file_in_hidden_path = os.path.join(hidden_dir,'.hidden.txt') + hidden_dir = "visible" + file_in_hidden_path = os.path.join(hidden_dir, ".hidden.txt") _make_dir(cm, hidden_dir) model = await ensure_async(cm.new(path=file_in_hidden_path)) - os_path = cm._get_os_path(model['path']) + os_path = cm._get_os_path(model["path"]) try: result = await ensure_async(cm.delete_file(os_path)) except HTTPError as e: assert e.status_code == 400 - #Test Save behavior - #Test save of file in hidden directory + # Test Save behavior + # Test save of file in hidden directory with pytest.raises(HTTPError) as excinfo: td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) - hidden_dir = '.hidden' - file_in_hidden_path = os.path.join(hidden_dir,'visible.txt') + hidden_dir = ".hidden" + file_in_hidden_path = os.path.join(hidden_dir, "visible.txt") _make_dir(cm, hidden_dir) model = await ensure_async(cm.new(path=file_in_hidden_path)) - os_path = cm._get_os_path(model['path']) + os_path = cm._get_os_path(model["path"]) try: - result = await ensure_async(cm.save(model,path=os_path)) + result = await ensure_async(cm.save(model, path=os_path)) except HTTPError as e: assert e.status_code == 400 - #Test save hidden file in visible directory + # Test save hidden file in visible directory with pytest.raises(HTTPError) as excinfo: td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) - hidden_dir = 'visible' - file_in_hidden_path = os.path.join(hidden_dir,'.hidden.txt') + hidden_dir = "visible" + file_in_hidden_path = os.path.join(hidden_dir, ".hidden.txt") _make_dir(cm, hidden_dir) model = await ensure_async(cm.new(path=file_in_hidden_path)) - os_path = cm._get_os_path(model['path']) + os_path = cm._get_os_path(model["path"]) try: - result = await ensure_async(cm.save(model,path=os_path)) + result = await ensure_async(cm.save(model, path=os_path)) except HTTPError as e: assert e.status_code == 400 - #Test rename behavior - #Test rename with source file in hidden directory + # Test rename behavior + # Test rename with source file in hidden directory with pytest.raises(HTTPError) as excinfo: td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) - hidden_dir = '.hidden' - file_in_hidden_path = os.path.join(hidden_dir,'visible.txt') + hidden_dir = ".hidden" + file_in_hidden_path = os.path.join(hidden_dir, "visible.txt") _make_dir(cm, hidden_dir) model = await ensure_async(cm.new(path=file_in_hidden_path)) - old_path = cm._get_os_path(model['path']) + old_path = cm._get_os_path(model["path"]) new_path = "new.txt" try: @@ -340,15 +341,15 @@ except HTTPError as e: assert e.status_code == 400 - #Test rename of dest file in hidden directory + # Test rename of dest file in hidden directory with pytest.raises(HTTPError) as excinfo: td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) - hidden_dir = '.hidden' - file_in_hidden_path = os.path.join(hidden_dir,'visible.txt') + hidden_dir = ".hidden" + file_in_hidden_path = os.path.join(hidden_dir, "visible.txt") _make_dir(cm, hidden_dir) model = await ensure_async(cm.new(path=file_in_hidden_path)) - new_path = cm._get_os_path(model['path']) + new_path = cm._get_os_path(model["path"]) old_path = "old.txt" try: @@ -356,15 +357,15 @@ except HTTPError as e: assert e.status_code == 400 - #Test rename with hidden source file in visible directory + # Test rename with hidden source file in visible directory with pytest.raises(HTTPError) as excinfo: td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) - hidden_dir = 'visible' - file_in_hidden_path = os.path.join(hidden_dir,'.hidden.txt') + hidden_dir = "visible" + file_in_hidden_path = os.path.join(hidden_dir, ".hidden.txt") _make_dir(cm, hidden_dir) model = await ensure_async(cm.new(path=file_in_hidden_path)) - old_path = cm._get_os_path(model['path']) + old_path = cm._get_os_path(model["path"]) new_path = "new.txt" try: @@ -372,15 +373,15 @@ except HTTPError as e: assert e.status_code == 400 - #Test rename with hidden dest file in visible directory + # Test rename with hidden dest file in visible directory with pytest.raises(HTTPError) as excinfo: td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) - hidden_dir = 'visible' - file_in_hidden_path = os.path.join(hidden_dir,'.hidden.txt') + hidden_dir = "visible" + file_in_hidden_path = os.path.join(hidden_dir, ".hidden.txt") _make_dir(cm, hidden_dir) model = await ensure_async(cm.new(path=file_in_hidden_path)) - new_path = cm._get_os_path(model['path']) + new_path = cm._get_os_path(model["path"]) old_path = "old.txt" try: @@ -388,38 +389,40 @@ except HTTPError as e: assert e.status_code == 400 -@pytest.mark.skipif(sys.platform.startswith('win'), reason="Can't test hidden files on Windows") + +@pytest.mark.skipif(sys.platform.startswith("win"), reason="Can't test hidden files on Windows") async def test_404(jp_file_contents_manager_class, tmp_path): - #Test visible file in hidden folder + # Test visible file in hidden folder with pytest.raises(HTTPError) as excinfo: td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) - hidden_dir = '.hidden' - file_in_hidden_path = os.path.join(hidden_dir,'visible.txt') + hidden_dir = ".hidden" + file_in_hidden_path = os.path.join(hidden_dir, "visible.txt") _make_dir(cm, hidden_dir) model = await ensure_async(cm.new(path=file_in_hidden_path)) - os_path = cm._get_os_path(model['path']) + os_path = cm._get_os_path(model["path"]) try: - result = await ensure_async(cm.get(os_path, 'w')) + result = await ensure_async(cm.get(os_path, "w")) except HTTPError as e: assert e.status_code == 404 - #Test hidden file in visible folder + # Test hidden file in visible folder with pytest.raises(HTTPError) as excinfo: td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) - hidden_dir = 'visible' - file_in_hidden_path = os.path.join(hidden_dir,'.hidden.txt') + hidden_dir = "visible" + file_in_hidden_path = os.path.join(hidden_dir, ".hidden.txt") _make_dir(cm, hidden_dir) model = await ensure_async(cm.new(path=file_in_hidden_path)) - os_path = cm._get_os_path(model['path']) + os_path = cm._get_os_path(model["path"]) try: - result = await ensure_async(cm.get(os_path, 'w')) + result = await ensure_async(cm.get(os_path, "w")) except HTTPError as e: assert e.status_code == 404 + async def test_escape_root(jp_file_contents_manager_class, tmp_path): td = str(tmp_path) cm = jp_file_contents_manager_class(root_dir=td) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/jupyter_server-1.17.1/tests/test_gateway.py new/jupyter_server-1.18.1/tests/test_gateway.py --- old/jupyter_server-1.17.1/tests/test_gateway.py 2022-06-07 19:02:17.000000000 +0200 +++ new/jupyter_server-1.18.1/tests/test_gateway.py 2022-07-05 22:22:43.000000000 +0200 @@ -1,17 +1,24 @@ """Test GatewayClient""" +import asyncio import json +import logging import os import uuid from datetime import datetime from io import BytesIO -from unittest.mock import patch +from queue import Empty +from unittest.mock import MagicMock, patch import pytest import tornado from tornado.httpclient import HTTPRequest, HTTPResponse from tornado.web import HTTPError -from jupyter_server.gateway.managers import GatewayClient +from jupyter_server.gateway.managers import ( + ChannelQueue, + GatewayClient, + GatewayKernelManager, +) from jupyter_server.utils import ensure_async from .utils import expected_http_error @@ -135,6 +142,9 @@ # Shutdown existing kernel if endpoint.rfind("/api/kernels/") >= 0 and method == "DELETE": requested_kernel_id = endpoint.rpartition("/")[2] + if requested_kernel_id not in running_kernels: + raise HTTPError(404, message="Kernel does not exist: %s" % requested_kernel_id) + running_kernels.pop( requested_kernel_id ) # Simulate shutdown by removing kernel from running set @@ -158,6 +168,15 @@ mock_http_user = "alice" +def mock_websocket_create_connection(recv_side_effect=None): + def helper(*args, **kwargs): + mock = MagicMock() + mock.recv = MagicMock(side_effect=recv_side_effect) + return mock + + return helper + + @pytest.fixture def init_gateway(monkeypatch): """Initializes the server for use as a gateway client.""" @@ -292,6 +311,101 @@ assert await is_kernel_running(jp_fetch, kernel_id) is False +@pytest.mark.parametrize("missing_kernel", [True, False]) +async def test_gateway_shutdown(init_gateway, jp_serverapp, jp_fetch, missing_kernel): + # Validate server shutdown when multiple gateway kernels are present or + # we've lost track of at least one (missing) kernel + + # create two kernels + k1 = await create_kernel(jp_fetch, "kspec_bar") + k2 = await create_kernel(jp_fetch, "kspec_bar") + + # ensure they're considered running + assert await is_kernel_running(jp_fetch, k1) is True + assert await is_kernel_running(jp_fetch, k2) is True + + if missing_kernel: + running_kernels.pop(k1) # "terminate" kernel w/o our knowledge + + with mocked_gateway: + await jp_serverapp.kernel_manager.shutdown_all() + + assert await is_kernel_running(jp_fetch, k1) is False + assert await is_kernel_running(jp_fetch, k2) is False + + +@patch("websocket.create_connection", mock_websocket_create_connection(recv_side_effect=Exception)) +async def test_kernel_client_response_router_notifies_channel_queue_when_finished( + init_gateway, jp_serverapp, jp_fetch +): + # create + kernel_id = await create_kernel(jp_fetch, "kspec_bar") + + # get kernel manager + km: GatewayKernelManager = jp_serverapp.kernel_manager.get_kernel(kernel_id) + + # create kernel client + kc = km.client() + + await ensure_async(kc.start_channels()) + + with pytest.raises(RuntimeError): + await kc.iopub_channel.get_msg(timeout=10) + + all_channels = [ + kc.shell_channel, + kc.iopub_channel, + kc.stdin_channel, + kc.hb_channel, + kc.control_channel, + ] + assert all(channel.response_router_finished if True else False for channel in all_channels) + + await ensure_async(kc.stop_channels()) + + # delete + await delete_kernel(jp_fetch, kernel_id) + + +async def test_channel_queue_get_msg_with_invalid_timeout(): + queue = ChannelQueue("iopub", MagicMock(), logging.getLogger()) + + with pytest.raises(ValueError): + await queue.get_msg(timeout=-1) + + +async def test_channel_queue_get_msg_raises_empty_after_timeout(): + queue = ChannelQueue("iopub", MagicMock(), logging.getLogger()) + + with pytest.raises(Empty): + await asyncio.wait_for(queue.get_msg(timeout=0.1), 2) + + +async def test_channel_queue_get_msg_without_timeout(): + queue = ChannelQueue("iopub", MagicMock(), logging.getLogger()) + + with pytest.raises(asyncio.TimeoutError): + await asyncio.wait_for(queue.get_msg(timeout=None), 1) + + +async def test_channel_queue_get_msg_with_existing_item(): + sent_message = {"msg_id": 1, "msg_type": 2} + queue = ChannelQueue("iopub", MagicMock(), logging.getLogger()) + queue.put_nowait(sent_message) + + received_message = await asyncio.wait_for(queue.get_msg(timeout=None), 1) + + assert received_message == sent_message + + +async def test_channel_queue_get_msg_when_response_router_had_finished(): + queue = ChannelQueue("iopub", MagicMock(), logging.getLogger()) + queue.response_router_finished = True + + with pytest.raises(RuntimeError): + await queue.get_msg() + + # # Test methods below... #