diff --git a/CHANGELOG.md b/CHANGELOG.md index a0600e3..aa5d38d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,12 +17,23 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm argument, enabling usage on single-file applications. Beforehand, only invocations of Python modules were possible. ```shell + # Install Responder with CLI extension. + pip install 'responder[cli]' + # Start Responder application defined in Python module. responder run acme.app:api # Start Responder application defined in a single Python file. responder run examples/helloworld.py:api ``` +- CLI: `responder run` now also accepts URLs. + ```shell + # Install Responder with CLI extension (full). + pip install 'responder[cli-full]' + + # Start Responder application defined in Python module at remote location. + responder run https://github.com/kennethreitz/responder/raw/refs/heads/main/examples/helloworld.py + ``` ### Changed diff --git a/docs/source/cli.rst b/docs/source/cli.rst index 14230b9..a92f4aa 100644 --- a/docs/source/cli.rst +++ b/docs/source/cli.rst @@ -2,72 +2,149 @@ Responder CLI ============= Responder installs a command line program ``responder``. Use it to launch -a Responder application from a file or module. +a Responder application from a file or module, either located on a local +or remote filesystem, or object store. -Launch application from file ----------------------------- +Launch Module Entrypoint +------------------------ + +For loading a Responder application from a Python module, you will refer to +its ``API()`` instance using a `Python entry point object reference`_ that +points to a Python object. It is either in the form ``importable.module``, +or ``importable.module:object.attr``. + +A basic invocation command to launch a Responder application: + +.. code-block:: shell + + responder run acme.app + +The command above assumes a Python package ``acme`` including an ``app`` +module ``acme/app.py`` that includes an attribute ``api`` that refers +to a ``responder.API`` instance, reflecting the typical layout of +a standard Responder application. + +Loading a Responder application using an entrypoint specification will +inherit the capacities of `Python's import system`_, as implemented by +`importlib`_. + +Launch Local File +----------------- -Acquire minimal example application, `helloworld.py`_, -implementing a basic echo handler, and launch the HTTP service. +Acquire a minimal example single-file application, ``helloworld.py`` [1]_, +to your local filesystem, giving you the chance to edit it, and launch the +Responder HTTP service. .. code-block:: shell wget https://github.com/kennethreitz/responder/raw/refs/heads/main/examples/helloworld.py responder run helloworld.py -In another terminal, invoke a HTTP request, for example using `HTTPie`_. +.. note:: -.. code-block:: shell + To validate the example application, invoke a HTTP request, for example using + `curl`_, `HTTPie`_, or your favourite browser at hand. + + .. code-block:: shell - http http://127.0.0.1:5042/hello + http http://127.0.0.1:5042/Hello -The response is no surprise. + The response is no surprise. -:: + :: - HTTP/1.1 200 OK - content-length: 13 - content-type: text/plain - date: Sat, 26 Oct 2024 13:16:55 GMT - encoding: utf-8 - server: uvicorn + HTTP/1.1 200 OK + content-length: 13 + content-type: text/plain + date: Sat, 26 Oct 2024 13:16:55 GMT + encoding: utf-8 + server: uvicorn - hello, world! + Hello, world! +.. [1] The Responder application `helloworld.py`_ implements a basic echo handler. -Launch application from module ------------------------------- +Launch Remote File +------------------ -If your Responder application has been implemented as a Python module, -launch it like this: +You can also launch a single-file application where its Python file is stored +on a remote location. + +Responder supports all filesystem adapters compatible with `fsspec`_, and +installs the adapters for Azure Blob Storage (az), Google Cloud Storage (gs), +GitHub, HTTP, and AWS S3 by default. .. code-block:: shell - responder run acme.app + # Works 1:1. + responder run https://github.com/kennethreitz/responder/raw/refs/heads/main/examples/helloworld.py + responder run github://kennethreitz:responder@/examples/helloworld.py -That assumes a Python package ``acme`` including an ``app`` module -``acme/app.py`` that includes an attribute ``api`` that refers -to a ``responder.API`` instance, reflecting the typical layout of -a standard Responder application. +If you need access other kinds of remote targets, see the `list of +fsspec-supported filesystems and protocols`_. The next section enumerates +a few synthetic examples. The corresponding storage buckets do not even +exist, so don't expect those commands to work. -.. rubric:: Non-standard instance name +.. code-block:: shell + + # Azure Blob Storage, Google Cloud Storage, and AWS S3. + responder run az://kennethreitz-assets/responder/examples/helloworld.py + responder run gs://kennethreitz-assets/responder/examples/helloworld.py + responder run s3://kennethreitz-assets/responder/examples/helloworld.py + + # Hadoop Distributed File System (hdfs), SSH File Transfer Protocol (sftp), + # Common Internet File System (smb), Web-based Distributed Authoring and + # Versioning (webdav). + responder run hdfs://kennethreitz-assets/responder/examples/helloworld.py + responder run sftp://user@host/kennethreitz/responder/examples/helloworld.py + responder run smb://workgroup;user:password@server:port/responder/examples/helloworld.py + responder run webdav+https://user:password@server:port/responder/examples/helloworld.py + +.. tip:: + + In order to install support for all filesystem types supported by fsspec, run: + + .. code-block:: shell + + uv pip install 'fsspec[full]' + + When using ``uv``, this concludes within an acceptable time of approx. + 25 seconds. If you need to be more selectively instead of using ``full``, + choose from one or multiple of the available `fsspec extras`_, which are: + + abfs, arrow, dask, dropbox, fuse, gcs, git, github, hdfs, http, oci, s3, + sftp, smb, ssh. -When your attribute that references the ``responder.API`` instance -is called differently than ``api``, append it to the launch target -address like this: +Launch with Non-Standard Instance Name +-------------------------------------- + +By default, Responder will acquire an ``responder.API`` instance using the +symbol name ``api`` from the specified Python module. + +If your main application file uses a different name than ``api``, please +append the designated symbol name to the launch target address. + +It works like this for module entrypoints and local files: .. code-block:: shell responder run acme.app:service + responder run /path/to/acme/app.py:service + +It works like this for URLs: -Within your ``app.py``, the instance would have been defined like this: +.. code-block:: shell + + responder run http://app.server.local/path/to/acme/app.py#service + +Within your ``app.py``, the instance would have been defined to use +the ``service`` symbol name instead of ``api``, like this: .. code-block:: python service = responder.API() - -Build JavaScript application +Build JavaScript Application ---------------------------- The ``build`` subcommand invokes ``npm run build``, optionally accepting @@ -78,7 +155,7 @@ where it expects a regular NPM ``package.json`` file. responder build -When specifying a target directory, responder will change to that +When specifying a target directory, Responder will change to that directory beforehand. .. code-block:: shell @@ -86,5 +163,12 @@ directory beforehand. responder build /path/to/project +.. _curl: https://curl.se/ +.. _fsspec: https://filesystem-spec.readthedocs.io/en/latest/ +.. _fsspec extras: https://github.com/fsspec/filesystem_spec/blob/1fe5695802679e8ce1ae242a44d4846195425509/pyproject.toml#L26-L68 .. _helloworld.py: https://github.com/kennethreitz/responder/blob/main/examples/helloworld.py .. _HTTPie: https://httpie.io/docs/cli +.. _importlib: https://docs.python.org/3/library/importlib.html +.. _list of fsspec-supported filesystems and protocols: https://github.com/fsspec/universal_pathlib?tab=readme-ov-file#currently-supported-filesystems-and-protocols +.. _Python entry point object reference: https://packaging.python.org/en/latest/specifications/entry-points/ +.. _Python's import system: https://docs.python.org/3/reference/import.html diff --git a/responder/util/common.py b/responder/util/common.py new file mode 100644 index 0000000..f8389cb --- /dev/null +++ b/responder/util/common.py @@ -0,0 +1,9 @@ +from urllib.parse import urlparse + + +def is_valid_url(url): + try: + result = urlparse(url) + return all([result.scheme, result.netloc]) + except ValueError: + return False diff --git a/responder/util/python.py b/responder/util/python.py index ce8ce55..1953ed4 100644 --- a/responder/util/python.py +++ b/responder/util/python.py @@ -5,8 +5,13 @@ import typing as t import uuid from pathlib import Path +from tempfile import NamedTemporaryFile from types import ModuleType +from upath import UPath + +from responder.util.common import is_valid_url + logger = logging.getLogger(__name__) @@ -31,12 +36,13 @@ def load_target(target: str, default_property: str = "api", method: str = "run") source to prevent security vulnerabilities. Args: - target: Module address (e.g., 'acme.app:foo') or file path (e.g., '/path/to/acme/app.py') + target: Module address (e.g., 'acme.app:foo'), file path (e.g., '/path/to/acme/app.py'), + or URL. default_property: Name of the property to load if not specified in target (default: "api") - method: Name of the method to verify on the loaded property (default: "run") + method: Name of the method to invoke on the API instance (default: "run") Returns: - The loaded property from the module + The API instance, loaded from the given property. Raises: ValueError: If target format is invalid @@ -48,6 +54,22 @@ def load_target(target: str, default_property: str = "api", method: str = "run") >>> api.run() """ # noqa: E501 + app_file = None + if is_valid_url(target): + upath = UPath(target) + frag = upath._url.fragment + suffix = upath.suffix + suffix = suffix.replace(f"#{frag}", "") + logger.info(f"Loading remote single-file application, source: {upath}") + name = "_".join([upath.parent.stem, upath.stem]) + app_file = NamedTemporaryFile(prefix=f"{name}_", suffix=suffix, delete=False) + target = app_file.name + if frag: + target = f"{app_file.name}:{frag}" + logger.info(f"Writing remote single-file application, target: {target}") + app_file.write(upath.read_bytes()) + app_file.flush() + # Sanity checks, as suggested by @coderabbitai. Thanks. if not target or (":" in target and len(target.split(":")) != 2): raise InvalidTarget(f"Invalid target format: {target}") diff --git a/setup.py b/setup.py index 1ec1b41..055801f 100644 --- a/setup.py +++ b/setup.py @@ -119,7 +119,11 @@ def run(self): setup_requires=[], install_requires=required, extras_require={ - "cli": ["docopt-ng"], + "cli": [ + "docopt-ng", + "fsspec[abfs,gcs,github,http,libarchive,s3]", + "universal-pathlib", + ], "develop": [ "poethepoet", "pyproject-fmt; python_version>='3.7'", diff --git a/tests/test_cli.py b/tests/test_cli.py index e878ab9..6451a10 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -150,21 +150,25 @@ def test_cli_build_invalid_package_json( assert any(item in stderr for item in to_list(expected_error)) +sfa_services_valid = [ + str(Path("examples") / "helloworld.py"), + "https://github.com/kennethreitz/responder/raw/refs/heads/main/examples/helloworld.py", +] + + # The test is marked as flaky due to potential race conditions in server startup # and port availability. Known error codes by platform: # - macOS: [Errno 61] Connection refused (Failed to establish a new connection) # - Linux: [Errno 111] Connection refused (Failed to establish a new connection) # - Windows: [WinError 10061] No connection could be made because target machine # actively refused it -@pytest.mark.flaky(reruns=5, reruns_delay=2) -def test_cli_run(capfd): +@pytest.mark.flaky(reruns=3, reruns_delay=2, only_rerun=["TimeoutError"]) +@pytest.mark.parametrize("target", sfa_services_valid, ids=sfa_services_valid) +def test_cli_run(capfd, target): """ Verify that `responder run` works as expected. """ - # Invoke `responder run`. - target = Path("examples") / "helloworld.py" - # Start a Responder service instance in the background, using its CLI. # Make it terminate itself after serving one HTTP request. server = ResponderServer(target=str(target), port=random_port(), limit_max_requests=1)