Version 5.0.0-beta.1 - Going live

- Migrated from internal repo

.github/workflows/publish-to-pypi.yml

@@ -0,0 +1,29 @@
+name: Publish Python Package
+
+on:
+  release:
+    types: [ created ]
+
+jobs:
+  build-and-publish:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: '3.8'
+      - name: Install Dependencies
+        run: |
+          python --version
+          python -m pip install --upgrade pip
+          pip install --upgrade setuptools wheel twine
+      - name: Build and Package
+        run: |
+          python setup.py sdist bdist_wheel
+      - name: Publish to PyPI
+        uses: pypa/[email protected]
+        with:
+          user: __token__
+          password: ${{ secrets.PYPI_API_TOKEN }}
+          verbose: true

.gitignore

@@ -0,0 +1,17 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Virtual environments
+venv*/
+.venv*/
+
+# IDEA Stuff
+.idea/
+
+# Local stuff
+/proto/
+/build/
+/tests/res/build/
+-/tests/res/build/empty.txt

CHANGELOG.md

@@ -0,0 +1,159 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+
+## [5.0.0] - 2024-04-15
+
+### Changed
+
+- Moved this entire project over to Github 
+- Bumped the version in order to not confuse older stuff that doesn't expect 
+  protoplasm to exist in Pypi.org (if we end up migriting this there and 
+  just open-sourcing the whole thing)
+  - Also in case something changes in the API while migrating, cause I tend 
+    to fiddle with the code and tidy up and refactor when moving stuff
+
+
+## [4.6.0] - 2024-03-19
+
+### Added
+
+- Docker image building of the latest stable version of Protoplasm and all 
+  its dependencies to this project for both Alpine and Debian and Python 
+  versions 3.8-3.12
+  - Thus, checking in and publishing a new stable version of Protoplasm 
+    should trigger new Docker base images with Protoplasm and all the gRPC 
+    binaries up-to-date and ready to be used by other projects (speeding up 
+    their testing and building considerably)
+  - ...at least in the old CI/CD environment... We'll see what's what once 
+    this is all in Github and Github Actions
+
+### Changed
+
+- A ton of CI/CD stuff that's irrelevant now that we've moved to Github
+
+
+
+## [4.3.0] - 2022-08-03
+
+Fixed ANOTHER bug in type hint importing
+
+### Fixed
+
+- Now importing `List` and `Dict` with nested dots work (e.g. `List[foo.Bar]`)
+
+
+## [4.2.0] - 2022-08-03
+
+Fixed bug in type hint importing
+
+### Fixed
+
+- Now importing `List` and `Dict` works (was only `typing.List` and `typing.Dict` before)
+
+
+
+## [4.1.0] - 2022-04-24
+
+### Added
+
+- Added the `ResponseIterator` which remote method calls with stream output now return as a wrapper around the previously returned iterable results.
+- Added `TimeoutResponseIterator` which wraps the new `ResponseIterator` with timeout functionality while waiting for response stream iterations.
+- `ResponseIterator` has a `with_timeout()` method which automatically wraps it in a `TimeoutResponseIterator`
+- Both `ResponseIterator` and `TimeoutResponseIterator` have a `one_and_done()` method which waits for one next response stream iteration and then closes the stream.
+- `RequestIterator` can now take in a `list` or `tuple` as `initial_request` to load up multiple initial requests.
+- Added `close()` method to `RequestIterator` to close the request stream (force a cancel event)
+- Added `unpack()` to `DataclassBase`
+
+
+## [4.0.0] - 2022-06-18
+
+Major update that unifies the API and functionality of versions 2 and 3.
+
+Skipping a bunch of versions and checkins between 3.0 and 4.0 cause they're 
+not really important at the moment (it's 2024 and I'm migrating this from 
+our internal Gitlab repo to Github).
+
+### Added
+
+- Unify the unary functionality of Protoplasm 2 with the streaming 
+  functionality of Protoplasm 3
+  - The two turn out to be completely incompatible and API shattering 
+    Protoplasm 4 must incorporate BOTH functionalities wile being backwards 
+    compatible enough for both Protoplasm 2 and 3 projects to be able to 
+    migrate to 4
+  -The key here is detecting the `stream` keyword in protos that denote 
+    streaming input and/or output
+- Add piled up functionality/utility/QoL improvements/bugs that's been on The 
+  List™ for a while
+  - Cast to/from base64 encoded strings
+  - Utilize the `__all__` directive to isolate `import *` side effects 
+  - Integrate the Neobuf Builder CLI (from various other projects) into 
+    Protoplasm and generalize it
+  - Address the "`None` is default value" issue
+  - Explore the pros/cons of making non-existing Message/Object attributes 
+    return `Empty` or `EmptyDict` to simplify nested attribute fetching...?
+
+### Changed
+
+- Refactor and restructure the package properly
+  - Separate the 4 main roles of the package logically
+    1. Cross-piling `*.proto` to `*_pb2.py` and `*_pb2_grpc.py`
+    2. Cross-piling `*_pb2.py` to `*_dc.py` Neobuf Dataclasses
+    3. Generating `*_api.py` interfaces
+    4. Generating gRPC implementation of Services
+
+
+## [3.0.0] - 2021-05-11
+
+Major update including support for gRPC streams.
+
+Skipping a bunch of versions and checkins between 2.0 and 3.0 cause they're 
+not really important at the moment (it's 2024 and I'm migrating this from 
+our internal Gitlab repo to Github).
+
+## Added
+
+- Support for gRPC streams
+
+## Changed
+
+- This broke backwards compatibility with version 1 and 2's non-streaming 
+  gRPC calls
+
+
+## [2.0.0] - 2021-02-18
+
+Major update including support for secure channels.
+
+Skipping a bunch of versions and checkins between 1.0 and 2.0 cause they're 
+not really important at the moment (it's 2024 and I'm migrating this from 
+our internal Gitlab repo to Github).
+
+### Added
+
+- Support for Secure gRPC channels
+
+
+## [1.0.0] - 2019-07-11
+
+Initial stable release. 
+
+Skipping a bunch of versions and checkins between 0.1 and 1.0 cause they're 
+not really important at the moment (it's 2024 and I'm migrating this from 
+our internal Gitlab repo to Github).
+
+### Added
+
+- A bunch of features
+
+
+## [0.1.0] - 2019-01-17
+
+### Added
+
+- The initial checkin of this project

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2024 CCP Games
+Copyright (c) 2019-2024 CCP Games
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,2 +1,61 @@
-# protoplasm
-Utilities for working with Protobuf & gRPC in Python, e.g; compiling Python 3 Dataclasses from compiled proto-message-python code and casting between the two, quickly initializing large/nested Protobuf object, simplifying gRPC service interfaces etc.
+# Protoplasm
+
+Utilities for working with Protobuf & gRPC in Python, e.g; compiling Python 3 
+Dataclasses from compiled proto-message-python code and casting between the two,
+quickly initializing large/nested Protobuf object, simplifying gRPC service 
+interfaces etc. 
+
+## IMPORTANT
+
+Update this README file after moving from CCP's internal code repo to Github.
+
+
+
+## Protoplasm 4
+
+* Unify the unary functionality of Protoplasm 2 with the streaming 
+  functionality of Protoplasm 3
+  * The two turn out to be completely incompatible and API shattering
+  * Protoplasm 4 must incorporate BOTH functionalities wile being backwards 
+    compatible enough for both Protoplasm 2 and 3 projects to be able to 
+    migrate to 4
+  * The key here is detecting the `stream` keyword in protos that denote 
+    streaming input and/or output
+* Add piled up functionality/utility/QoL improvements/bugs that's been on The 
+  List™ for a while
+  * Cast to/from base64 encoded strings
+  * Utilize the `__all__` directive to isolate `import *` side effects 
+  * Integrate the Neobuf Builder CLI (from various other projects) into 
+    Protoplasm and generalize it
+  * Address the "`None` is default value" issue
+  * Explore the pros/cons of making non-existing Message/Object attributes 
+    return `Empty` or `EmptyDict` to simplify nested attribute fetching...?
+* Refactor and restructure the package properly
+  * Separate the 4 main roles of the package logically
+    1. Cross-piling `*.proto` to `*_pb2.py` and `*_pb2_grpc.py`
+    2. Cross-piling `*_pb2.py` to `*_dc.py` Neobuf Dataclasses
+    3. Generating `*_api.py` interfaces
+    4. Generating gRPC implementation of Services
+
+## Troubleshooting
+
+* I get `TypeError: Plain typing.NoReturn is not valid as type argument`
+    * Upgrade to Python 3.9. This TypeError arises from [a bug in Python 3.7](https://bugs.python.org/issue34921)
+
+
+## Clever bits to document...
+
+- Code Generation (e.g. `foo.proto`) + how to build
+    - Dataclasses -> `foo_dc.py` + how to use (+ DataclassBase freebies)
+        - Extending Dataclasses (a no-no for pb2 files apparently)
+    - Service API -> `foo_api.py` + how to use and implement
+        - Automatic parameter unpacking
+        - Return value packing
+        - Raising `protoplasm.errors.api.*` on errors or non-ok returns
+        - Using `protoplasm.decorators` for param and type checking
+        - The `takes_context` decorator and how to use it
+    - GRPC DC Service Servicer -> `foo_dc_grpc.py` + how to use
+- Utilities
+    - Proto <-> dict <-> Dataclass casters
+    - The `mkproto` and `mkdataclass` helpers
+    - The `unpack_dataclass_request` and `pack_dataclass_response` helpers

protoplasm/__init__.py

@@ -0,0 +1,5 @@
+__version__ = '5.0.0-beta.1'
+
+__author__ = 'Thordur Matthiasson <[email protected]>'
+__license__ = 'MIT License'
+__copyright__ = 'Copyright 2019-2024 - CCP Games ehf'

protoplasm/bases/__init__.py

@@ -0,0 +1,3 @@
+from .dataclass_bases import *
+from .grpc_bases import *
+from ._exwrap import *

protoplasm/bases/_client.py

@@ -0,0 +1,61 @@
+__all__ = [
+    'BaseGrpcClientImplementation',
+]
+import grpc  # noqa
+
+from protoplasm.structs import *
+from ._remotewrap import *
+from .dataclass_bases import *
+
+import logging
+log = logging.getLogger(__name__)
+
+
+class BaseGrpcClientImplementation:
+    def __init__(self, service_stub_cls: Type, grpc_host='localhost:50051',
+                 credentials: Optional[Union[bool, grpc.ChannelCredentials]] = None,
+                 options: Optional[Dict] = None, *args, **kwargs):
+        """
+        credentials:
+        - None -> Automatic... will use `grpc.ssl_channel_credentials()` and secure_channel if port is 443
+        - True -> Will use `grpc.ssl_channel_credentials()` and secure_channel
+        - False -> Will use insecure_channel and no creds
+        - grpc.ChannelCredentials -> Will use given creds and secure_channel
+        """
+        options = options or {}
+        if 'grpc.enable_http_proxy' not in options:
+            options['grpc.enable_http_proxy'] = 0
+        option_tuple = tuple(options.items())
+
+        # Backwards compatibility
+        if kwargs:
+            if not credentials and 'credential' in kwargs:
+                credentials = kwargs.pop('credential')
+                log.warning('Deprecated keyword "credential" used when initiating BaseGrpcClient.'
+                            ' Please use "credentials" instead!')
+
+        # This keeps happening...!
+        if ':' not in grpc_host:
+            if credentials:
+                log.warning('No port used in host... adding 443 by default (cause this is a secure connection)!')
+                grpc_host = f'{grpc_host}:443'
+            else:
+                log.warning('No port used in host... adding 50051 by default!')
+                grpc_host = f'{grpc_host}:50051'
+
+        if credentials is None:
+            if grpc_host.endswith(':443'):
+                credentials = grpc.ssl_channel_credentials()
+        elif credentials is True:
+            credentials = grpc.ssl_channel_credentials()
+
+        if credentials:
+            self.grpc_channel: grpc.Channel = grpc.secure_channel(grpc_host, credentials, options=option_tuple)
+        else:
+            self.grpc_channel: grpc.Channel = grpc.insecure_channel(grpc_host, options=option_tuple)
+
+        self.stub = service_stub_cls(self.grpc_channel)
+
+    def _forward_to_grpc(self, request_cls: Type[T_DCB], stub_method: Callable,
+                         *args, **kwargs) -> Optional[Union[Any, Tuple[Any], Iterable[T_DCB]]]:
+        return RemoteMethodWrapper(self, request_cls, stub_method, *args, **kwargs).call()