[Testing] Investigate cost/benefit of in-memory network vs. E2E tests

[bryanchriswhite]

Objective

Determine whether in-memory tests can be sufficiently sped up to comparable runtime of an isomorphic test implemented using the E2E test harness by outlining the necessary steps and estimating level of effort of either approach.

Origin Document

Implementing claim/proof lifecycle integration tests (#403) depends on either a refactoring of existing in-memory network test helpers (implementing InMemoryCosmosNetwork; see: #290 & #307) OR framing such tests as "E2E tests" with along with the implementation of a mechanism sufficient to support isolation of such tests/scenarios (i.e. local/devnet state reset; likely via k8s go API).

Relates to and informs #255.

Goals

• Determine whether multiple in-memory networks can run concurrently.

  • Outcome:
    It does not seem to be possible to run multiple in-memory networks at the same time. A package wide lock is used by the CosmosSDK to prevent that.
    
    https://github.com/cosmos/cosmos-sdk/blob/main/testutil/network/network.go#L306
    
    https://github.com/cosmos/cosmos-sdk/blob/main/testutil/network/network.go#L60

• Determine the viability and level of effort of supporting E2E test isolation as compared to in-memory network helper refactoring.

  • Outcome:

• Determine the path of least resistance for implementing missing unit, integration tests (see: [Proof] Claim & proof lifecycle testing #403).

Deliverables

• A notion document outlining the necessary steps & level of effort for either approach, sufficient to make an informed decision.
• New issue(s) with corresponding goals and deliverables based on the determined path of least resistance. I.e., either resuming [Testing, Tooling] feat: implement in-memory networks #290 & [Testing, Tooling] refactor: in-memory network usage #307 OR supporting local/devenet state reset in E2E tests.

Non-goals / Non-deliverables

• Refactoring in-memory network test helpers.
• Implementing E2E test isolation support (network state reset).
• Refactoring/rewriting any test code.

General deliverables

• Comments: Add/update TODOs and comments alongside the source code so it is easier to follow.
• Testing: Add new tests (unit and/or E2E) to the test suite.
• Makefile: Add new targets to the Makefile to make the new functionality easier to use.
• Documentation: Update architectural or development READMEs; use mermaid diagrams where appropriate.

---

Creator: @bryanchriswhite
Co-Owners: @okdas @Olshansk

[bryanchriswhite]

Closing as obsolete.