Question on asynchronous (concurrent) testing, hiding subtests' parent function

[cglacet]

I just asked a question on stackoverflow about ways to run test concurrently using pytest, it was suggested to use pytest-subtests so I tried it and it seems to work for a simple example.

I just wanted to have your thoughts on this strategy (limitations/interest/ideas).

import pytest
import sys
import asyncio
import inspect
import re
import time


pytestmark = pytest.mark.asyncio
io_test_pattern = re.compile("io_.*")


async def tests(subtests):

    def find_io_tests(subtests, ignored_names):
        functions = inspect.getmembers(sys.modules[__name__], inspect.isfunction)
        for (f_name, function) in functions:
            if f_name in ignored_names:
                continue
            if re.search(io_test_pattern, f_name):
                yield run(subtests, f_name, function)

    async def run(subtests, test_name, test_function):
        with subtests.test(msg=test_name):
            await test_function()

    self_name = inspect.currentframe().f_code.co_name
    return await asyncio.gather(*find_io_tests(subtests, {self_name}))


async def io_test_1():
    await assert_sleep_duration_ok(1)

async def io_test_2():
    await assert_sleep_duration_ok(2)

async def io_test_3():
    await assert_sleep_duration_ok(3)

async def io_test_4():
    await assert_sleep_duration_ok(4, fail=True)

MAX_ERROR = 0.1

async def assert_sleep_duration_ok(duration, fail=False):
    start = time.time()
    await asyncio.sleep(duration)
    actual_duration = time.time() - start
    assert abs(actual_duration - duration) < MAX_ERROR
    assert not fail

The output summary of this is a bit unclear for my use case but from my understanding this is related to #9 and pytest-dev/pytest#5047. Right?

============================= test session starts =============================
platform darwin -- Python 3.7.0, pytest-4.6.2, py-1.8.0, pluggy-0.12.0
cachedir: .pytest_cache
rootdir: /Users/cglacet/test/async-tests
plugins: asyncio-0.10.0, trio-0.5.2, subtests-0.2.1
collected 2 items

asyncio_test.py::tests PASSED                                           [ 50%]
asyncio_test.py::tests PASSED                                           [ 50%]
asyncio_test.py::tests PASSED                                           [ 50%]
asyncio_test.py::tests FAILED                                           [ 50%]
asyncio_test.py::tests PASSED                                           [ 50%]

================================== FAILURES ===================================

It would make more sense for me to have something that completely hides the existence of the tests function and only shows something like:

asyncio_test.py::io_test_1 PASSED                                       [ 25%]
asyncio_test.py::io_test_2 PASSED                                       [ 50%]
asyncio_test.py::io_test_3 PASSED                                       [ 75%]
trio_test.py::io_test_4 FAILED                                          [100%]