Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't instantiate abstract class AsyncPayload with abstract methods decode. When I push to oss #10587

Open
ws1336 opened this issue Oct 15, 2024 · 2 comments
Labels
A: data-sync Related to dvc get/fetch/import/pull/push fs: oss Related to the Alibaba Cloud OSS filesystem help wanted triage Needs to be triaged

Comments

@ws1336
Copy link

ws1336 commented Oct 15, 2024

Bug Report

Can't instantiate abstract class AsyncPayload with abstract methods decode.

Issue name

I can't push my data to oss.

Description

dvc push --verbose
2024-10-15 15:03:18,167 DEBUG: v3.42.0 (pip), CPython 3.8.16 on Windows-10-10.0.22631-SP0
2024-10-15 15:03:18,167 DEBUG: command: D:\anaconda3\envs\work38\Scripts\dvc push --verbose
Collecting                                                                                   |0.00 [00:00,    ?entry/s]
2024-10-15 15:03:18,761 DEBUG: Preparing to transfer data from 'E:\dvc_test\.dvc\cache\files\md5' to 'oss://sigmasquares/DVC_Storage/files/md5'
2024-10-15 15:03:18,761 DEBUG: Preparing to collect status from 'sigmasquares/DVC_Storage/files/md5'
2024-10-15 15:03:18,762 DEBUG: Collecting status from 'sigmasquares/DVC_Storage/files/md5'
2024-10-15 15:03:18,763 DEBUG: Querying 1 oids via object_exists
2024-10-15 15:03:19,578 DEBUG: Estimated remote size: 4096 files
2024-10-15 15:03:19,579 DEBUG: Large remote (4 oids < 40.96 traverse weight), using object_exists for remaining oids
2024-10-15 15:03:19,580 DEBUG: Querying 4 oids via object_exists
2024-10-15 15:03:19,774 DEBUG: Preparing to collect status from 'E:\dvc_test\.dvc\cache\files\md5'
2024-10-15 15:03:19,774 DEBUG: Collecting status from 'E:\dvc_test\.dvc\cache\files\md5'
2024-10-15 15:03:19,865 DEBUG: transfer dir: md5: fdde843b2edc481dd70d4032e35d6226.dir with 3 files
2024-10-15 15:03:20,251 ERROR: failed to transfer '4186881925838e1c3136150473324c01' - Can't instantiate abstract class AsyncPayload with abstract methods decode
Traceback (most recent call last):
  File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\generic.py", line 330, in transfer
    _try_links(
  File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\generic.py", line 262, in _try_links
    return copy(
  File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\generic.py", line 87, in copy
    return _put(
  File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\generic.py", line 156, in _put
    return _put_one(from_paths[0], to_paths[0])
  File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\generic.py", line 148, in _put_one
    return put_file(from_path, to_path, callback=callback, **put_file_kwargs)
  File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\callbacks.py", line 146, in sync_wrapper
    res = fn(*args, **kwargs)
  File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\callbacks.py", line 164, in sync_wrapper
    return fn(path1, path2, **kwargs)
  File "D:\anaconda3\envs\work38\lib\site-packages\dvc_objects\fs\base.py", line 643, in put_file
    self.fs.put_file(os.fspath(from_file), to_info, callback=callback, **kwargs)
  File "D:\anaconda3\envs\work38\lib\site-packages\fsspec\asyn.py", line 118, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "D:\anaconda3\envs\work38\lib\site-packages\fsspec\asyn.py", line 103, in sync
    raise return_result
  File "D:\anaconda3\envs\work38\lib\site-packages\fsspec\asyn.py", line 56, in _runner
    result[0] = await coro
  File "D:\anaconda3\envs\work38\lib\site-packages\ossfs\async_oss.py", line 396, in _put_file
    await self._call_oss(
  File "D:\anaconda3\envs\work38\lib\site-packages\ossfs\async_oss.py", line 162, in _call_oss
    out = await method(*args, **kwargs)
  File "D:\anaconda3\envs\work38\lib\site-packages\aiooss2\api.py", line 581, in put_object_from_file
    return await self.put_object(
  File "D:\anaconda3\envs\work38\lib\site-packages\aiooss2\api.py", line 291, in put_object
    resp: "AioResponse" = await self._do_object(
  File "D:\anaconda3\envs\work38\lib\site-packages\aiooss2\api.py", line 254, in _do_object
    return await self._do(method, self.bucket_name, key, **kwargs)
  File "D:\anaconda3\envs\work38\lib\site-packages\aiooss2\api.py", line 144, in _do
    resp: "AioResponse" = await self.session.do_request(req, timeout=self.timeout)
  File "D:\anaconda3\envs\work38\lib\site-packages\aiooss2\http.py", line 150, in do_request
    resp = await self.session.request(
  File "D:\anaconda3\envs\work38\lib\site-packages\aiohttp\client.py", line 631, in _request
    req = self._request_class(
  File "D:\anaconda3\envs\work38\lib\site-packages\aiohttp\client_reqrep.py", line 338, in __init__
    self.update_body_from_data(data)
  File "D:\anaconda3\envs\work38\lib\site-packages\aiohttp\client_reqrep.py", line 588, in update_body_from_data
    body = payload.PAYLOAD_REGISTRY.get(body, disposition=None)
  File "D:\anaconda3\envs\work38\lib\site-packages\aiohttp\payload.py", line 116, in get
    return factory(data, *args, **kwargs)
TypeError: Can't instantiate abstract class AsyncPayload with abstract methods decode

dvc doctor

DVC version: 3.42.0 (pip)
-------------------------
Platform: Python 3.8.16 on Windows-10-10.0.22631-SP0
Subprojects:
        dvc_data = 3.8.0
        dvc_objects = 3.0.6
        dvc_render = 1.0.1
        dvc_task = 0.40.2
        scmrepo = 2.0.4
Supports:
        http (aiohttp = 3.10.10, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.10.10, aiohttp-retry = 2.8.3),
        oss (ossfs = 2023.12.0)
Config:
        Global: C:\Users\Sigma Squares\AppData\Local\iterative\dvc
        System: C:\ProgramData\iterative\dvc
Cache types: hardlink
Cache directory: NTFS on E:\
Caches: local
Remotes: oss
Workspace directory: NTFS on E:\
Repo: dvc, git
Repo.site_cache_dir: C:\ProgramData\iterative\dvc\Cache\repo\aee995bc5ba11f246beffb6ea62fed52
@shcheklein shcheklein added A: data-sync Related to dvc get/fetch/import/pull/push fs: oss Related to the Alibaba Cloud OSS filesystem help wanted triage Needs to be triaged labels Oct 15, 2024
@ws1336
Copy link
Author

ws1336 commented Oct 16, 2024

I tried both Windows and Linux systems, but they didn't work. I can manually upload cache to OSS and pull it locally, but I cannot push it to OSS through DVC

@ws1336
Copy link
Author

ws1336 commented Oct 16, 2024

I downgraded the aiohttp version from 3.10.10 to 3.9.5, which temporarily resolved this issue. However, compatibility between DVC and the new version of aiohttpis still an issue, and the released PyInstaller packaged version of Windows still cannot be used properly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: data-sync Related to dvc get/fetch/import/pull/push fs: oss Related to the Alibaba Cloud OSS filesystem help wanted triage Needs to be triaged
Projects
None yet
Development

No branches or pull requests

2 participants