borg.helpers.msgpack.UnpackException raised when attempting to create a new backup

[arthurzenika]

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

1.2.8

Operating system (distribution) and version.

Ubuntu 24.02

Hardware / network configuration, and filesystems used.

n/a

How much data is handled by borg?

------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
All archives:              448.81 GB            265.74 GB             36.63 GB

                       Unique chunks         Total chunks
Chunk index:                  867475             10303773

Full borg commandline that lead to the problem (leave away excludes and passwords)

Platform: Linux zenika-x1 6.8.0-51-generic #52-Ubuntu SMP PREEMPT_DYNAMIC Thu Dec  5 13:09:44 UTC 2024 x86_64
Linux: Unknown Linux  
Borg: 1.2.8  Python: CPython 3.12.7 msgpack: 1.0.8 fuse: pyfuse3 3.3.0 [pyfuse3,llfuse]
PID: 324  CWD: /home/arthur
sys.argv: ['/app/bin/borg', 'create', '--rsh', 'ssh -o BatchMode=yes -o StrictHostKeyChecking=accept-new', '--progress', '--json', '--compression=zstd', '--log-json', '--exclude=fm:.terraform', [snip], '--', '/home/arthur/.local/backup-zenika-x1-arthur::1467c9-645435b3', '/home/arthur/']
SSH_ORIGINAL_COMMAND: None

Describe the problem you're observing.

Can't create new backup, borg.helpers.msgpack.UnpackException raised, it then leaves a lock which I attempt to break with the break-lock command..

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

Relaunch the backup in Pika

Include any warning/errors/backtraces from the system logs

Exception: Local Exception

Error: Exception – Local Exception
Error: Traceback (most recent call last):
  File "/app/lib/python3.12/site-packages/borg/helpers/msgpack.py", line 117, in __next__
    return super().__next__()
           ^^^^^^^^^^^^^^^^^^
  File "msgpack/_unpacker.pyx", line 540, in msgpack._cmsgpack.Unpacker.__next__
  File "msgpack/_unpacker.pyx", line 483, in msgpack._cmsgpack.Unpacker._unpack
msgpack.exceptions.StackError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/lib/python3.12/site-packages/borg/archiver.py", line 5401, in main
    exit_code = archiver.run(args)
                ^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.12/site-packages/borg/archiver.py", line 5321, in run
    return set_ec(func(args))
                  ^^^^^^^^^^
  File "/app/lib/python3.12/site-packages/borg/archiver.py", line 190, in wrapper
    return method(self, args, repository=repository, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.12/site-packages/borg/archiver.py", line 663, in do_create
    with Cache(repository, key, manifest, progress=args.progress,
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.12/site-packages/borg/cache.py", line 387, in __new__
    return local()
           ^^^^^^^
  File "/app/lib/python3.12/site-packages/borg/cache.py", line 378, in local
    return LocalCache(repository=repository, key=key, manifest=manifest, path=path, sync=sync,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.12/site-packages/borg/cache.py", line 487, in __init__
    self.open()
  File "/app/lib/python3.12/site-packages/borg/cache.py", line 542, in open
    self.rollback()
  File "/app/lib/python3.12/site-packages/borg/cache.py", line 662, in rollback
    self._do_open()
  File "/app/lib/python3.12/site-packages/borg/cache.py", line 531, in _do_open
    self._read_files()
  File "/app/lib/python3.12/site-packages/borg/cache.py", line 568, in _read_files
    for path_hash, item in u:
                           ^
  File "/app/lib/python3.12/site-packages/borg/helpers/msgpack.py", line 121, in __next__
    raise UnpackException(e)
borg.helpers.msgpack.UnpackException

[infectormp]

Could you please update borg to the latest version and try to reproduce the issue again.

[arthurzenika]

Asked the upstream project https://gitlab.gnome.org/World/pika-backup/-/issues/585 (I currently have no flatpack building skills... yet?)

[ThomasWaldmann]

It looks like your files cache got corrupted somehow:

1. Check your borg client hardware, especially RAM and disk / filesystem.
2. borg delete --cache-only YOURREPO

The first backup after deleting the cache will be slower than usual.

[arthurzenika]

@ThomasWaldmann thanks for looking into this. The borg delete --cache-only YOURREPO command combined with a release-lock, fixes the issue. The borg commands needed to be done in the flatpak environment flatpak run --command=bash org.gnome.World.PikaBackup.

@infectormp sorry couldn't test with an updated version of borg yet, the nightly version of PikaBackup has a separate config apparently