Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid using xargs when creating package tarballs #136

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bscott-zebra
Copy link

Fix for this issue: #110

xargs was being used to run tar to create each package tarball. But xargs has a default --max-chars limit (per command it will run) of 128KiB (assuming the system ARG_MAX is significantly larger than this). For packages that contain a large number of files, this limit is exceeded, and the argument list is split into multiple tar runs. Since the -c option is used to create a new tar archive each run, only the files passed into the last execution would wind up in the resulting tar file.

With this change, the tar file list is now being passed via tar's --files-from option, using process substitution, there-by avoiding the above mentioned limit, and simplifying tarball creation as well.

@koppor
Copy link
Contributor

koppor commented Sep 17, 2024

@awalsh128 May we ask for a review here? 😅 For me, the code looks good!

@koppor
Copy link
Contributor

koppor commented Sep 18, 2024

@bscott-zebra Do I need to recreate the cache? I tried the action at koppor/plantuml#41:

Output at https://github.com/koppor/plantuml/actions/runs/10921258797/job/30312933717?pr=41

Run bscott-zebra/cache-apt-pkgs-action@bdb5bbaae8f11b3e4e63cf78e303ac1519dcff7f
...
11:32:31.703 Found 60 files in the cache.
...
11:32:31.727 - inkscape=1.1.2-3ubuntu1.tar
...
Run dpkg -L inkscape
dpkg-query: package 'inkscape' is not installed

Action:

      - uses: bscott-zebra/cache-apt-pkgs-action@bdb5bbaae8f11b3e4e63cf78e303ac1519dcff7f
        with:
          packages: graphviz inkscape
          version: 1.0
          execute_install_scripts: true

@koppor
Copy link
Contributor

koppor commented Sep 18, 2024

Even with version increase, it does not work.

11:41:00.350 - inkscape=1.1.2-3ubuntu1.tar restoring...
11:41:00.745   done
...
Run dpkg -L inkscape
  
dpkg-query: package 'inkscape' is not installed
  1. Cache-creation-run: https://github.com/koppor/plantuml/actions/runs/10921345076/job/30313209058?pr=41
  2. Cacche-usage-run: https://github.com/koppor/plantuml/actions/runs/10921345076/job/30313325880?pr=41

@bscott-zebra
Copy link
Author

@koppor Yes, the problem was on the cache creation side of things, so a new cache file would have to be created after using the fixed version of this action. Your subsequent version increase did this.

But, your first test on the restored package is to run dpkg -L on it. For this to work, it would require that the files in /var/lib/dpkg be updated as part of, or after cache restore. I don't believe this action makes any attempt to do that. I can see from your workflow run that files were cached and restored for inkscape, so theoretically you should be able to use it successfully after the restore (i.e. inkscape should be available in the path) but dpkg won't know about its status.

Can you try your workflow again, but with the dpkg -L inkscape step removed?

@koppor
Copy link
Contributor

koppor commented Sep 18, 2024

Can you try your workflow again, but with the dpkg -L inkscape step removed?

@bscott-zebra Thank you for the hint! That was a "leftover" from debugging. Removed. Works. Thank you! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants