[gatsby-plugin-manifest] Don't regenerate unchanged/cached assets

[missmatsuko]

Summary

Create and check a cached hash file. Don't regenerate assets (icons, manifest) unless the hashes actually changed.

Basic example

On another (non-Gatsby) project, I'm using app-manifest-webpack-plugin to generate app icons and the app manifest file. It uses a cache file so that files don't get regenerated until the source image and/or data actually changes.

Motivation

In general, app icons and manifests probably aren't changing that frequently. This plugin adds several minutes to build time on every build when I added it to my Netlify-hosted gatsby site that is based on gatsby-starter-netlify-cms.

[pieh]

This plugin adds several minutes to build time on every build

This is super weird. It definitely shouldn't take that long. Are you sure that's this plugin that cause this? I would suspect that this is not manifest plugin, but rather image processing from gatsby-plugin-sharp.

Also on netlify .cache or public directories is not cached between builds. You need https://www.npmjs.com/package/gatsby-plugin-netlify-cache for it.

[missmatsuko]

I didn't know about the gatsby-plugin-netlify-cache plugin. I'm not sure why, but that gatsby-starter-netlify-cms starter deletes public and cache on every build.

I tried adding the plugin and updating the build script so it doesn't clear out public and cache. Then I tried deploying on Netlify a few times, and it took about 10 minutes each build.

From the deploy details, it looks like the manifest icons are still generated each time, and it takes about 9 minutes:

9:48:25 AM: info bootstrap finished - 22.247 s
9:57:08 AM: done generating icons for manifest

When I remove gatsby-plugin-manifest, the build time was reduced to 52 seconds!

[pieh]

Can you share your code that cause this? I've never seen gatsby-plugin-manifest to take that long.

[missmatsuko]

Sure, this is my project: https://github.com/missmatsuko/matsuko-ca

EDIT: I might switch to eleventy. Here's the latest commit where I was still working on this: https://github.com/missmatsuko/matsuko-ca/tree/4920c3f18cef630d04ce1f0538c3af4e71fbe6a4

It doesn't take so long locally, I was told here that it was because we only have 1 CPU on Netlify: decaporg/gatsby-starter-decap-cms#191 (comment)

I could try reducing the size of the source SVG, I'm not sure if that would help. I made it have a large viewbox because very blurry images were produced when I used an SVG with a small viewbox (even though it's a vector/SVG, so it shouldn't matter).

[gatsbot]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

[moonmeister]

@missmatsuko I've never seen this plugin run for more that a couple seconds (usually <1s). Doesn't mean it's not causing you troubles, but this is definitely not typical.

Your troubles aside, this is an excellent suggestion and future i18n support from #13471 makes this more important. We're happy to take a PR to support this from you or anyone who would like to contribute.

[moonmeister]

Update: #13471 Implemented some rudimentary caching. This implementation does not work across builds but only in a single build. With the implementation of internationalization it was possible we would be regenerating a single image multiple of time. This caching creates a record of every image it builds and doesn't rebuild if it has already been created during that build cycle.

For anyone wanting to contribute this we're looking to implement caching across builds. meaning these images wouldn't be recreated every build. only if the src image changed, or the output is changed would we need to regenerate the images.

[gatsbot]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[moonmeister]

@LekoArts is there a reason not stale got removed?

[gatsbot]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community!