Suggestion: Adding additional 3rd party CDN

[mark-galvin]

Adding additional 3rd party Content Delivery Network (CDN).

Because of the recent issues with CDNJS I'm looking into adding additional CDN to Libman. I was thinking of adding the following functionalities:

1. Google CDN
2. The ability to add custom CDN's by URL
3. Adding fallback CDN

Should I create my own fork or can I contribute these features here?

[jimmylewis]

Can you give a little more explanation of how you're looking to augment? Are you proposing a new provider (which would require a catalog behavior, for discovery of library names, versions, and files)? Or augmenting the existing provider with fallbacks for downloading files?

[mark-galvin]

Hi Jimmy,

I'm looking for both. To provide the ability to bring in a new provider, then as another feature to fall back to another provider if the files are not available. Full disclosure I haven't looked at the codebase yet. So I'm not sure the lift. What I can do is look at the code and get back to you maybe next week so I can talk more intelligently about it?
-m

[michaelswells]

If I have a share,
\server
|-myPathToTheLibraryVersion
|-0.0.1
|-0.0.2

what about
{
"provider": "URL",
"library": "https://myURL/[email protected]",
"destination": "myLibrary/",
"files":[] //optional
}

Yes, I know about filesystem but AFAIK, it requires me to have one entry per file instead of just using the folder

[CrazyPyro]

@jimmylewis , seconded - It would be very handy to be able to point at internal Gitlab/Nexus/etc repos. Many organizations maintain "blessed" CDNs that mirror only the latest approved packages, to help with bandwidth and compliance.

[jimmylewis]

A few thoughts/responses:

fall back to another provider if the files are not available

This one is tricky because it's quite common that the files are not the same across providers. For example, jsdelivr provides .min.js and .min.css files for any .js or .css file in the library - even if the package owner doesn't publish it to NPM. Using unpkg as a fallback won't work for these, because they only exist from the JSDelivr provider. I would also expect similar from cdnjs, which primarily only provides the dist files for a library, not the original sources; again, it wouldn't be a suitable fallback for files from jsdelivr or unpkg depending on which assets you're consuming. And a 3rd example in this case is that the file paths are inconsistent across some providers as well: once again, cdnjs has simpler file paths (e.g. d3.min.js) vs the others (e.g. dist/d3.min.js). Remapping all of these would be a considerable effort and is unfortunately not feasible to maintain.

what about
{
"provider": "URL",
"library": "https://myurl/[email protected]",
"destination": "myLibrary/",
"files":[] //optional
}

This looks really appealing, but given a URL, libman cannot determine the list of files available - either for the file preview, or for actually downloading the library. As it is, the filesystem provider can take a URL to a specific file, but you need a separate library for each file defined this way. It could work to have a URL provider where the "files" are required, but that still creates ambiguity between a single file and a directory if the user doesn't fully specify all the information - it seems like a usability issue. It could be addressed with extra effort (e.g. special validation for this provider), but certainly would need to be taken into account.

It would be very handy to be able to point at internal Gitlab/Nexus/etc repos...

I think a big challenge here is implementing something that can authenticate with each of these systems, but also handles credentials securely. Currently libman does allow storing a password for the proxy server, but that setting is stored on the local machine and only works on Windows (the .NET Framework API used is not present on other platforms, or in newer versions of .NET).