Wrapping media processing libraries

[daenney]

First of, apologies for raising this as an issue but I couldn't find a good place to reach out about this.

I'm one of the GoToSocial developers (an activitypub server in Go) and on principle we don't use CGo anywhere in order to simplify self-hosting (we can ship a static binary). We recently undertook integrating @ncruces go-sqlite3 as an alternative to the C-to-Go transpiled SQLite driver and we've been very happy with the result. That's what brought the wasilibs organisation to our attention.

One other area where CGo is helpful is image and video manipulation (transcoding, wiping out certain metadata etc.). So far we've implemented native solutions in Go for certain formats to do some basic metadata redaction but we don't support some image and video formats. Keeping up with this is hard too and many image, video and container formats have big and dense specifications.

What I've been wondering is if you believe it might be possible to take tools like ffmpeg and imagemagick and make them accessible through WASM? Both tools have some library dependencies of their own for various formats too.

I want to stress here that I'm not asking for anyone to do the work. I'm only trying to understand if embarking on this might actually lead to a good outcome or if there's fundamental issues that make it infeasible for this kind of library and utility to be used through WASM with the current state of affairs.

[ncruces]

Quick answer.

It's certainly possible, it's all about effort.

For image processing I'd start by looking at libraries on the wazero users page:
https://wazero.io/community/users/

[ncruces]

For video, I'd start with FFmpeg-WASI.

For that approach, maybe this package can be an inspiration. It just runs the dcraw CLI, compiled to WASI, and wraps running some commands into "nice" function APIs. It's pretty dumb, but it works.

For more effort, porting rust-ffmpeg-wasi to Go+wazero is likely a better bet.

[ncruces]

For image metadata, I'm very partial to ExifTool.

Ideally, the folks at webassembly-language-runtimes would do a Perl, and I'd give a shot at converting go-exiftool to it.