Mock-up/Prototype representing a still image service for a video and extracting a still

[jronallo]

Description

We want to be able to advertise a service which can extract still images from a video. We need more examples of what this video still service might look like. We could use other implementations for doing extracting images from video as well.

Proposed Solutions

In a prototype I created (in https://github.com/NCSU-Libraries/eyebright) here's an example of what this could look like. This is based on an example @azaroth42 mocked-up: https://gist.github.com/azaroth42/c850a409edc512be5465d15c3a2562ee

{"thumbnail": {
    "@id": "https://iiif-staging02.lib.ncsu.edu/iiifvi/pets/2/full/full/0/default.jpg",
    "@type": "Image",
    "format": "image/jpeg",
    "width": 720,
    "height": 480,
    "service": {
        "@context": "http://iiif.io/api/image/2/context.json",
        "@id": "https://iiif-staging02.lib.ncsu.edu/iiifvi/pets",
        "profile": "http://iiif.io/api/image/2/level2.json"
    }
}

Based on that service you can request a still image with a URL beginning like the following and ending the with the Image API parameters:
https://iiif-staging02.lib.ncsu.edu/iiifvi/pets/time-slice/

So a full example of requesting an image at the 2 second mark:
https://iiif-staging02.lib.ncsu.edu/iiifvi/pets/2/full/full/0/default.jpg

Or using a more exact time notation:
https://iiif-staging02.lib.ncsu.edu/iiifvi/pets/00:00:14.875/full/full/0/default.jpg

The issue with doing it this way is that there's no indication that this particular image service requires a time parameter in order to get an image back.

The variation suggested at the meeting was the the service should instead encode the time slice into the identifier allowing it to be used just how the Image API is used now. So in that case I suppose that the service id would be something like one of the following:
https://iiif-staging02.lib.ncsu.edu/iiifvi/pets/00:00:14.875
https://iiif-staging02.lib.ncsu.edu/iiifvi/pets,00:00:14.875
https://iiif-staging02.lib.ncsu.edu/iiifvi/pets00:00:14.875
https://iiif-staging02.lib.ncsu.edu/iiifvi/pets,2
https://iiif-staging02.lib.ncsu.edu/iiifvi/pets2

The issue with doing it this way is that you're only advertising the image service as available for a particular time slice and not that a service is available that can provide a still image from multiple times or any arbitrary time in the video.

[zimeon]

For specification of time notation it seems that following https://www.w3.org/TR/media-frags/#naming-time would be good. Example above seems to do that using npt-hhmmss form.

[jronallo]

A couple questions arise when specifying a video still service:

• How would we advertise a level 0 service where still images are pre-computed? Would we list out all the times available and their sizes? For a long video this could be a very long list.
• How would implementers avoid allowing for requests for an infinite number of images based on slightly different times? Time is noted as hh:mm:ss.decimal-fraction where the decimal fraction is potentially unbounded.
• Some already create images every N seconds and others use key frames or scene detection to create thumbnails. How can we represent both?

See our call notes for more information on the discussion around video stills: https://docs.google.com/document/d/1INONT-hgxPJ9zWK1P0CrBxVVwelg4uRbOqanw5ZzMng/edit#

[azaroth42]

I think that this is out of scope for 0.1 given the discussions in the hague that all we want to do is to refer to existing derivatives.

[jronallo]

I'm not sure that's all we want to do, but using existing derivatives is certainly one of the things we need to allow to be done. Even then you want to be able to advertise which time slices you would make available. From my testing it also seems to be feasible to dynamically extract images from videos with reasonable performance. I'd be fine with not working on this right now, but a video still service seems to follow quickly on after getting information about the videos.

[azaroth42]

It seems very inconsistent to do some dynamic extraction and not other dynamic extraction ... particularly when we have many use cases for the dynamic extraction that we're not doing!

I would feel much more comfortable either delaying all of it or none of it. Performance of current tool chains really shouldn't be much of a driver so long as there's the possibility to trade off space for computation (as discussed).