Demo harness: 3+ viewers using a common JSON annotation format

[JulieWinchester]

This is a task to create a milestone technical set of experiments that relates to user story #14, possibly others.

Acceptance criteria:

• Establish a common JSON format that specifies two simple label annotations on a 3D model
• Create a set of 3+ viewer demos with each viewer loading the same 3D model (astronaut GLB) and the 2 label annotations

Current progress:

• Currently, we have 5 working demos (Aleph, Google Model Viewer, Sketchfab, Smithsonian Voyager, and X3D)
• See VIEWER_JSON_DEMOS.md in the repo for links and further details

Progress needed:

• X3D demo should load second annotation point and should load from external JSON file
• Sketchfab demo should be based in code sandbox and should load standard annotations from external JSON file
• All demos should be made more comparable for future work

After these points are finished, this task can potentially be closed and further work described as a new task (possibly in a GitHub project).

[JulieWinchester]

Acceptance criteria for initial demos:

• Example demo using a viewer, perhaps in code sandbox, but could use other environments
• Demo should load the astronaut GLB
• Demo should create 2 label annotations:
  • Label 'visor' on the astronaut's visor (should add specific coordinates on visor)
  • Label 'glove' on the astronaut's left glove (should add specific coordinates on left glove)
• Important: the annotations should be loaded from a JSON-format object, whether fetched as a .json file (and applied on initial load) or as input from a text field box (and applied dynamically in response to button input)

[JulieWinchester]

Of the existing demos, the Google Model Viewer demo needs an additional glove annotation label. The X3D demo needs an additional glove annotation label, and to load the annotations from JSON input.

[JulieWinchester]

Here is the JSON currently used by the Google Model Viewer demo. Need to add glove annotation and change face mask label to "visor".

[
  {
    "id":1,
    "normal":"0.29259561389217825 0.11383937564155769 0.9494358342113489",
    "position":"0.0652451665004884 1.8182700174153779 0.32465106720430725",
    "value":"face mask"
  }
]

[edsilv]

Updated model-viewer demo: https://codesandbox.io/s/model-viewer-annotations-demo-3k5tqo

[JulieWinchester]

Perfect, thanks! I'm also adding the new JSON used by the updated demo:

[
    {
        "id": 0,
        "normal": "0.29259561389217825 0.11383937564155769 0.9494358342113489",
        "position": "0.017023790299119268 1.8062894401300538 0.34094835034109217",
        "value": "visor"
    },
    {
        "id": 1,
        "normal": "0.47124483745139667 0.05608789306010443 0.8802172751244348",
        "position": "0.5175891304384852 0.9555791975125096 0.12190797586809543",
        "value": "glove"
    }
]

[JulieWinchester]

I've also updated #18, but here is the Aleph viewer demo loading the same annotation information as the other two examples. Specifically, it loads an annotation for the visor and the glove on the Astronaut 3D model. It loads the annotations from an external JSON file, as well.

https://codesandbox.io/s/aleph-annotation-demo-teh3pf?file=/index.html

{
  "nodes": [
    [
      "Visor",
      {
        "normal": "0.29259561389217825 0.11383937564155769 0.9494358342113489",
        "position": "0.017023790299119268 1.8062894401300538 0.34094835034109217",
        "scale": 0.025,
        "title": "Visor"
      }
    ],
    [
      "Glove",
      {
        "normal": "0.47124483745139667 0.05608789306010443 0.8802172751244348",
        "position": "0.5175891304384852 0.9555791975125096 0.12190797586809543",
        "scale": 0.025,
        "title": "Glove"
      }
    ]
  ]
}

It's worth noting the format is different for Aleph compared to Google Model Viewer, but with the exception of the scale tag, the information is equivalent and interchangeable between those two viewers.

[gjcope]

I've created a Voyager version of the demo here: https://codesandbox.io/s/voyager-annotations-demo-o9l1rq?file=/index.html

The relevant JSON for the annotations is as follows:

"annotations": [
  {
      "id": "qs8H9mXZSr8p",
      "titles": {
          "EN": "visor"
      },
      "leads": {
          "EN": ""
      },
      "position": [
          -0.0142194,
          0.783568,
          0.3500529
      ],
      "direction": [
          -0.2855858,
          0.1212762,
          0.9506486
      ],
      "scale": 0.0603153,
      "color": [
          0,
          0.61,
          0.87
      ]
  },
  {
      "id": "J2gO1sGNobxk",
      "titles": {
          "EN": "glove"
      },
      "leads": {
          "EN": ""
      },
      "position": [
          0.5040075,
          -0.0485156,
          0.1340355
      ],
      "direction": [
          0.3971733,
          0.2949306,
          0.8690623
      ],
      "scale": 0.08,
      "color": [
          0,
          0.61,
          0.87
      ]
  }
]

[JulieWinchester]

A similar (but not exactly the same) SketchFab demo has been contributed via the IIIF 3D Slack: https://jsfiddle.net/nebulousflynn/uykbgjaw/1/

[JulieWinchester]

I have created VIEWER_JSON_DEMOS.md in the associated repository to better track the growing number of demos and updates to the demos over time. I've modified the top post as well, which should be up to date as of this date of this comment.

[Cook4986]

Thanks for the thorough documentation, Julie. Here are a few initial observations:

Aleph

• Annotation anchors (node spheres) color coding may be useful for categorizing sub-component types but the lack of visible connection between the ball and the text might confuse viewers when the number and proximity of annotations begin to increase. Occluded node silhouettes may require explanation as they look like annotations on opposite surfaces.

Google Model Viewer

• Same issue as Aleph, regarding the gap between node anchor and a given text string, a quirk that could cause problems with increased annotation density. Annotations disappear too quickly on rotation (90 degrees off-axis?). @edsilv , I'm looking at the setAnnotations function to modify...

Smithsonian

• The node spheres are lacking but text boxes are anchored directly to the surface of the object, making placements/associations easily discernable. Fading annotation text gives a subtle depth cue. Annotation toggle would benefit from an index/list (drop-down?).

Sketchfab

• Out of the box, Sketchfab annotations aren't compatible with camera limits, a feature we use regularly, "except via the Viewer API". Otherwise, Sketchfab (like Aleph) does a good job keeping occluded annotations onscreen.

X3D

• As above, the link between the annotation text and the surface of the object is valuable. In some positions, the text panel intersects the mesh geometry, obscuring portions of the object. Seeing this demo with multiple annotations will be helpful.

There were no obvious performance issues with any of the viewer tests. Regarding JSON formatting, the Aleph and Voyager key headers ("nodes", "annotations") benefit from human readability. I'll be curious to see the Sketchfab and X3D JSON.

Please let me know if this thread isn't the place for such feedback and I'll move these comments to one of our IIIF TSG Google docs.

[JulieWinchester]

As tracked in the Annotation Demo project and especially in #25, this task is now successfully complete. Our markdown document describing JSON viewer demos has been updated to reflect latest work. Thanks all!