Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Camera Class #2257

Open
tomcrane opened this issue Jul 14, 2023 · 7 comments
Open

Introduce Camera Class #2257

tomcrane opened this issue Jul 14, 2023 · 7 comments
Labels
3d presentation Ready-for-Eds Editorial changes ready for Editorial review

Comments

@tomcrane
Copy link
Contributor

See IIIF/3d#15 (comment) from Napoli

The publisher will often (usually?) wish to indicate what the user / view is looking at when the Scene initializes. The publisher may also wish to provide additional specific views.

Cameras are added to the scene as annotations, just as model content resources are. They target a point.

  • Are cameras added as painting annotations and therefore under the items property of the Scene?
  • Or do they have a different motivation and live under the annotations property of the Scene?
  • Or is there a cameras property of the Scene, a separate list of annotations (with motivation…)

The camera’s view is determined by two properties of the Camera class, lookAt and fieldOfView
(split these into individual issues? Probably not, need to be considered together)

lookAt

(from Rob’s draft)

The property's value is either a Selector or a Model instance that has been annotated into the Scene with the camera. The Selector will describe a polygon or point within the Scene, such as a PointSelector instance that gives the x,y and z coordinates.

For the Model or polygonal Selector, the client should treat this as a PointSelector at the origin of the model or polygon. If the Model which the Camera is configured to look at is not, in fact, annotated into the Scene, then the client SHOULD treat it as if the lookAt property was not defined.

If not defined, then the client SHOULD calculate the bounding cube of all content in the Scene, and look at the origin (0,0,0) of that cube. If there are no contents in the Scene, the camera will look at the origin of the Scene itself.

fieldOfView

“defined as a floating point degree value similar to Three.js”

See https://threejs.org/docs/#api/en/cameras/PerspectiveCamera.fov

Camera Annotation Example

This shows two cameras in the Scene’s items property, as painting annotations:

"items": [
    {
        "id": "https://example.org/iiif/3d/anno1",
        "type": "Annotation",
        "motivation": [
            "painting"
        ],
        "body": {
            "type": "Camera",
            "lookAt": {
                "id": "https://example.org/iiif/selectors/1",
                "type": "PointSelector",
                "x": 0.0,
                "y": 0.0,
                "z": 0.0
            },
            "fieldOfView": 50.0
        },
        "target": [
            {
                "type": "SpecificResource",
                "source": [
                    {
                        "id": "https://example.org/iiif/scene1/page/p1/1",
                        "type": "Scene"
                    }
                ],
                "selector": [
                    {
                        "type": "PointSelector",
                        "x": 100.0,
                        "y": 100.0,
                        "z": 0.0
                    }
                ]
            }
        ]
    },
    {
        "id": "https://example.org/iiif/3d/anno1",
        "type": "Annotation",
        "motivation": [
            "painting"
        ],
        "body": {
            "type": "Camera",
            "lookAt": {
                "id": "https://example.org/iiif/3d-models/spaceman-1.obj",
                "type": "Model"
            },
            "fieldOfView": 35.0
        },
        "target": {..} // similar syntax as above
    }
]
@azaroth42
Copy link
Member

I think they're painted into the scene, as they're "of" the scene, rather than "about" the scene. Lights should be the same.
Similarly, a model format that allows a camera to be part of the model would be a painting annotation ... and if a format allows a camera to be the ONLY thing in the model, that would be the equivalent of the above JSON.

@mikeapp
Copy link
Member

mikeapp commented Jul 20, 2023

  • We would want to allow for a rotation transform, we might want a tilt the camera.
  • I think lookAt should be an optional, we should allow positioning of the camera via PointSelector and rotation. For example, an IIIF authoring tool might allow the user to "fly" the camera to a desired position. One could calculate an arbitrary point for lookAt, but that doesn't seem useful, unless we need to have a point to allow an "orbit" view mode?

@tomcrane
Copy link
Contributor Author

tomcrane commented Oct 25, 2023

Updated example with rotation:

The third annotation has no lookAt but has a position and a rotation. In this case, what is being rotated - where did it start?

{
  "items": [
    {
      "id": "https://example.org/iiif/3d/anno1",
      "type": "Annotation",
      "motivation": ["painting"],
      "body": {
        "type": "Camera",
        "lookAt": {
          "id": "https://example.org/iiif/selectors/1",
          "type": "PointSelector",
          "x": 0.0,
          "y": 0.0,
          "z": 0.0
        },
        "fieldOfView": 50.0
      },
      "target": [
        {
          "type": "SpecificResource",
          "source": [
            {
              "id": "https://example.org/iiif/scene1/page/p1/1",
              "type": "Scene"
            }
          ],
          "selector": [
            {
              "type": "PointSelector",
              "x": 100.0,
              "y": 100.0,
              "z": 0.0
            }
          ]
        }
      ]
    },
    {
      "id": "https://example.org/iiif/3d/anno1",
      "type": "Annotation",
      "motivation": ["painting"],
      "body": {
        "type": "Camera",
        "lookAt": {
          "id": "https://example.org/iiif/3d-models/spaceman-1.obj",
          "type": "Model"
        },
        "fieldOfView": 35.0
      },
      "target": [{ "// as above": "" }]
    },
    {
      "id": "https://example.org/iiif/3d/anno3",
      "type": "Annotation",
      "motivation": ["painting"],
      "body": {
        "type": "SpecificResource",
        "source": [
          {
            "id": "camera-sr-1",
            "type": "Camera",
            "fieldOfView": 65.0
          }
        ],
        "transforms": [
          {
            "type": "RotationTransform",
            "x": 0.0,
            "y": 90.0,
            "z": 0.0
          }
        ]
      },
      "target": [
        {
          "type": "SpecificResource",
          "source": [
            {
              "id": "https://example.org/iiif/scene1/page/p1/1",
              "type": "Scene"
            }
          ],
          "selector": [
            {
              "type": "PointSelector",
              "x": 100.0,
              "y": 100.0,
              "z": 0.0
            }
          ]
        }
      ]
    }
  ]
}

@JulieWinchester
Copy link
Contributor

JulieWinchester commented Nov 3, 2023

At the Basel working meeting on October 26, 2023, general consensus was present for the following:

  • Cameras should be placed as annotations like lights. Camera position should be specifiable by a PointSelector, like a model annotation. Camera orientation/facing should be specified by a DirectionTransform. For lights, DirectionTransform should take either a unit-length vector or a Scene reference target combined with a PointSelector. (Julie's note: should cameras be required to specify a Scene reference and PointSelector and not be allowed to use a vector orientation?)
  • Aspect ratio, near, and far values should be left up to implementation.
  • Field of view should be specifiable for the camera annotation as a floating point number. By convention and analogous to some other viewers like Meshlab, FOV=0 will be considered orthogonal perspective.
  • One or more cameras may be active at the same time, with the expectation that the client creates one viewport per camera.
  • For a single viewport with a single camera, the user should be able to orbit that camera around its look-at target point (at least in some cases, maybe this should be able to be turned off on the Scene level).
  • Only cameras in the main Scene are usable. Cameras in nested sub-scenes and cameras specified in models are treated as if they do not exist.
  • It should be possible to specify multiple cameras with durations associated with each as a list of annotations, in order to have a viewer switch between cameras at different time intervals.
  • It should be possible to specify multiple cameras that a user can select between as a single annotation with a Choice of cameras, in which selecting a camera turns on the selected camera and turns off the previously selected camera.

@JulieWinchester
Copy link
Contributor

Question arising from TSG meeting on November 15 2023: What happens (default expectation) when multiple camera annotations are present in a single manifest? Is it choice, or is it multiple viewports? This has implications on importing cameras from sub-manifests or scene files, if we were to possibly want to support somehow pulling those in?

@azaroth42
Copy link
Member

IMO:

  • Multiple camera annotations merely calls into existence multiple cameras. A rendering application can choose how to handle those, including (a) treat as a choice, and the user has to select between them (b) treat as a list, and generate multiple viewports, (c) ignore all but the first (whatever that is), (d) something else [e.g. two cameras for a stereoscopic view]. I think choice is a reasonable default but not the only possibility
  • The question of 'what is the camera's initial rotation / focal length' is a good one. I think it should look at the origin, but another option would be to look at its own x,y where z=0. That would likely make rotation calculation easier as it's starting from 0s rather than an angle

@tomcrane
Copy link
Contributor Author

Discussion:

Camera(s) are optional.

Where will the viewer put the camera if you don't provide one?

What is the initial state of a camera placed in the scene?
What do you have to provide?

If we are rotating it what are we rotating it from?

Can we have a "show-all" - viewer works out where to put the camera to see everything in the scene.

@azaroth42 azaroth42 added the Ready-for-Eds Editorial changes ready for Editorial review label Jan 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3d presentation Ready-for-Eds Editorial changes ready for Editorial review
Projects
None yet
Development

No branches or pull requests

4 participants