Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Video Inference #200

Merged
merged 18 commits into from
Nov 3, 2023
Merged

Feature: Video Inference #200

merged 18 commits into from
Nov 3, 2023

Conversation

capjamesg
Copy link
Collaborator

Description

This PR adds support for video inference to the Roboflow Python package.

Type of change

  • New feature (non-breaking change which adds functionality)

How has this change been tested, please provide a testcase or example of how you tested the change?

Will add a description once the draft PR is complete.

Any specific deployment considerations

N/A

Docs

We will publish:

  1. A page on docs.roboflow.com that describes how video inference works.
  2. A blog post announcing video inference with an example.
  3. Pricing information.

@capjamesg capjamesg added the enhancement New feature or request label Oct 27, 2023
@capjamesg capjamesg requested a review from bigbitbus October 27, 2023 21:26
@capjamesg capjamesg self-assigned this Oct 27, 2023
@bigbitbus bigbitbus requested a review from PacificDou October 30, 2023 14:05
>>> prediction = model.predict("video.mp4", fps=5, inference_type="object-detection"
"""

url = urljoin(API_URL, "/video_upload_signed_url/?api_key=", self.__api_key)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this API will only return a signed_url for uploading, and user still need to upload the video by himself, right ? @bigbitbus

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the user can use that returned value to upload a video to that signed url


data = response.json()

if data["success"] != 0 or data["status_info"] != "success":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if int(data['status']) == 0:
success
elif int(data['status']) == 1:
still running
else:
failed....will retry it if int(data['retries']) <= 3


attempts += 1

if response != {}:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest moving this check before sleep

>>> prediction = model.predict("video.mp4", fps=5, inference_type="object-detection"
"""

url = urljoin(API_URL, "/video_upload_signed_url/?api_key=", self.__api_key)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the user can use that returned value to upload a video to that signed url

url = urljoin(API_URL, "/video_upload_signed_url/?api_key=", self.__api_key)

if fps > 5:
raise Exception("FPS must be less than or equal to 5.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets set this to a higher value, up to the frame rate of the video itself.

@PacificDou lets add logic in the video server to use fps = min(video_fps,s infer_fps); and set this actual value into the database object returned to the user when they query the videojob.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The user needs to know that the FPS can be upto the video frame rate, but given the fps-based pricing, set this value to as low as makes sense to their app. Maybe make it part of the blogpost?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great if the server raised an exception if FPS is higher. Otherwise, if the user provides a video URL we have to download it in the package to check its FPS before we can do anything.

Copy link
Contributor

@bigbitbus bigbitbus Nov 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current behavior is that if the video frame-rate < infer_fs, the job fails and the user gets a helpful error message (this check happens when the video finally gets downloaded into the server and processing starts). Unfortunately probing a video to get its true video framerate is not possible at the time the request is made, its part of a batch operation.

@capjamesg capjamesg merged commit 080d588 into main Nov 3, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants