Skip to content

Commit

Permalink
Allow empty items to be cached (#8890)
Browse files Browse the repository at this point in the history
<!-- Raise an issue to propose your change
(https://github.com/cvat-ai/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution guide](https://docs.cvat.ai/docs/contributing/).
-->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
<!-- Why is this change required? What problem does it solve? If it
fixes an open
issue, please link to the issue here. Describe your changes in detail,
add
screenshots. -->
Details are described
[here](cvat-ai/cvat_enterprise#252).
### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [ ] I submit my changes into the `develop` branch
- [ ] I have created a changelog fragment <!-- see top comment in
CHANGELOG.md -->
- [ ] I have updated the documentation accordingly
- [ ] I have added tests to cover my changes
- [ ] I have linked related issues (see [GitHub docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [ ] I have increased versions of npm packages if it is necessary

([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning),

[cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning)
and

[cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning))

### License

- [ ] I submit _my code changes_ under the same [MIT License](
https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Bug Fixes**
- Improved cache error handling by introducing a specific exception for
empty cache scenarios
- Enhanced data validation when creating cache items to prevent caching
of invalid or empty data

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
  • Loading branch information
Marishka17 authored Jan 6, 2025
1 parent 76de839 commit 30fbfb2
Showing 1 changed file with 30 additions and 17 deletions.
47 changes: 30 additions & 17 deletions cvat/apps/engine/cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -218,17 +218,19 @@ def _create_and_set_cache_item(
item_data = create_callback()
item_data_bytes = item_data[0].getvalue()
item = (item_data[0], item_data[1], cls._get_checksum(item_data_bytes), timestamp)
if item_data_bytes:
cache = cls._cache()
with get_rq_lock_for_job(
cls._get_queue(),
key,
):
cached_item = cache.get(key)
if cached_item is not None and timestamp <= cached_item[3]:
item = cached_item
else:
cache.set(key, item, timeout=cache_item_ttl or cache.default_timeout)

# allow empty data to be set in cache to prevent
# future rq jobs from being enqueued to prepare the item
cache = cls._cache()
with get_rq_lock_for_job(
cls._get_queue(),
key,
):
cached_item = cache.get(key)
if cached_item is not None and timestamp <= cached_item[3]:
item = cached_item
else:
cache.set(key, item, timeout=cache_item_ttl or cache.default_timeout)

return item

Expand Down Expand Up @@ -353,11 +355,18 @@ def _make_frame_context_images_chunk_key(self, db_data: models.Data, frame_numbe
def _to_data_with_mime(self, cache_item: _CacheItem) -> DataWithMime: ...

@overload
def _to_data_with_mime(self, cache_item: Optional[_CacheItem]) -> Optional[DataWithMime]: ...
def _to_data_with_mime(
self, cache_item: Optional[_CacheItem], *, allow_none: bool = False
) -> Optional[DataWithMime]: ...

def _to_data_with_mime(self, cache_item: Optional[_CacheItem]) -> Optional[DataWithMime]:
def _to_data_with_mime(
self, cache_item: Optional[_CacheItem], *, allow_none: bool = False
) -> Optional[DataWithMime]:
if not cache_item:
return None
if allow_none:
return None

raise ValueError("A cache item is not allowed to be None")

return cache_item[:2]

Expand Down Expand Up @@ -385,7 +394,8 @@ def get_task_chunk(
return self._to_data_with_mime(
self._get_cache_item(
key=self._make_chunk_key(db_task, chunk_number, quality=quality),
)
),
allow_none=True,
)

def get_or_set_task_chunk(
Expand Down Expand Up @@ -413,7 +423,8 @@ def get_segment_task_chunk(
return self._to_data_with_mime(
self._get_cache_item(
key=self._make_segment_task_chunk_key(db_segment, chunk_number, quality=quality),
)
),
allow_none=True,
)

def get_or_set_segment_task_chunk(
Expand Down Expand Up @@ -510,7 +521,9 @@ def remove_context_images_chunks(self, params: Sequence[dict[str, Any]]) -> None
self._bulk_delete_cache_items(keys_to_remove)

def get_cloud_preview(self, db_storage: models.CloudStorage) -> Optional[DataWithMime]:
return self._to_data_with_mime(self._get_cache_item(self._make_preview_key(db_storage)))
return self._to_data_with_mime(
self._get_cache_item(self._make_preview_key(db_storage)), allow_none=True
)

def get_or_set_cloud_preview(self, db_storage: models.CloudStorage) -> DataWithMime:
return self._to_data_with_mime(
Expand Down

0 comments on commit 30fbfb2

Please sign in to comment.