Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE][BUGFIX] Beautiful subscriptions syntax update + fix #758

Merged
merged 9 commits into from
Oct 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 13 additions & 9 deletions docs/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -309,12 +309,12 @@ For example:
.. code-block:: yaml
:caption: subscription.yaml

tv_show:
only_recent:
[News]:
TV Show Full Archive:
= News:
"Breaking News": "https://www.youtube.com/@SomeBreakingNews"

[Tech]:
TV Show Only Recent:
= Tech | TV-Y:
"Two Minute Papers": "https://www.youtube.com/@TwoMinutePapers"

Will create two subscriptions named "Breaking News" and "Two Minute Papers", equivalent to:
Expand All @@ -323,8 +323,7 @@ Will create two subscriptions named "Breaking News" and "Two Minute Papers", equ

"Breaking News":
preset:
- "tv_show"
- "only_recent"
- "TV Show Full Archive"

overrides:
subscription_indent_1: "News"
Expand All @@ -333,25 +332,30 @@ Will create two subscriptions named "Breaking News" and "Two Minute Papers", equ

"Two Minute Papers":
preset:
- "tv_show"
- "TV Show Only Recent"

overrides:
subscription_indent_1: "Tech"
subscription_indent_2: "TV-Y"
subscription_name: "Two Minute Papers"
subscription_value: "https://www.youtube.com/@TwoMinutePapers"

You can provide as many parent presets in the form of keys, and subscription indents as ``[keys]``.
You can provide as many parent presets in the form of keys, and subscription indents as ``=keys``.
This can drastically simplify subscription definitions by setting things like so in your
parent preset:

.. code-block:: yaml

presets:
tv_show_name:
"TV Show Preset":
overrides:
subscription_indent_1: "default-genre"
subscription_indent_2: "default-content-rating"

tv_show_name: "{subscription_name}"
url: "{subscription_value}"
genre: "{subscription_indent_1}"
content_rating: "{subscription_indent_2}"

.. _subscription value:

Expand Down
23 changes: 9 additions & 14 deletions examples/tv_show_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ configuration:
presets:

# Your main TV show preset - all your tv show subscriptions will use this.
tv_show:
"TV Show Full Archive":
preset:
# Choose one of the following player types:
# - "kodi_tv_show_by_date"
Expand All @@ -41,10 +41,8 @@ presets:

# Include any of the presets listed below in your 'main preset' if you want
# it applied to every TV show. Or, use them on the individual subscriptions.
# - "only_recent_videos"
# - "add_subtitles"
# - "sponsorblock"
# - "include_info_json"

# To download age-restricted videos, you will need to uncomment and set your cookie
# file here as a ytdl parameter. For more info, see
Expand All @@ -62,8 +60,12 @@ presets:

####################################################################################################

# Preset to only download and keep recent videos
only_recent_videos:
# A secondary TV Show preset that only keeps recent videos
"TV Show Only Recent":

# Inherit the `TV Show Full Archive` preset
preset:
- "TV Show Full Archive"

# Only download videos within the download_range
date_range:
Expand All @@ -89,7 +91,8 @@ presets:
# subtitles_name: "{episode_file_path}.{lang}.{subtitles_ext}"
# subtitles_type: "srt"

languages: "en" # supports list of multiple languages
languages:
- "en"
allow_auto_generated_subtitles: True # allow auto subtitles

####################################################################################################
Expand All @@ -113,11 +116,3 @@ presets:
- "music_offtopic"
remove_sponsorblock_categories: "all"
force_key_frames: False

####################################################################################################

# Preset for the hoarders who want to also save the info.json file
include_info_json:
output_options:
info_json_name: "{episode_file_path}.{info_json_ext}"

46 changes: 33 additions & 13 deletions examples/tv_show_subscriptions.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,36 @@
# Define any number subscriptions in a single file, or use multiple
# files to organize (i.e by genre)
# Global overrides for all subscriptions. Can either be set here or in your config.yaml
__preset__:
overrides:
tv_show_directory: "/tv_shows" # Root folder of ytdl-sub TV Shows
download_range: "2months" # Range to keep for only recent

rick_a:
# Mix and match any number of presets to use. You can either lump all settings into
# a single preset to save some lines, or modularize them and use many.
preset:
- "tv_show"
- "only_recent_videos"

# All subscriptions under this will use the `TV Show Full Archive` preset
TV Show Full Archive:
# Sets "Music" for genre
= Music:
"Rick A": "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
"Opeth": "https://www.youtube.com/channel/UCmQSJTFZaXN85gYk6W3XbdQ"

# Sets "Kids" for genre, "TV-Y" for content rating
= Kids | TV-Y:
"Jake Trains": "https://www.youtube.com/@JakeTrains"
"Kids Toys Play": "https://www.youtube.com/@KidsToysPlayChannel"


# All subscriptions under this will use the `TV Show Only Recent` preset
TV Show Only Recent:
= News | TV-14:
"BBC": "https://www.youtube.com/@BBCNews"


# ADVANCED USAGE:
# Subscriptions can use the same format as a preset found within a config.
"Equivalent to BBC":
preset:
- "TV Show Only Recent"
overrides:
# Required to specify for every subscription
tv_show_name: "Rick A"
url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
# Optional
tv_show_genre: "Music"
tv_show_name: "BBC"
url: "https://www.youtube.com/@BBCNews"
tv_show_genre: "News"
tv_show_content_rating: "TV-14"
7 changes: 5 additions & 2 deletions src/ytdl_sub/prebuilt_presets/tv_show/episode.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,11 @@ presets:
# season_number_padded
# episode_number
# episode_number_padded
tv_show_genre: "ytdl-sub"
tv_show_content_rating: "TV-14"
subscription_indent_1: "ytdl-sub" # genre
subscription_indent_2: "TV-14" # content rating
tv_show_name: "{subscription_name}"
tv_show_genre: "{subscription_indent_1}"
tv_show_content_rating: "{subscription_indent_2}"
season_title: "{season_number_padded}"
episode_title: "{upload_date_standardized} - {title}"
episode_plot: "{webpage_url}\n\n{description}"
Expand Down
1 change: 1 addition & 0 deletions src/ytdl_sub/prebuilt_presets/tv_show/tv_show.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ presets:
- url: "{url19}"
- url: "{url20}"
overrides:
url: "{subscription_value}"
url2: ""
url3: ""
url4: ""
Expand Down
5 changes: 4 additions & 1 deletion src/ytdl_sub/subscriptions/subscription.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import copy
from pathlib import Path
from typing import Any
from typing import Dict
from typing import List
Expand Down Expand Up @@ -92,7 +93,9 @@ def _maybe_get_subscription_value(
return subscription_value_key

@classmethod
def from_file_path(cls, config: ConfigFile, subscription_path: str) -> List["Subscription"]:
def from_file_path(
cls, config: ConfigFile, subscription_path: str | Path
) -> List["Subscription"]:
"""
Loads subscriptions from a file and applies ``__preset__`` to all of them if present.
If a subscription is in the form of key: value, it will set value to the override
Expand Down
32 changes: 14 additions & 18 deletions src/ytdl_sub/subscriptions/subscription_validators.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,16 @@ def subscription_value_variable_name() -> str:
return "subscription_value"


def maybe_indent_override_value(value: str) -> Optional[str]:
def maybe_indent_override_values(value: str) -> List[str]:
"""
Returns
-------
Value if it is an overide [Value]. None otherwise.
"""
if value.startswith("[") and value.endswith("]"):
return value[1:-1]
return None
if value.startswith("="):
# Drop the =, split on |, and strip each indent_value (both left + right)
return [indent_value.strip() for indent_value in value[1:].split("|")]
return []


class SubscriptionOutput(Validator, ABC):
Expand Down Expand Up @@ -112,24 +113,19 @@ def __init__(
f"{self._leaf_name} conflicts with an existing preset name and cannot be "
f"used as a subscription name"
)

if subscription_value is None:
raise self._validation_exception(
f"Subscription {self._leaf_name} is a string, but the subscription value "
f"is not set to an override variable"
)

self._subscription_value: str = subscription_value
self._subscription_value: Optional[str] = subscription_value

def subscription_dicts(self) -> Dict[str, Dict]:
subscription_value_dict: Dict[str, str] = {"subscription_value": self.value}
# TODO: Eventually delete in favor of {subscription_value}
if self._subscription_value:
subscription_value_dict[self._subscription_value] = self.value

return {
self._leaf_name: {
"preset": self._presets,
"overrides": dict(
{
self._subscription_value: self.value,
"subscription_value": self.value,
},
subscription_value_dict,
**self._indent_overrides_dict(),
),
}
Expand Down Expand Up @@ -179,14 +175,14 @@ def __init__(
subscription_value=subscription_value,
)
)
elif override_value := maybe_indent_override_value(key):
elif override_values := maybe_indent_override_values(key):
self._children.append(
SubscriptionValidator(
name=obj_name,
value=obj,
config=config,
presets=presets,
indent_overrides=indent_overrides + [override_value],
indent_overrides=indent_overrides + override_values,
subscription_value=subscription_value,
)
)
Expand Down
5 changes: 5 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,11 @@ def channel_as_tv_show_config(working_directory) -> ConfigFile:
)


@pytest.fixture()
def tv_show_subscriptions_path() -> Path:
return Path("examples/tv_show_subscriptions.yaml")


@pytest.fixture()
def music_audio_config(working_directory) -> ConfigFile:
return _load_config(
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e/plugins/test_date_range.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
@pytest.fixture
def recent_preset_dict(output_directory):
return {
"preset": "tv_show",
"preset": "TV Show Full Archive",
"date_range": {"after": "20150101"},
"format": "worst[ext=mp4]", # download the worst format so it is fast
"ytdl_options": {
Expand Down
2 changes: 1 addition & 1 deletion tests/e2e/youtube/test_channel.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
@pytest.fixture
def channel_preset_dict(output_directory):
return {
"preset": "tv_show",
"preset": "TV Show Full Archive",
"format": "worst[ext=mp4]", # download the worst format so it is fast
"ytdl_options": {
"max_views": 100000, # do not download the popular PJ concert
Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
{
".ytdl-sub-split_by_chapters_video-download-archive.json": "a4337aeeaded1a1ef5d4c17d8e5ed71f",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/01 - 01. Intro (Feat. Racheal Ofori & Barney Artist.mp3": "73a98c0f110519e42a5ef6629fd5c84d",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/02 - 02. Answers (Feat. Rick David & Kaya Thomas - Dyke.mp3": "37b52830e5860546548c1f1c73433c9f",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/03 - 03. Blaze (Feat. Kaya Thomas - Dyke.mp3": "44859cc006906ad20da3b5cc966636a3",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/04 - 04. What If (Interlude.mp3": "1e81ccbb8ca22e2a9ff7b7dc0e2264fe",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/05 - 05. No Peace (Feat. Tom Misch.mp3": "959ce7ebe3e39680c36c9e4b0472e138",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/06 - 06. Closer (Feat. Lester Duval.mp3": "f812f8614d95dfa535e2049d6344e0a8",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/07 - 07. Delusions: Rumination (Interlude) (Feat. Racheal Ofori.mp3": "d704c2e5c4c26ca11007c62880aa2f6e",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/08 - 08. Dreams (Feat. Carmody.mp3": "94cd88bc220e77239ac02affd3e676f9",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/09 - 09. Dreaming (Interlude) (Feat. Racheal Ofori.mp3": "d379a7f5d14dcc9432447a1751a14179",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/10 - 10. Hopeful (Feat. Jordan Rakei.mp3": "bd63f123d442166784df208b09e9c6b7",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/11 - 11. Sunrise (Pillows) (Feat. Emmavie.mp3": "7b5725a2d452d0457179084ce5d1c34e",
".ytdl-sub-split_by_chapters_video-download-archive.json": "c3fb0b4f31caaa10ac7954ea93da33c4",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/01 - 01. Intro (Feat. Racheal Ofori & Barney Artist).mp3": "8827ac62201abeb94f44a46d91628aa8",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/02 - 02. Answers (Feat. Rick David & Kaya Thomas - Dyke).mp3": "f17e02119bf073c4ed98da2a960b85d3",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/03 - 03. Blaze (Feat. Kaya Thomas - Dyke).mp3": "63c0bc97cbdaf35203011378b1860418",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/04 - 04. What If (Interlude).mp3": "eb1a8533390243a5c4c49d8b6871c0ce",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/05 - 05. No Peace (Feat. Tom Misch).mp3": "a1b54772207228bd4922ad93d249c9f5",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/06 - 06. Closer (Feat. Lester Duval).mp3": "1d217264368d5f078e67691ea1b3df97",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/07 - 07. Delusions: Rumination (Interlude) (Feat. Racheal Ofori).mp3": "4d1a1728dbd69a30bbc2ffa354851169",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/08 - 08. Dreams (Feat. Carmody).mp3": "c204cf16f9f44945634c6171a5192e99",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/09 - 09. Dreaming (Interlude) (Feat. Racheal Ofori).mp3": "795e22297f8db00697563774328ffb3a",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/10 - 10. Hopeful (Feat. Jordan Rakei).mp3": "3904a5cd81b5ac43850b7858fb8dd378",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/11 - 11. Sunrise (Pillows) (Feat. Emmavie).mp3": "a89549db5b16ff76d04fd0bbbfba3424",
"Proved Records/[2017] Alfa Mist - Nocturne [Full Album]/folder.jpg": "bd3685acc53072e591bae2505ecb0648"
}
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
{
".ytdl-sub-split_by_chapters_with_regex_video-download-archive.json": "d13f803a92fe9630d5e6fa6f1baa46a9",
"Alfa Mist/[2017] Nocturne/01 - Intro (Feat. Racheal Ofori & Barney Artist.mp3": "98c381825f8e0358711c21d084bc3568",
"Alfa Mist/[2017] Nocturne/02 - Answers (Feat. Rick David & Kaya Thomas - Dyke.mp3": "8c971f3f0ef230797b033043b49a8888",
"Alfa Mist/[2017] Nocturne/03 - Blaze (Feat. Kaya Thomas - Dyke.mp3": "50e74885a2a19a11420dc8c27b96b7bd",
"Alfa Mist/[2017] Nocturne/04 - What If (Interlude.mp3": "d655ca75ba791731c6c49b022188bb79",
"Alfa Mist/[2017] Nocturne/05 - No Peace (Feat. Tom Misch.mp3": "14e8492c8784913a0acd3be9a496ad5d",
"Alfa Mist/[2017] Nocturne/06 - Closer (Feat. Lester Duval.mp3": "2371ae9951b04b436ac2994aed6684cb",
"Alfa Mist/[2017] Nocturne/07 - Delusions: Rumination (Interlude) (Feat. Racheal Ofori.mp3": "644754ff7bcf926b34109f5d263a21b7",
"Alfa Mist/[2017] Nocturne/08 - Dreams (Feat. Carmody.mp3": "d9d6a014e453e3c6ede1544f3ec1bb03",
"Alfa Mist/[2017] Nocturne/09 - Dreaming (Interlude) (Feat. Racheal Ofori.mp3": "ea833ef0f0af9c347d3f5920899b8b44",
"Alfa Mist/[2017] Nocturne/10 - Hopeful (Feat. Jordan Rakei.mp3": "714fae9edfc7844411984e664ced7285",
"Alfa Mist/[2017] Nocturne/11 - Sunrise (Pillows) (Feat. Emmavie.mp3": "871d94d7e402cc3a29e80a5150735873",
".ytdl-sub-split_by_chapters_with_regex_video-download-archive.json": "9798e8289742586d0efd295a97c6c906",
"Alfa Mist/[2017] Nocturne/01 - Intro (Feat. Racheal Ofori & Barney Artist).mp3": "c10cff170976c3c5139f6073eeb3e25e",
"Alfa Mist/[2017] Nocturne/02 - Answers (Feat. Rick David & Kaya Thomas - Dyke).mp3": "881ecdbee4a0d565c0ec6eacf03115df",
"Alfa Mist/[2017] Nocturne/03 - Blaze (Feat. Kaya Thomas - Dyke).mp3": "aa5c6fbab58dd2be2660bfb65be6a172",
"Alfa Mist/[2017] Nocturne/04 - What If (Interlude).mp3": "a64ce02ed1188276c9744f92cfc81e8e",
"Alfa Mist/[2017] Nocturne/05 - No Peace (Feat. Tom Misch).mp3": "a6dbe96141515caa32f3df4f66e46f75",
"Alfa Mist/[2017] Nocturne/06 - Closer (Feat. Lester Duval).mp3": "9b8ff5357109c376cce4bdd25e724d7b",
"Alfa Mist/[2017] Nocturne/07 - Delusions: Rumination (Interlude) (Feat. Racheal Ofori).mp3": "022b077892b838c478c65ba3b5f00b55",
"Alfa Mist/[2017] Nocturne/08 - Dreams (Feat. Carmody).mp3": "75c9c0f789c378c45bb176aa6cc6ec57",
"Alfa Mist/[2017] Nocturne/09 - Dreaming (Interlude) (Feat. Racheal Ofori).mp3": "af59a3ac869249ac6ef2586fa6c59009",
"Alfa Mist/[2017] Nocturne/10 - Hopeful (Feat. Jordan Rakei).mp3": "27d36f90dc87264c57e88f3663b29467",
"Alfa Mist/[2017] Nocturne/11 - Sunrise (Pillows) (Feat. Emmavie).mp3": "be7085f9a536dfd2f70ce001c65a416c",
"Alfa Mist/[2017] Nocturne/folder.jpg": "bd3685acc53072e591bae2505ecb0648"
}
Loading
Loading