Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updating master #781

Merged
merged 32 commits into from
Jan 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
76ea5f0
[transcribe] add FFmpeg package to requierments test file
aviaIguazio Dec 10, 2023
b035929
[transcribe] add FFmpeg package to requierments test file
aviaIguazio Dec 10, 2023
8e50b51
[CLI] add execute build commands from item yaml for specific testing
aviaIguazio Dec 13, 2023
a75ff9e
[CLI] add execute build commands from item yaml for specific testing
aviaIguazio Dec 13, 2023
6d98263
Merge remote-tracking branch 'upstream/development' into development
aviaIguazio Dec 20, 2023
4cb6ca7
Merge remote-tracking branch 'upstream/development' into development
aviaIguazio Dec 31, 2023
31b8138
[CLI] add state_thresholds support in item_to_function
aviaIguazio Dec 31, 2023
dabcd9e
[CLI] add state_thresholds support in item_to_function
aviaIguazio Dec 31, 2023
50b9787
[CLI] add state_thresholds support in item_to_function
aviaIguazio Dec 31, 2023
40386ec
[CLI] add state_thresholds support in item_to_function
aviaIguazio Dec 31, 2023
aa83330
Merge pull request #764 from aviaIguazio/development
aviaIguazio Jan 1, 2024
e176ee2
[Structured-data-generator] Fixed call of
yonishelach Jan 4, 2024
f85b60f
Merge pull request #767 from yonishelach/fix-structured
aviaIguazio Jan 4, 2024
95b7dc1
[pyannote_audio] Rename speech diarization to this new name (#766)
guy1992l Jan 9, 2024
8aa45c5
[Structured-data-generator] Fix get secret section (#768)
yonishelach Jan 9, 2024
bf5c665
[silero_vad] New function (#769)
guy1992l Jan 9, 2024
231fb7f
[silero_vad] Multiprocessing fix (#771)
guy1992l Jan 11, 2024
7ff1c92
[Pii-recognizer] Update the function to be compatible with the call c…
yonishelach Jan 14, 2024
63846a1
[Question-Answering] Update the function to be compatible with the ca…
yonishelach Jan 14, 2024
ee2136b
[Text-to-audio-generator] Make output_dir optional
yonishelach Jan 14, 2024
d826e35
[Transcribe] Moving from whisper to torch
yonishelach Jan 15, 2024
f31d643
remove project.yaml
yonishelach Jan 15, 2024
9a1db5d
debug
yonishelach Jan 15, 2024
4ad02e3
skip if ffmpeg is not installed
yonishelach Jan 15, 2024
f45dce6
Merge pull request #773 from yonishelach/pii-update
aviaIguazio Jan 15, 2024
1a57426
Merge pull request #774 from yonishelach/qa-update
aviaIguazio Jan 15, 2024
ae24f1e
Merge pull request #776 from yonishelach/transcribe-v2
aviaIguazio Jan 15, 2024
738f494
Merge pull request #775 from yonishelach/text-to-audio-update
aviaIguazio Jan 15, 2024
d0e34f0
Revert "[Question-Answering] Update the function to be compatible wit…
yonishelach Jan 16, 2024
ed77485
[Build] Restrict sphinxcontrib-applehelp in requirements (#778)
yonishelach Jan 16, 2024
9a33fff
[build] Restrict sphinxcontrib-devhelp in requirements (#779)
yonishelach Jan 16, 2024
0164e2a
[Build] restrict sphinxcontrib packages (#780)
yonishelach Jan 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cli/helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,8 @@ def get_item_yaml_values(
if values:
if isinstance(values, list):
values_set = set(values)
elif isinstance(values, dict):
values_set = values
else:
values_set.add(values)
values_dict[key] = values_set
Expand Down
31 changes: 16 additions & 15 deletions cli/item_to_function.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,17 +55,17 @@
help="If -b/--bump_version is enabled, increase the minor version in the item.yaml file",
)
def item_to_function_cli(
item_path: str, output_path: Optional[str], code_output: bool, format_code: bool, bump_version: bool
item_path: str, output_path: Optional[str], code_output: bool, format_code: bool, bump_version: bool
):
item_to_function(item_path, output_path, code_output, format_code, bump_version)


def item_to_function(
item_path: str,
output_path: Optional[str] = None,
code_output: bool = False,
format_code: bool = True,
bump_version: bool = False,
item_path: str,
output_path: Optional[str] = None,
code_output: bool = False,
format_code: bool = True,
bump_version: bool = False,
):
item_path = Path(item_path)
if item_path.is_dir():
Expand All @@ -78,9 +78,9 @@ def item_to_function(
# That means we need to search for items inside this direcotry
else:
for inner_dir in PathIterator(
root=item_path.parent,
rule=is_item_dir,
as_path=True,
root=item_path.parent,
rule=is_item_dir,
as_path=True,
):
try:
_output_path = output_path or (inner_dir / "function.yaml")
Expand Down Expand Up @@ -119,11 +119,11 @@ def _get_item_yaml(item_path: Path) -> dict:


def create_function_yaml(
item_path: Union[str, Path],
output_path: Optional[str] = None,
code_output: bool = False,
format_code: bool = True,
bump_version: bool = False,
item_path: Union[str, Path],
output_path: Optional[str] = None,
code_output: bool = False,
format_code: bool = True,
bump_version: bool = False,
):
item_path = Path(item_path)
if bump_version:
Expand Down Expand Up @@ -161,7 +161,8 @@ def create_function_yaml(
# remove build info from object
function_object.spec.build.code_origin = ''
function_object.spec.build.origin_filename = ''
function_object.spec.state_thresholds=None
if 'state_thresholds' not in spec:
function_object.spec.state_thresholds = None

custom_fields = spec.get("customFields", {})
for key, value in custom_fields.items():
Expand Down
117 changes: 58 additions & 59 deletions pii_recognizer/function.yaml

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion pii_recognizer/item.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,5 @@ spec:
- st-annotated-text
- https://huggingface.co/beki/en_spacy_pii_distilbert/resolve/main/en_spacy_pii_distilbert-any-py3-none-any.whl
url: ''
version: 0.1.0
version: 0.2.0
test_valid: False
Loading
Loading