Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip EnsureRenderedStringPattern preprocessor if parameter has verbatim directives #19

Merged
merged 2 commits into from
Jan 8, 2019
Merged

Skip EnsureRenderedStringPattern preprocessor if parameter has verbatim directives #19

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c143e86
Skip 'EnsureRenderedStringPattern' preprocessor and warn if parameter…
Jan 8, 2019
6fd01f0
Address mchalek's comments
Jan 8, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions boundary_layer_default_plugin/preprocessors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,14 @@ def process_arg(self, arg, node, raw_args):
regex.pattern)
return arg

VERBATIM_REGEX = '<<item_name>>|<<item.name>>|<<item.value>>'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey so one feature that I would like to support is the ability to actually require that the argument be a (or contain a) verbatim string... For example, there are some operators that take methods as arguments, and the only way we can support those right now is for the parameter passed to the operator to be a lambda (at the moment we do not enforce any restrictions on these, but we probably should).

So could we do the VERBATIM_REGEX check only if the regex.match() check fails? Maybe change the code to something like (starting at line 114 and replacing all of the rest of the method):

if regex.match(rendered_arg):
    return arg

VERBATIM_REGEX = ...
if re.compile(VERBATIM_REGEX).search(rendered_arg):
    logger.warning(...)
    return arg

raise Exception('Invalid argument...')

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh and also, I think we can loosen the definition of the VERBATIM_STRING regex to something like <<.+>>, because generally speaking it's possible for any verbatim string to be present, and we can't check any of them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comments. Done!

if re.compile(VERBATIM_REGEX).search(rendered_arg):
logger.warning(
'Argument generated from `%s` may not match the required pattern `%s` and fail.',
rendered_arg,
regex.pattern)
return arg

if not regex.match(rendered_arg):
raise Exception(
'Invalid argument `{}`: does not match expected pattern `{}`'.format(
Expand Down
18 changes: 18 additions & 0 deletions test/default_plugin/test_preprocessors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,21 @@ def test_ensure_rendered_string_pattern(mocker):
) == "my-cluster-{{unknown_var.task_id}}"
warning_mock.assert_called_once()


warning_mock = Mock()
mocker.patch.object(logger, 'warning', new=warning_mock)
assert renderer.process_arg(
"my-<<item_name>>-cluster", None, {},
) == "my-<<item_name>>-cluster"
warning_mock.assert_called_once()

warning_mock = Mock()
mocker.patch.object(logger, 'warning', new=warning_mock)
assert renderer.process_arg(
"my-cluster-<<item.value>>", None, {},
) == "my-cluster-<<item.value>>"
warning_mock.assert_called_once()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice tests, thanks!


with pytest.raises(Exception):
renderer.process_arg("my-cluster-", None, {})

Expand All @@ -46,3 +61,6 @@ def test_ensure_rendered_string_pattern(mocker):
"my-cluster-{{execution_date.strftime('%Y_%m_%d')}}",
None,
{})

with pytest.raises(Exception):
renderer.process_arg("My-<<non-item.value>>-cluster", None, {})