docs: doc repro --glob and update targets arg info. #1983

skshetry · 2020-11-30T05:28:52Z

Fixes #1975. The feature was introduced in iterative/dvc#4976 which was released in 1.11.0.

content/docs/command-reference/repro.md

shcheklein

This looks great @skshetry ... let's just find a place to put it.

For now section will do, but we can consider doing something standard here ... any good ideas that will make it easier to navigate for users?

content/docs/command-reference/repro.md

shcheklein · 2020-12-01T04:50:36Z

content/docs/command-reference/repro.md

+  With `--glob`, the targets are used as a pattern to match stages in the
+  dvc.yaml file (example: `train-*`).
+
+  A stage from a dvc.yaml in a different directory can be specified using a path


I wonder if in this case it's easier to just show 5 or more examples with their meaning (instead of explaining : explicitly)?

Yeah, it might be too much to read if you just need to get it done. But, I think it's good to also layout the spec of it, so I added an example for each scenario.

yeah, spec is good, and I saw your examples (that's why I'm actually suggesting to do emphasis on them) the question is - should the spec be part of the examples explanation? this way most people with get the idea from what they see, almost immediately, and they can still read if needed.

That sounds good. Will make changes later in the day. Thanks.

easier to just show 5 or more examples with their meaning (instead of explaining :

it might be too much to read if you just need to get it done

It's good to have the spec thoough. I can probably help summarize this text when it's ready.

Consider instead adding an actual Example about wildcards in addition to this (and link to it from here).

content/docs/command-reference/repro.md

jorgeorpinel · 2020-12-01T17:48:16Z

CI fail is unrelated, should be fixed in master

This has been fixed. Please merge master, @skshetry.

content/docs/command-reference/repro.md

jorgeorpinel

~~I'd reorder these paragraphs a little so that everything --glob is together, like this~~

content/docs/command-reference/repro.md

jorgeorpinel

Thanks @skshetry this is really good.

~~Side-note guys: I'm a bit concerned about the --glob name for UX reasons. See iterative/dvc#4976 (comment).~~
Moved to iterative/dvc#4816 (comment)

jorgeorpinel · 2020-12-03T19:02:04Z

OK so it looks like we have all the info we need for this doc. The Q is whether you can help move stuff around and address some of the feedback above @skshetry please lmk so we can plan to finish this up when ready. Thanks

skshetry · 2020-12-09T06:42:08Z

@jorgeorpinel, this should be ready, I don't have anything to add.
I was trying to show examples before the spec explanation like how Ivan mentioned above, but I am not sure how to do that, so I'm skipping it.

content/docs/command-reference/repro.md

shcheklein · 2020-12-15T02:35:48Z

content/docs/command-reference/repro.md

 `dvc repro` does not run `dvc fetch`, `dvc pull` or `dvc checkout` to get data
 files, intermediate or final results (except if the `--pull` option is used).

-By default, this command checks all [pipeline](/doc/command-reference/dag)


all not relevant?

This paragraph is redundant (with the new targets text a well as the first p in the Description) and slightly wrong (but that will be corrected for #2024). Removing it aims to make the Description shorter so that the whole doc doesn't grow too much as we introduced the targets text.

But I can easily stash the change if you prefer and leave it for later. Lmk.

Yes, let's make PRs focused if possible.

I expect some changes here, but it probably should be only the paragraph about targets?

OK, I reverted this. I'm afraid that if we just keep adding paragraphs without reviewing the context we'll end up with lots of redundancy and a text that may be unnecessary long.

shcheklein · 2020-12-15T02:36:32Z

content/docs/command-reference/repro.md

- `-R`, `--recursive` - determines the stages to reproduce by searching each
-  target directory and its subdirectories for stages (in `dvc.yaml`) to inspect.
-  If there are no directories among the targets, this option is ignored.
+- `-R`, `--recursive` - looks for `dvc.yaml` files to reproduce in any


not relevant?

This is directly related to repro targets. It used to read "determines the stages ... by searching ... for stages". Isn't this a good opportunity to fix that? TBH I already have a mountain of stashed changes so anything I can't get into existing PRs will probably get lost until we happen to notice it again 🙁

content/docs/command-reference/repro.md

shcheklein · 2020-12-15T02:39:40Z

content/docs/command-reference/repro.md

+    using by using a colon `:` following the path to that file. E.g.
+    `models/dvc.yaml:prepare`
+
+  - Files and directories tracked by **`.dvc` files** given as `targets` are


let's skip this I think?

Yes! I would like that too but are you sure? The help output (and Synopsis above) mention .dvc files so if we don't explain it here, it'd be up to the user to assume what will happen.

Synopsis is a copy/paste from DVC which should probably change, so, let's not complicate. The point is not cover everything to the very small detail - the point is to focus on important stuff and convey it in the simplest possible format (e.g. people don't read long texts, unless they want to dive in)- thus short self-explainable examples are better

Agree. Removed info about .dvc file targets from this ref.

content/docs/command-reference/repro.md

per #1983 (review)

content/docs/command-reference/repro.md

per #1983 (review)

jorgeorpinel · 2020-12-17T19:36:56Z

content/docs/command-reference/repro.md

+  Different things can be provided as targets depending on the flags used (more
+  details in each option), namely:
+
+  - With `-R` you can provide directory paths to search for `dvc.yaml` files in,
+    recursively.
+  - With `--glob`, you can use special patterns (using wildcards) to match
+    groups of stage names.
+
+- `-R`, `--recursive` - looks for `dvc.yaml` files to reproduce in any
+  directories given as `targets`, and in their subdirectories. If there are no
+  directories among the targets, this option has no effect.
+
+- `--glob` - causes the `targets` to be interpreted as wildcard


Added -R and --glob to targets but also moved those options up right next to targets, so I still think it seems a bit redundant.

Suggestion:

Keep only

Different things can be provided as targets depending on the flags used (more details in each option), namely:

then a list of examples:

dvc reporo -R pipeline - will look recursively into ....

dvc reporo --glob ... - will execute stages that match pattern

dvc repro - will find dvc.yaml(s) and execute all stages

it's way simpler to grasp the idea, no need to read long long text (if some details are need let's try very hard to squeeze them into each item above)

reporo? 😆 Sounds funny

OK good idea, applied! PTAL.

Note that issue #1614 is pretty evident this way, should we prioritize it?

per #1983 (review)

skshetry · 2020-12-19T13:34:58Z

content/docs/command-reference/repro.md

+  depending on the flags used (more details in each option). Examples:
+
+  - `dvc repro linear/dvc.yaml`: Specific `dvc.yaml` file to reproduce
+  - `dvc repro -R pipelines/`: Directory path to explore recursively for for


Suggested change

- `dvc repro -R pipelines/`: Directory path to explore recursively for for

- `dvc repro -R pipelines/`: Directory path to explore recursively for

Oops thanks! Want to send a quick typo fix PR since this is merged?

skshetry added the A: docs Area: user documentation (gatsby-theme-iterative) label Nov 30, 2020

skshetry requested review from jorgeorpinel and shcheklein November 30, 2020 05:28

skshetry self-assigned this Nov 30, 2020

shcheklein temporarily deployed to dvc-landing-repro-glob-llcj9qf November 30, 2020 05:29 Inactive

skshetry commented Nov 30, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

shcheklein approved these changes Dec 1, 2020

View reviewed changes

shcheklein temporarily deployed to dvc-landing-repro-glob-llcj9qf December 1, 2020 04:43 Inactive

skshetry requested a review from shcheklein December 1, 2020 04:48

shcheklein reviewed Dec 1, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

shcheklein reviewed Dec 1, 2020

View reviewed changes

This comment has been minimized.

Sign in to view

shcheklein temporarily deployed to dvc-landing-repro-glob-llcj9qf December 1, 2020 05:37 Inactive

jorgeorpinel reviewed Dec 1, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

jorgeorpinel reviewed Dec 1, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

jorgeorpinel reviewed Dec 1, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

jorgeorpinel reviewed Dec 1, 2020

View reviewed changes

skshetry added 3 commits December 9, 2020 12:24

docs: add flag --glob for repro

ac71914

Move target details to Options

c78da03

codify dvc files

a4f8404

skshetry force-pushed the repro-glob branch from d45d215 to a4f8404 Compare December 9, 2020 06:39

This comment has been minimized.

Sign in to view

jorgeorpinel requested a review from shcheklein December 15, 2020 02:32

shcheklein reviewed Dec 15, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

shcheklein reviewed Dec 15, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

shcheklein reviewed Dec 15, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

Merge branch 'master' into repro-glob

a34e74d

shcheklein reviewed Dec 16, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

shcheklein reviewed Dec 16, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

jorgeorpinel mentioned this pull request Dec 16, 2020

docs: adjust help for config #2022

Merged

jorgeorpinel added 6 commits December 16, 2020 20:49

cmd: roll back correction in repro

822576a

per #1983 (review)

cmd: revert moving repro -p

9be4347

per #1983 (review)

cmd: revert moving text about multiple commands in repro

1c85228

per #1983 (review)

cmd: remove info about .dvc files from repro

474a2a6

per #1983 (review)

cmd: simplify repro targets section and reorder related options

9734ec6

per #1983 (review)

cmd: typo and roll backs in repro

6e87280

jorgeorpinel reviewed Dec 17, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

jorgeorpinel requested a review from shcheklein December 17, 2020 19:26

cmd: correct targets list in repro -P

10aedde

per #1983 (review)

jorgeorpinel reviewed Dec 17, 2020

View reviewed changes

cmd: rewrite targets arg as a list of examples

ebb2feb

per #1983 (review)

shcheklein approved these changes Dec 19, 2020

View reviewed changes

shcheklein merged commit f641643 into master Dec 19, 2020

skshetry deleted the repro-glob branch December 19, 2020 05:47

skshetry commented Dec 19, 2020

View reviewed changes

skshetry mentioned this pull request Sep 23, 2022

doc: add doc about foreach target #3989

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: doc repro --glob and update targets arg info. #1983

docs: doc repro --glob and update targets arg info. #1983

skshetry commented Nov 30, 2020 •

edited

Loading

shcheklein left a comment

shcheklein Dec 1, 2020

skshetry Dec 1, 2020

shcheklein Dec 1, 2020

skshetry Dec 1, 2020

jorgeorpinel Dec 1, 2020 •

edited

Loading

This comment has been minimized.

jorgeorpinel commented Dec 1, 2020

jorgeorpinel left a comment •

edited

Loading

jorgeorpinel left a comment •

edited

Loading

jorgeorpinel commented Dec 3, 2020

skshetry commented Dec 9, 2020

This comment has been minimized.

shcheklein Dec 15, 2020

jorgeorpinel Dec 16, 2020 •

edited

Loading

jorgeorpinel Dec 16, 2020 •

edited

Loading

shcheklein Dec 16, 2020

shcheklein Dec 16, 2020

jorgeorpinel Dec 17, 2020

shcheklein Dec 15, 2020

jorgeorpinel Dec 17, 2020

shcheklein Dec 15, 2020

jorgeorpinel Dec 16, 2020

shcheklein Dec 16, 2020

jorgeorpinel Dec 17, 2020 •

edited

Loading

jorgeorpinel Dec 17, 2020

shcheklein Dec 18, 2020

jorgeorpinel Dec 19, 2020

skshetry Dec 19, 2020

jorgeorpinel Dec 20, 2020

	- `dvc repro -R pipelines/`: Directory path to explore recursively for for
	- `dvc repro -R pipelines/`: Directory path to explore recursively for

docs: doc repro --glob and update targets arg info. #1983

docs: doc repro --glob and update targets arg info. #1983

Conversation

skshetry commented Nov 30, 2020 • edited Loading

shcheklein left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorgeorpinel Dec 1, 2020 • edited Loading

Choose a reason for hiding this comment

This comment has been minimized.

jorgeorpinel commented Dec 1, 2020

jorgeorpinel left a comment • edited Loading

Choose a reason for hiding this comment

jorgeorpinel left a comment • edited Loading

Choose a reason for hiding this comment

jorgeorpinel commented Dec 3, 2020

skshetry commented Dec 9, 2020

This comment has been minimized.

Choose a reason for hiding this comment

jorgeorpinel Dec 16, 2020 • edited Loading

Choose a reason for hiding this comment

jorgeorpinel Dec 16, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorgeorpinel Dec 17, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

skshetry commented Nov 30, 2020 •

edited

Loading

jorgeorpinel Dec 1, 2020 •

edited

Loading

jorgeorpinel left a comment •

edited

Loading

jorgeorpinel left a comment •

edited

Loading

jorgeorpinel Dec 16, 2020 •

edited

Loading

jorgeorpinel Dec 16, 2020 •

edited

Loading

jorgeorpinel Dec 17, 2020 •

edited

Loading