Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stage: list/show/get stage details #10179

Closed
dberenbaum opened this issue Dec 16, 2023 · 3 comments
Closed

stage: list/show/get stage details #10179

dberenbaum opened this issue Dec 16, 2023 · 3 comments
Labels
A: api Related to the dvc.api A: cli Related to the CLI A: pipelines Related to the pipelines feature

Comments

@dberenbaum
Copy link
Collaborator

dberenbaum commented Dec 16, 2023

CLI command and API method to get all stage info (cmd, params, deps, outs, etc.).

  • CLI command is useful for seeing the resolved info for stages using interpolation
  • API method is useful for getting paths so you don't have to manually align them between code and dvc.yaml

Loosely related to iterative/vscode-dvc#5048:

With a complicated DVC pipeline, with dynamic parametrized dependencies it's not easy to get an exact command that is needed to run a specific stage under debugger outside of DVC.

Example:

$ cat dvc.yaml
stages:
  train:
    cmd: python src/train.py ${data_path}/features ${model_path} ${train}
    deps:
    - ${data_path}/features
    - src/train.py
    params:
    - train
    outs:
    - ${model_path}
    - dvclive:
        cache: false

$ dvc stage show train --json
{
  "cmd": "python src/train.py data/features model.pkl min_split=0.2 n_est=50 seed=101",
  "deps": [
    "data/features",
    "model.pkl"
  ],
  "params": {
    "train": {
      "min_split": 0.2,
      "n_est": 50,
      "seed": 101,
    }
  },
  "outs": {
    "model.pkl": {},
    "dvclive": {"cache": false}
  }
}

API usage:

from dvc.api import stage_show

outs = stage_show("train").outs
with open(outs[0].path, "w") as f:
    f.write(model)
@dberenbaum dberenbaum added A: api Related to the dvc.api A: cli Related to the CLI A: pipelines Related to the pipelines feature labels Dec 16, 2023
@dberenbaum dberenbaum changed the title dvc stage list/show/get stage details stage: list/show/get stage details Dec 16, 2023
@dberenbaum
Copy link
Collaborator Author

@iterative/vs-code How can we show the resolved stages in the extension so it's easy to edit and debug dvc.yaml?

@julieg18
Copy link

julieg18 commented Jan 2, 2024

Maybe we could show stages in a tree in a sidebar view.

image

@dberenbaum
Copy link
Collaborator Author

@julieg18 I was initially thinking along the lines of the ideas in iterative/vscode-dvc#1192, but maybe a dedicated sidebar view like you suggest is better 🤔.

@dberenbaum dberenbaum added the p1-important Important, aka current backlog of things to do label Jan 9, 2024
@dberenbaum dberenbaum added this to DVC Jan 9, 2024
@github-project-automation github-project-automation bot moved this to Backlog in DVC Jan 9, 2024
@dberenbaum dberenbaum removed the p1-important Important, aka current backlog of things to do label Mar 4, 2024
@dberenbaum dberenbaum closed this as not planned Won't fix, can't repro, duplicate, stale Mar 4, 2024
@github-project-automation github-project-automation bot moved this from Backlog to Done in DVC Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: api Related to the dvc.api A: cli Related to the CLI A: pipelines Related to the pipelines feature
Projects
No open projects
Archived in project
Development

No branches or pull requests

2 participants