v0.10.2

[0.10.2] - 2023-11-06

Added

• gbasf2: Local basf2 log level setting is now passed over to the grid jobs. You can now limit the log size of jobs with many warnings via basf2.set_log_level(basf2.LogLevel.ERROR). This could fix some errors due to too large log sizes. Implemented by pickling local basf2.logging.log_level.

Fixed

• gbasf2: Fixes to make gbasf2 wrapper work with gbasf2 release v5.8.2:

  • Change the default gbasf2 setup script path to CVMFS location in gbasf2 v5.8.2, i.e.

    /cvmfs/belle.kek.jp/grid/gbasf2/pro/bashrc
    

    Reminder that this can still be customized via the gbasf2_setup_path setting.
    #207
  • Fix to gbasf2-interacting scripts for querying project status and listing datasets. #209 @eckerpatrick and #210 @0ctagon.

• gbasf2: Fix gbasf2_setup_path setting not being passed through in some function calls. #203 @AlexanderHeidelbach

Removed

• gbasf2: Fully deprecate gbasf2_install_directory setting. It will be ignored from now on and a warning given if used. Instead please use the gbasf2_setup_path setting introduced in v0.10.1 to provide the exact path to the gbasf2 setup script. gbasf2_install_directory will not be used as a fallback anymore as was the case in v0.10.1.

New Contributors

• Tristan Fillinger @0ctagon

Full Changelog: v0.10.1...v0.10.2

v0.10.1

[0.10.1] - 2023-04-17

Added

• gbasf2: New setting gbasf2_setup_path which can be used to customize the path to the gbasf2 setup file directly (default: "/cvmfs/belle.kek.jp/grid/gbasf2/pro/tools/setup.sh"). It is a more flexible replacement for the gbasf2_install_directory setting, which will be removed in the future, since we can't predict potential name and path changes of the setup script between gbasf2 releases. @meliache #162

Fixed

• gbasf2: Fix the issues caused by gbasf2 release v5r7 #197. Thanks to @MarcelHoh.

• gbasf2: #197 also includes the removal of the --new flag for gb2_ds_get when proxy group is not belle, as then the downloaded directory structure is different from what b2luigi expects. This is a hotfix, in the future we should aim to always use the new download style, therefore issue #200 was opened.

• gbasf2: Switch to the --new flag in gb2_ds_get which downloads files significantly faster than previously. Gbasf2 release v5r6 (November 2022) is required. #190.

Full Changelog: v0.10.0...v0.10.1-rc1

v0.10.0

Changed

• For local basf2 versions, change how hash for basf2_release Parameter is calculated. Now use basf2 functionality to get the version, to be consistent with the output of basf2 --version. The new hash encodes both the local and central basf2 release, the basf2 function getCommitID. When basf2 is not set up, print warning before returning "not_set". Thanks to @GiacomoXT in #193.

  Warning: If you use local basf2 versions, that is your basf2_release is a git hash, this will change your b2luigi target output paths. This means that tasks that were marked complete, might suddenly not be complete anymore after updating to this release. A workaround is to check for the new expected path via python3 <steering_fname>.py --show_output and rename the git_hash=<…> directory.

• Apply max_events Parameter not by changing the environment singleton, but instead forward it to basf2.process call. This should hopefully not affect the behaviour in practice. Also by @GiacomoXT in #193

• Refactor the basf2 related examples to use more idiomatic, modern basf2 code, e.g. using basf2.Path() instead of basf2.create_path(). . Also by @GiacomoXT in #193

Fixed

• Fix example SimulationTask task in basf2_chain_example.py, which probably wasn't working as it was missing the Geometry module. Also by @GiacomoXT in #193

Full Changelog: v0.10.0...main

v0.9.1

[0.9.1] - 2023-03-20

Fixed

• Fix circular import #188

Added

• Add the ability to pass a custom hashing function to parameters via the hash_function keyword argument. The function must take one argument, the value of the parameter. It is up to the user to ensure unique strings are created. #189
• gbasf2: Switch to the --new flag in gb2_ds_get which downloads files significantly faster than previously. Gbasf2 release v5r6 (November 2022) is required. #190.

Full Changelog: v0.9.0...v0.9.1

v0.9.0

Fixed

• gbasf2: Fix bug introduced in #181 when generating basf2 queries with just a simple .root extension, raising a wrong false positive errors. Now moved splitting functionality into separate function and added extensive unit tests. Thanks @schmitca for reporting #184.

Added

• task_iterator now returns a unique list of tasks. The task graph is a DAG which is traversed through recursion in task_iterator like a tree. If multiple tasks had the same task as a requirement (i.e. multiple nodes share a child), it was returned multiple times in the task iterator. This results in performance improvements when checking the requirements. #186. Thanks @MarcelHoh for the initial PR.

Full Changelog: v0.8.2...v0.9.0

v0.8.2

Fixed

• gbasf2: Fix gbasf2 glob queries (e.g. for downloading) for basf2 output files with multiple extensions, e.g. <file>.udst.root <file>.mdst.root. #180. Thanks @schmitca for reporting, reviewing and testing.

Full Changelog: v0.8.1...v0.8.2

v0.8.1

Fixed

• gbasf2: Fix ioctl error in gb2_proxy_init by reading in password via b2luigi and then supplying password to that command directly, instead of letting gb2_proxy_init handle the password prompt. #172 @Bilokin

Added

• gbasf2: Add gbasf2_proxy_group and gbasf2_project_lpn_path parameters to switch between gbasf2 groups. #175 @Bilokin
• add automatic "needs changelog" PR labeller as github workflow #166

Changed

• Update pre-commit hooks. Most notably for the developers, update the flake8 syntax and style-checker to version 5.0.4, which might change slightly what style is accepted. This should also fix an issue with the old flake8 version not being compatible with the latest version of importlib_meta, which the pre-commit flake8 hook in the github actions to fail. In the process also migrated the pre-commit config format to the new layout.

You can also find the changelog in the CHANGELOG.md.

For all commits between releases see: v0.7.6...v0.8.1.

v0.7.6

Fixed

• htcondor: Make HTCondorProcess.get_job_status a method again instead. It was turned into a property accidentally in #158. See issue #164 @eckerpatrick and PR @165 @mschnepf.

Full Changelog: v0.7.5...v0.7.6

v0.7.5

Added

• htcondor: Do up to 3 retries for getting job status with condor_q #155
• gbasf2: Add caching and unit tests to get_dirac_user #156
• Add @mschnepf to the contributors for #158
• Some minor documentation improvements #151 and typo fix in help message #153.

Fixed

• gbasf2: Adapt to new file name for gbasf2 setup file (setup → setup.sh) #160
• gbasf2: Ensure proxy is initalized before running get_proxy_info to get dirac user #156
• htcondor: Don't fail when htcondor job status is suspended or transferring_output #158. Thanks to @mschnepf 🙇.

Changed

• gbasf2: Use retry2 package for retrying getting of gbasf2 project status instead of my own recursive loop #161

Full Changelog: v0.7.4...v0.7.5

v0.7.4

[0.7.4] - 2021-11-03

Added

• Add a CHANGELOG.md file in addition to the release notes on github

Fixed

• gbasf2: Fix moving of downloaded datasets with multiple datablocks (subs) #150
• gbasf2: If an error happens during proxy initialization, there was an error raised, but the stderr argument was wrong, which was fixed in #149

Changed

• gbasf2: get_unique_lfns in some cases returned a set and in some cases a list. Changed it to always return sets.