Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High disk space usage due to CodeSnapshotCallback saving all git-tracked files #63

Open
XinYu-Andy opened this issue May 18, 2023 · 4 comments
Labels
enhancement New feature or request

Comments

@XinYu-Andy
Copy link

Hi, thank you for your well-structured code. I've noticed that each time it runs, it defaults to backing up the code from the current run to the experiment storage files, seemingly excluding the "outputs" folder. However, if I change exp_root_dir in the config file, say, to "exp_results", it causes a loop that stores the results in this folder, leading to an exponential increase in memory usage. Could you advise me on how I should modify the code?

@XinYu-Andy
Copy link
Author

Seems I can manually change .gitignore to avoid it.

@bennyguo
Copy link
Collaborator

@XinYu-Andy Yes the CodeSnapshotCallback saves all git-tracked files (except for load/). This could be a problem, I'll reopen this issue and mark it as "enhancement". Thanks for reporting!

@bennyguo bennyguo reopened this May 19, 2023
@bennyguo bennyguo added the enhancement New feature or request label May 19, 2023
@claforte claforte changed the title Possible Bug High disk space usage due to CodeSnapshotCallback saving all git-tracked files Jun 3, 2023
@claforte
Copy link
Collaborator

claforte commented Jun 3, 2023

Not sure if this bug has been addressed... can we close it?

@thuliu-yt16
Copy link
Collaborator

Not yet and no elegant method has been thought of yet

Not sure if this bug has been addressed... can we close it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants