-
-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize fetching of tensorflow header files #104
Conversation
Rather than cloning the whole tensorflow repository, which is rather large, clone only the header files. Accomplished by utilizing sparse checkout feature. During testing cloning tensorflow like this was over 5 times faster than the old approach. It can be optimized further by limiting which folders to grab header files from too.
WalkthroughThe recent modifications involve integrating additional Git commands into both the Dockerfile and Makefile. These commands are specially designed to set up a sparse checkout process for TensorFlow headers. This adjustment aims to streamline the download and retrieval stages, making them more efficient and less resource-intensive by focusing only on the necessary parts of the TensorFlow repository. Changes
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
Review Status
Configuration used: CodeRabbit UI
Files selected for processing (2)
- Dockerfile (1 hunks)
- Makefile (1 hunks)
Additional comments not posted (2)
Dockerfile (1)
34-37
: Consider refining the sparse checkout pattern to be more specific to the required TensorFlow header files, if a thorough analysis of dependencies allows for it. This could further optimize the fetching process by reducing the amount of data cloned even more. However, ensure that all necessary dependencies are included to avoid build issues.Makefile (1)
36-39
: Consider refining the sparse checkout pattern in the Makefile to be more specific to the required TensorFlow header files, if a thorough analysis of dependencies allows for it. This could further optimize the fetching process by reducing the amount of data cloned even more. However, ensure that all necessary dependencies are included to avoid build issues.
Rather than cloning the whole tensorflow repository, which is rather large, clone only the header files. Accomplished by utilizing sparse checkout feature.
During testing, cloning tensorflow like this was over 5 times faster than the old approach. It can be optimized further by limiting which folders to grab header files from too. However, currently I am not too familiar with how the header files are dependent on each-other, so I decided not to go for the extreme "tensorflow/lite/**/*.h" pattern which I was able to build and run the application with, since some of the headers there had dependencies to tensorflow/core.
Summary by CodeRabbit