-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use graph for build-sequence #439
Comments
Another advantage is that it can allow us to implement some parallelism (relates to #65) by picking packages smartly and then building them in parallel |
+1 to using graph.json for build-sequence as it looks like it will improve the way we build wheels and make it more efficient. |
We can use topological sorting to get an ordering of the packages which will also help us identify packages that are independent of each other. Those packages can be built parallely. Additionally for certain heavy packages that take a lot of resources to build, we want to build them one at a time. We can store the time a package took to build in the graph as an indicator of a heavy resource package to ensure they are built one at a time. Alternatively, we could also look for certain files in the sdist of the package that would indicate whether they are going to be resource intensive packages (suggested by @tiran) |
The graph provides us a more accurate recipe of what needs to be built and how. Currently we use build-order.json file and build packages from top down and for each package we try to construct the build dependencies again using the
prepare_build_environment
function. If we use the graph then we already have this information and won't need to reproduce the build dependencies set again. We also have the exact version that we can use for each requirement instead of relying on the constraints file while installing these build dependencies. This will fix #391The text was updated successfully, but these errors were encountered: