Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Setup scripts #12321

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

refactor: Setup scripts #12321

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/linux-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ on:
- "CMake/**"
- "third_party/**"
- "scripts/setup-ubuntu.sh"
- "scripts/setup-common.sh"
- "scripts/setup-versions.sh"
- "scripts/setup-helper-functions.sh"
- ".github/workflows/linux-build.yml"
- ".github/workflows/linux-build-base.yml"
Expand All @@ -36,6 +38,8 @@ on:
- "CMakeLists.txt"
- "CMake/**"
- "third_party/**"
- "scripts/setup-common.sh"
- "scripts/setup-versions.sh"
- "scripts/setup-ubuntu.sh"
- "scripts/setup-helper-functions.sh"
- ".github/workflows/linux-build.yml"
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/macos.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ jobs:
env:
HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK: "TRUE"
run: |
source scripts/setup-macos.sh
scripts/setup-macos.sh
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will run the full script and not just the functions below. The purpose of this was to get most dependencies from the bundle and not rely on the system installation of the dependencies. You can see the functions below only run a subset.

Please read the comment from @majetideepak in my original PR about this. We need more changes to refactor this build script or we leave the original intent and separate the bundling vs non-bundling between the versions of macOS later.

install_build_prerequisites
install_velox_deps_from_brew
install_double_conversion
Expand All @@ -84,7 +84,6 @@ jobs:

- name: Configure Build
env:
folly_SOURCE: BUNDLED #brew folly does not have int128
fmt_SOURCE: BUNDLED #brew fmt11 is not supported
run: |
ccache -sz -M 5Gi
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/scheduled.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ on:
- "CMake/**"
- "third_party/**"
- "scripts/setup-ubuntu.sh"
- "scripts/setup-common.sh"
- "scripts/setup-versions.sh"
- "scripts/setup-helper-functions.sh"
- ".github/workflows/scheduled.yml"
- "setup.py"
Expand All @@ -37,6 +39,8 @@ on:
- "CMake/**"
- "third_party/**"
- "scripts/setup-ubuntu.sh"
- "scripts/setup-common.sh"
- "scripts/setup-versions.sh"
- "scripts/setup-helper-functions.sh"
- ".github/workflows/scheduled.yml"

Expand Down
6 changes: 4 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,11 +176,13 @@ libraries. Once you have checked out Velox, you can setup and build like so:

```shell
$ ./scripts/setup-centos9.sh
$ ./scripts/setup-adapters.sh
$ ./scripts/setup-centos9.sh install_adapters
$ make
```

Note that `setup-adapters.sh` supports macOS and Ubuntu 20.04 or later.
Note that the `install_adapters` command is available for the supported MacOS and
Ubuntu (20.04 or later) scripts. Individual adapters can be installed by specifying
the individual install command, e.g. `setup-centos9.sh install_aws`.

### Using Clang on Linux

Expand Down
6 changes: 4 additions & 2 deletions scripts/adapters.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@
ARG image=ghcr.io/facebookincubator/velox-dev:centos9
FROM $image

COPY scripts/setup-centos9.sh /
COPY scripts/setup-common.sh /
COPY scripts/setup-versions.sh /
COPY scripts/setup-helper-functions.sh /
COPY scripts/setup-adapters.sh /
RUN mkdir build && ( cd build && source /opt/rh/gcc-toolset-12/enable && \
bash /setup-adapters.sh ) && rm -rf build && dnf remove -y conda && dnf clean all
bash /setup-centos9.sh install_adapters ) && rm -rf build && dnf remove -y conda && dnf clean all

# install miniforge
RUN curl -L -o /tmp/miniforge.sh https://github.com/conda-forge/miniforge/releases/download/23.11.0-0/Mambaforge-23.11.0-0-Linux-x86_64.sh && \
Expand Down
2 changes: 2 additions & 0 deletions scripts/centos.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ ARG image=quay.io/centos/centos:stream9
FROM $image

COPY scripts/setup-helper-functions.sh /
COPY scripts/setup-versions.sh /
COPY scripts/setup-common.sh /
COPY scripts/setup-centos9.sh /

# Building libvelox.so requires folly and gflags to be built shared as well for now
Expand Down
252 changes: 0 additions & 252 deletions scripts/setup-adapters.sh
View file
Open in desktop

This file was deleted.

Loading
Loading