setup-macos.sh fails on Apple M1

[xumingming]

env: Apple M1

error message:

-- Build files have been written to: /Users/abei/Code/velox/range-v3/_build
+ ninja -C _build install
ninja: Entering directory `_build'
[0/1] Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/lib/cmake/range-v3/range-v3-targets.cmake
-- Installing: /usr/local/lib/cmake/range-v3/range-v3-config-version.cmake
-- Up-to-date: /usr/local/lib/cmake/range-v3/range-v3-config.cmake
-- Installing: /usr/local/include
CMake Error at cmake_install.cmake:64 (file):
  file INSTALL cannot make directory "/usr/local/include": Permission denied.

I have to add sudo for cmake_install to walk around.

[xumingming]

also this error:

CMake Error at /opt/homebrew/Cellar/cmake/3.23.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find ICU (missing: ICU_LIBRARIES) (found version "70.1.0")
Call Stack (most recent call first):
  /opt/homebrew/Cellar/cmake/3.23.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  CMake/FindICU.cmake:267 (find_package_handle_standard_args)
  CMakeLists.txt:242 (find_package)

[xumingming]

seems because brew install libs to a different folder. velox expect /usr/local/opt, but actually brew install libs to /opt/homebrew/opt, I fixed it by:

sudo ln -s /opt/homebrew/opt /usr/local

[yingsu00]

We also saw the same error recently. @xumingming will you be able to fix the setup-macos.sh by adding the line above? Ideally it will be nice to detect the brew install directory automatically, then link it to /usr/local.

[xumingming]

@yingsu00 sure, will open an PR soon.

[kgpai]

Hi @xumingming , Velox doesnt build on an Apple M1 yet. @Yuhta is working on that and we hope to have this ready by end of May; we will be fixing the scripts for the M1 in that process.

[yingsu00]

@kgpai This is a permission issue on directory /usr/local. We got it on Intel processor too (MacOS).

[senthh]

@xumingming Were you able to resolve below error by adding sudo to cmake_install?

CMake Error at cmake_install.cmake:64 (file):
file INSTALL cannot make directory "/usr/local/include": Permission denied.

Itried the similar way but still not able to resolve this issue. It will be helpful if you add somemore insight on this errror

[jayzhan211]

It seems there is still build error for m1

-- Performing Test RANGES_HAS_MTUNE_NATIVE - Success
-- Performing Test RANGE_V3_ALIGNED_NEW_PROBE
-- Performing Test RANGE_V3_ALIGNED_NEW_PROBE - Success
-- Performing Test RANGES_HAS_FCOROUTINES_TS
-- Performing Test RANGES_HAS_FCOROUTINES_TS - Failed
-- Performing Test RANGE_V3_HAS_FCONCEPTS
-- Performing Test RANGE_V3_HAS_FCONCEPTS - Failed
-- Doxygen not found; the 'range-v3-doc' and 'range-v3-gh-pages.{clean,copy,update}' targets will be unavailable.
-- Configuring done (2.6s)
-- Generating done (0.0s)
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_TESTING


-- Build files have been written to: /Users/jayzhan/velox/range-v3/_build
+ ninja -C _build install
ninja: Entering directory `_build'
[0/1] Install the project...
-- Install configuration: "Release"
CMake Error at cmake_install.cmake:57 (file):
  file cannot create directory: /usr/local/lib/cmake/range-v3.  Maybe need
  administrative privileges.


FAILED: CMakeFiles/install.util 
cd /Users/jayzhan/velox/range-v3/_build && /opt/homebrew/Cellar/cmake/3.26.3/bin/cmake -P cmake_install.cmake
ninja: build stopped: subcommand failed.

real    0m15.734s
user    0m3.557s
sys     0m1.973s
+ echo 'Failed to run install_ranges_v3 .'
Failed to run install_ranges_v3 .
+ exit 1

Any workaround or PR to fix?

[majetideepak]

It is recommended to use INSTALL_PREFIX say export INSTALL_PREFIX=/Users/velox/velox_dependency_install
when installing the dependencies to avoid polluting the default install location.
This should also help with the permissions issue.

[majetideepak]

On MacOS, you have to set -DCMAKE_CXX_FLAGS=" -isystem /Users/velox/velox_dependency_install"
-DCMAKE_PREFIX_PATH=/Users/velox/velox_dependency_install for the build to pick up the dependencies.

[rishitc]

Hi @majetideepak,
I'm facing the same issue as @jayzhan211 when setting up Velox on my Mac M1:

+ ninja -C _build install
ninja: Entering directory `_build'
[0/1] Install the project...
-- Install configuration: "Release"
CMake Error at cmake_install.cmake:57 (file):
  file cannot create directory: /usr/local/lib/cmake/range-v3.  Maybe need
  administrative privileges.


FAILED: CMakeFiles/install.util 

However, I'm unable to correctly apply the fixes you shared, namely the addition of the INSTALL_PREFIX environment variable and the two CMake flags.

Can you please share where the two changes must be applied in the file setup-macos.sh for the build script to start working on M1 Macs?

Thanks.

[jayzhan211]

@rishitc Run export INSTALL_PREFIX=/Users/{username}/velox/velox_dependency_install in terminal, and run it. I don't even need to add those two flags

[rishitc]

Thanks @jayzhan211.
Running the shared export command before running CPU_TARGET="arm64" ./scripts/setup-macos.sh worked 👍

For future reference to anyone who stumbles on the same issue, here is the list of steps I followed for successfully setting up Velox on M1 Macs.

$ export INSTALL_PREFIX=/Users/{username}/velox/velox_dependency_install
$ CPU_TARGET="arm64" ./scripts/setup-macos.sh
$ export OPENSSL_ROOT_DIR=/opt/homebrew/opt/[email protected]
$ CPU_TARGET="arm64" make

NOTE: The command for exporting OPENSSL_ROOT_DIR was to solve a setup error wherein the build script could not find OpenSSL because it required this environment variable to be defined.

@majetideepak I think adding these commands to the project README.md would be worthwhile, possibly under a Troubleshooting heading. This would be a helpful exercise because I found multiple similar issues/threads on the same topic while trying to debug this issue. The number of such problems can be reduced by modifying the project README.md.

If it makes sense, I would love to create a PR.

Thanks.

[majetideepak]

CPU_TARGET="arm64" should not be required with this change #5236
Did you try the latest main?
Setting OPENSSL_ROOT_DIR should not be required. Let me look into that.

[majetideepak]

We should fix the OPENSSL_ROOT_DIR issue as well. Filed #5309
@rishitc Do you want to work on the fix?

[rishitc]

Hi @majetideepak,
Yes, I'd like to work on it 👍

[rishitc]

@majetideepak Also, I noticed a similar bug wherein Velox requires a specific dependency version. Still, Velox does not track the path to that particular version's folder, causing a build error if the default system version is incompatible with Velox.

For example, even in the case of Protobuf, my system already had version 23.x installed (this came from some other packages I had installed via brew), and Velox required version 21.x. This also caused a failing build (linker error, to be specific). This bug I faced is similar to #5218, though it manifested for me for a slightly different reason.

I'd solved this issue by uninstalling the 23.x version of Protobuf, but I believe this may not always be feasible, i.e., if they need the other version of Protobuf as well. There should be a way to tell the build system to take the specific version(s) of the dependencies it needs from specific paths.

Edit: I added some more details of a similar I faced.

[rishitc]

Hi @majetideepak,
Quick question: I'm getting a set of failed test cases when running CPU_TARGET="arm64" make unittest on my M1 Mac. I'm sharing the list below.

Is it normal for these test cases to fail? i.e. they don't work correctly in specific dev environments as they may not be applicable there, or is there some other root case?

54% tests passed, 109 tests failed out of 239

Total Test time (real) = 322.75 sec

The following tests did not run:
         73 - */MemoryPoolTest.memoryLeakCheck/* (Disabled)

The following tests FAILED:
          1 - velox_buffer_test (SIGTRAP)
          2 - velox_base_test (SIGTRAP)
          4 - velox_cache_test (SIGTRAP)
          5 - cached_factory_test (SIGTRAP)
          6 - velox_file_test (SIGTRAP)
         19 - HashStringAllocatorTest.stlAllocatorOverflow (SIGTRAP)
         33 - */MemoryAllocatorTest.allocateBytesWithAlignment/* (SIGTRAP)
         35 - */MemoryAllocatorTest.StlMemoryAllocator/* (SIGTRAP)
         36 - */MemoryAllocatorTest.badNonContiguousAllocation/* (SIGTRAP)
         39 - */MemoryAllocatorTest.allocatorCapacityWithThreads/* (SIGTRAP)
         43 - MmapArenaTest.managedMmapArenasFreeError (SIGTRAP)
         46 - MemoryArbitrationTest.create (SIGTRAP)
         51 - */MemoryCapExceededTest.singleDriver/* (SIGTRAP)
         52 - */MemoryCapExceededTest.multipleDrivers/* (SIGTRAP)
         53 - */MemoryCapExceededTest.memoryManagerCapacityExeededError/* (SIGTRAP)
         54 - MemoryManagerTest.Ctor (SIGTRAP)
         55 - MemoryManagerTest.addPool (SIGTRAP)
         56 - MemoryManagerTest.addPoolWithArbitrator (SIGTRAP)
         62 - MemoryManagerTest.GlobalMemoryManagerQuota (SIGTRAP)
         63 - MemoryManagerTest.alignmentOptionCheck (SIGTRAP)
         65 - MemoryManagerTest.quotaEnforcement (SIGTRAP)
         67 - */MemoryPoolTest.Ctor/* (SIGTRAP)
         68 - */MemoryPoolTest.AddChild/* (SIGTRAP)
         69 - */MemoryPoolTest.dropChild/* (SIGTRAP)
         79 - */MemoryPoolTest.MemoryCapExceptions/* (SIGTRAP)
         81 - */MemoryPoolTest.MemoryManagerGlobalCap/* (SIGTRAP)
         85 - */MemoryPoolTest.allocatorOverflow/* (SIGTRAP)
         87 - */MemoryPoolTest.contiguousAllocateExceedLimit/* (SIGTRAP)
         88 - */MemoryPoolTest.badContiguousAllocation/* (SIGTRAP)
         90 - */MemoryPoolTest.allocationFailStats/* (SIGTRAP)
         92 - */MemoryPoolTest.persistentNonContiguousAllocateFailure/* (SIGTRAP)
         93 - */MemoryPoolTest.transientNonContiguousAllocateFailure/* (SIGTRAP)
         95 - */MemoryPoolTest.persistentContiguousAllocateFailure/* (SIGTRAP)
         96 - */MemoryPoolTest.transientContiguousAllocateFailure/* (SIGTRAP)
         97 - */MemoryPoolTest.contiguousAllocateExceedMemoryPoolLimit/* (SIGTRAP)
         98 - */MemoryPoolTest.badNonContiguousAllocation/* (SIGTRAP)
         99 - */MemoryPoolTest.nonContiguousAllocateExceedLimit/* (SIGTRAP)
        100 - */MemoryPoolTest.nonContiguousAllocateError/* (SIGTRAP)
        101 - */MemoryPoolTest.mmapAllocatorCapAllocationError/* (SIGTRAP)
        102 - */MemoryPoolTest.mmapAllocatorCapAllocationZeroFilledError/* (SIGTRAP)
        103 - */MemoryPoolTest.mmapAllocatorCapReallocateError/* (SIGTRAP)
        104 - */MemoryPoolTest.validCheck/* (SIGTRAP)
        105 - */MemoryPoolTest.concurrentUpdateToDifferentPools/* (SIGTRAP)
        106 - */MemoryPoolTest.concurrentUpdatesToTheSamePool/* (SIGTRAP)
        107 - */MemoryPoolTest.concurrentUpdateToSharedPools/* (SIGTRAP)
        111 - */MemoryPoolTest.shrinkAndGrowAPIs/* (SIGTRAP)
        112 - */MemoryPoolTest.memoryReclaimerSetCheck/* (SIGTRAP)
        116 - */MemoryPoolTest.memoryUsageUpdateCheck/* (SIGTRAP)
        117 - */MemoryPoolTest.maybeReserve/* (SIGTRAP)
        118 - */MemoryPoolTest.raceBetweenFreeAndFailedAllocation/* (SIGTRAP)
        120 - */MemoryPoolTest.abortAPI/* (SIGTRAP)
        121 - */MemoryPoolTest.abort/* (SIGTRAP)
        124 - MockSharedArbitrationTest.failedArbitration (SIGTRAP)
        125 - MockSharedArbitrationTest.singlePoolGrowCapacityWithArbitration (SIGTRAP)
        127 - MockSharedArbitrationTest.arbitrateWithMemoryReclaim (SIGTRAP)
        128 - MockSharedArbitrationTest.arbitrateBySelfMemoryReclaim (SIGTRAP)
        134 - MockSharedArbitrationTest.enterArbitrationException (SIGTRAP)
        135 - MockSharedArbitrationTest.noArbitratiognFromAbortedPool (SIGTRAP)
        136 - MockSharedArbitrationTest.failedToReclaimFromRequestor (SIGTRAP)
        137 - MockSharedArbitrationTest.failedToReclaimFromOtherQuery (SIGTRAP)
        138 - MockSharedArbitrationTest.memoryPoolAbortThrow (SIGTRAP)
        139 - MockSharedArbitrationTest.arbitrationFailureRetry (SIGTRAP)
        140 - MockSharedArbitrationTest.concurrentArbitrations (SIGTRAP)
        141 - MockSharedArbitrationTest.concurrentArbitrationWithTransientRoots (SIGTRAP)
        151 - SharedArbitrationTest.concurrentArbitration (SIGTRAP)
        153 - velox_serialization_test (SIGTRAP)
        159 - TestValueTest.scopeUsageEnabled (SIGTRAP)
        163 - velox_type_test (SIGTRAP)
        164 - velox_type_tz_test (SIGTRAP)
        165 - velox_vector_fuzzer_test (SIGTRAP)
        166 - velox_vector_test (SIGTRAP)
        167 - simple_vector_test (SIGTRAP)
        168 - bias_vector_test (SIGTRAP)
        170 - velox_expression_test (SIGTRAP)
        171 - velox_parse_test (SIGTRAP)
        172 - velox_dwio_common_test (SIGTRAP)
        176 - velox_dwio_dwrf_decompression_test (SIGTRAP)
        177 - velox_dwio_dwrf_stripe_stream_test (SIGTRAP)
        180 - velox_dwio_dwrf_dictionary_encoder_test (SIGTRAP)
        181 - velox_dwio_dwrf_encoding_selector_test (SIGTRAP)
        185 - velox_dwio_dwrf_writer_test (SIGTRAP)
        189 - velox_dwio_dwrf_data_buffer_holder_test (SIGTRAP)
        191 - velox_dwio_dwrf_decryption_test (SIGTRAP)
        192 - velox_dwio_dwrf_encryption_test (SIGTRAP)
        193 - velox_dwio_dwrf_stripe_reader_base_test (SIGTRAP)
        194 - velox_dwio_dwrf_reader_base_test (SIGTRAP)
        197 - velox_dwio_dwrf_flush_policy_test (SIGTRAP)
        198 - velox_dwio_dwrf_byte_rle_test (SIGTRAP)
        201 - velox_dwio_dwrf_rle_test (SIGTRAP)
        202 - velox_dwio_dwrf_int_direct_test (SIGTRAP)
        204 - velox_dwio_dwrf_column_reader_test (SIGTRAP)
        205 - velox_dwio_dwrf_reader_test (SIGTRAP)
        208 - velox_dwrf_column_writer_test (SIGTRAP)
        211 - velox_dwrf_e2e_writer_test (SIGTRAP)
        216 - velox_dwio_type_fbhive_test (SIGTRAP)
        219 - velox_functions_lib_test (SIGTRAP)
        220 - velox_functions_json_test (SIGTRAP)
        222 - velox_windows_rank_test (SIGTRAP)
        223 - velox_windows_value_test (SIGTRAP)
        226 - velox_aggregates_test (SIGTRAP)
        227 - velox_functions_test (SIGTRAP)
        228 - velox_spark_windows_value_test (SIGTRAP)
        229 - velox_functions_spark_aggregates_test (SIGTRAP)
        230 - velox_functions_spark_test (SIGTRAP)
        233 - velox_hive_connector_test (SIGTRAP)
        234 - velox_tpch_connector_test (SIGTRAP)
        237 - velox_exec_test (SIGTRAP)
        238 - velox_exec_infra_test (SIGTRAP)
        239 - velox_duckdb_conversion_test (SIGTRAP)
Errors while running CTest
make: *** [unittest] Error 8

[majetideepak]

my system already had version 23.x installed (this came from some other packages I had installed via brew), and Velox required version 21.x

23.x protobuf issue is very recent. We could throw an error, but it is cumbersome to keep track of and maintain such incompatible versions on different OSes.
The ideal solution is to bundle Protobuf and other dependencies which I believe is a work in progress. Installing to the system path via brew or other means is a recipe for such conflicts.
Users can alternatively use a container or VM to avoid such build conflicts.

[majetideepak]

Quick question: I'm getting a set of failed test cases when running CPU_TARGET="arm64" make unittest on my M1 Mac. I'm sharing the list below.

This is expected. The unit tests are not tested on MacOS and some of them fail as you reported.

[jensenojs]

I had tried with the commands above, however it can't pass the last step.

This message may be a bit out of the blue, but my guess after looking through the README docs and some mac-related config issue is that this seems to be the latest development here.

The problem I'm having now also seems similar to some existing problems(such as fmt version), and i fixed some of them, except below one.

i found #5421 seems have similar problem, but it fixed after all.

anyone could help ? @majetideepak

i just want to learn how to build database from velox, and the info above help me generate compile_comands.json successfully, so I can read the source code happily, tks!

Mac m2, clang 14.0, Mac OSX 13.4.1, protobuf@21, commit-id : 5b32965

Not sure if this way of writing supports the M2 chip, but when I comment out this condition and re-run, it still reports

Thanks @jayzhan211. Running the shared export command before running CPU_TARGET="arm64" ./scripts/setup-macos.sh worked 👍

For future reference to anyone who stumbles on the same issue, here is the list of steps I followed for successfully setting up Velox on M1 Macs.

$ export INSTALL_PREFIX=/Users/{username}/velox/velox_dependency_install
$ CPU_TARGET="arm64" ./scripts/setup-macos.sh
$ export OPENSSL_ROOT_DIR=/opt/homebrew/opt/[email protected]
$ CPU_TARGET="arm64" make

NOTE: The command for exporting OPENSSL_ROOT_DIR was to solve a setup error wherein the build script could not find OpenSSL because it required this environment variable to be defined.

@majetideepak I think adding these commands to the project README.md would be worthwhile, possibly under a Troubleshooting heading. This would be a helpful exercise because I found multiple similar issues/threads on the same topic while trying to debug this issue. The number of such problems can be reduced by modifying the project README.md.

If it makes sense, I would love to create a PR.

Thanks.

[rishitc]

Hi @jensenojs,
Which version of OpenSSL is installed on your system?

I think the issue you are facing is related to OpenSSL because of the error messages displayed in the first screenshot; it appears that you are using OpenSSL version 3.x when version 1.1 is required for building Velox[1].

Note:
[1] The error message mentions OSSL_DEPRECATEDIN_3_0, usually related to issues using OpenSSL 3.x.

[jensenojs]

Hi @jensenojs, Which version of OpenSSL is installed on your system?

I think the issue you are facing is related to OpenSSL because of the error messages displayed in the first screenshot; it appears that you are using OpenSSL version 3.x when version 1.1 is required for building Velox[1].

Note: [1] The error message mentions OSSL_DEPRECATEDIN_3_0, usually related to issues using OpenSSL 3.x.

@rishitc thanks for helping! and yes i used OpenSSL in wrong version, The OpenSSL under my /usr/bin is version 3.x, but unfortunately it is really quite difficult to solve it (this seems to be related to System Integrity Protection, so I can't delete it directly even after i turn off it (chmod, sudo)... ), and homebrew Refused to use 1.1 as the default, so I added [email protected] to the environment variable.

however, i don't know how to set LDFLAGS and CPPFLAGS in velox, this CMake project is just too complicated for me

so folly complain about opensslv.h not found, but opensslv.h is there.

Am I on the right path to troubleshoot the problem and any advice?

[majetideepak]

@jensenojs you have to brew unlink openssl and brew link [email protected].
Can you delete the build folder and try a clean install again?

[jensenojs]

@majetideepak thanks for reply! But I seem to have encountered a new problem, which looks somewhat similar to this issue #5218 , but i have to go to work and i will check later

[jensenojs]

i used same way to deal with protobuf, and finally suceessed!

@majetideepak @rishitc thanks you so much!

[dulvinw]

I am still facing the error running the script by itself. Can I contribute to the setup-macos.sh to add export INSTALL_PREFIX=/Users/{username}/velox/velox_dependency_install

[majetideepak]

@dulvinw sure! We have to make this change to the other OS setup scripts as well.
We should modify the README and suggest users invoke the setup scripts from the Velox root directory.
We need a BUILD_PREFIX for builds similar to INSTALL_PREFIX.
We can then create a dependencies_build directory for the builds and a dependencies_install directory for the installs in the Velox root directory.
And add these directories to .gitignore
Then change setups scripts to add the above defaults for BUILD_PREFIX, INSTALL_PREFIX, and also the root CMakeLists.txt to include CMAKE_PREFIX_PATH
CC: @kgpai

[laofan13]

hi, I also complie velox in mac m1:

export INSTALL_PREFIX=/Users/{username}/velox/velox_dependency_install
CPU_TARGET="arm64" ./scripts/setup-macos.sh
make

but, output error:

CMake Error at CMake/resolve_dependency_modules/re2.cmake:38 (if):
  if given arguments:

    "STREQUAL" "BUNDLED"

  Unknown arguments specified
Call Stack (most recent call first):
  CMake/ResolveDependency.cmake:43 (include)
  CMake/ResolveDependency.cmake:75 (build_dependency)
  CMakeLists.txt:411 (resolve_dependency)

[unigof]

brew unlink openssl and brew link [email protected]

$ export INSTALL_PREFIX=/Users/{username}/velox/velox_dependency_install
$ CPU_TARGET="arm64" ./scripts/setup-macos.sh
$ export OPENSSL_ROOT_DIR=/opt/homebrew/opt/[email protected]
$ CPU_TARGET="arm64" make

I tried it worked on macos M2!! Can you add this step to README and update build scripts? Thank you
@majetideepak @kgpai @rishitc

[majetideepak]

@unigof we should not need CPU_TARGET="arm64" anymore. Do you want to open a PR to update the README?

[xumingming]

Closing this as original issue is already solved.