Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cross-compilation issue with projects that use RocksDB built using Conan or Cmake on macOS with x86_64 CPU and targeted for arm64 CPUs #12239

Closed
shpala opened this issue Jan 16, 2024 · 4 comments
Closed

Cross-compilation issue with projects that use RocksDB built using Conan or Cmake on macOS with x86_64 CPU and targeted for arm64 CPUs #12239

shpala opened this issue Jan 16, 2024 · 4 comments

Comments

@shpala
Copy link
Contributor

shpala commented Jan 16, 2024 •
edited

Cross-compilation of a project that is linked to RocksDB on an x86_64 CPU with target CPU as arm64 fails after RocksDB was built using the following command:
cmake -G "Unix Makefiles" -DCMAKE_SYSTEM_PROCESSOR=arm64 ..
The RocksDB build itself succeeds, but executable linking with it will return a linking error.
This specific issue only happens when cross-compiling on an macOS machine due to a possible Cmake bug that causes CMAKE_SYSTEM_PROCESSOR to be equal to CMAKE_HOST_SYSTEM_PROCESSOR regardless of the value of CMAKE_OSX_ARCHITECTURES, which in turn causes an issue in RocksDBs own CMakeLists.txt.
In my case the issue happened when I was building RocksDB on x86_64 with a CMAKE_OSX_ARCHITECTURES set to "arm64".
Even after manually setting CMAKE_SYSTEM_PROCESSOR with "-DCMAKE_SYSTEM_PROCESSOR=arm64" or even when setting it manually at the top of CMakeLists.txt using set(CMAKE_SYSTEM_PROCESSOR arm64), immediately after the project() call, the CMAKE_SYSTEM_PROCESSOR was reset back to x86_64.
The linking error happens, because due to wrong CMAKE_SYSTEM_PROCESSOR, the variable HAS_ARMV8_CRC gets the wrong value.
This in turn causes Cmake to skip the source append command of "util/crc32c_arm64.cc".

Expected behavior

Successfully compiled executable.

Actual behavior

The following linking error:
Undefined symbols for architecture arm64:
"crc32c_arm64(unsigned int, unsigned char const*, unsigned long)", referenced from:
rocksdb::crc32c::ExtendARMImpl(unsigned int, char const*, unsigned long) in librocksdb.a(crc32c.cc.o)
"crc32c_runtime_check()", referenced from:
rocksdb::crc32c::IsFastCrc32Supported() in librocksdb.a(crc32c.cc.o)
__GLOBAL__sub_I_crc32c.cc in librocksdb.a(crc32c.cc.o)
"crc32c_pmull_runtime_check()", referenced from:
rocksdb::crc32c::IsFastCrc32Supported() in librocksdb.a(crc32c.cc.o)
__GLOBAL__sub_I_crc32c.cc in librocksdb.a(crc32c.cc.o)

nm -C librocksdb.a returns the following:
crc32c.cc.o:
0000000000000dfc s GCC_except_table1
0000000000000e20 s GCC_except_table21
0000000000000e4c s __GLOBAL__sub_I_crc32c.cc
U __Unwind_Resume
U crc32c_arm64(unsigned int, unsigned char const*, unsigned long)
U crc32c_runtime_check()
U crc32c_pmull_runtime_check()
0000000000000860 T rocksdb::DecodeFixed32(char const*)"

Steps to reproduce the behavior

Cross-compile a project that is linked to RocksDB on an x86_64 CPU with target CPU as arm64.
cmake -G "Unix Makefiles" -DCMAKE_SYSTEM_PROCESSOR=arm64 ..
shpala pushed a commit to shpala/rocksdb that referenced this issue Jan 16, 2024
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
9ada5e5 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_HOST_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
@shpala shpala mentioned this issue Jan 16, 2024
Closed
shpala pushed a commit to shpala/rocksdb that referenced this issue Jan 16, 2024
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
2ba5996 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_HOST_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala pushed a commit to shpala/rocksdb that referenced this issue Jan 16, 2024
@shpala
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
6c53d16 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_HOST_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala pushed a commit to shpala/rocksdb that referenced this issue Jan 16, 2024
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
39d6ca8 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala pushed a commit to shpala/rocksdb that referenced this issue Jan 17, 2024
@shpala
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
c783fc7 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
@adamretter
Copy link
Collaborator

We currently support cross-compilation on macOS using Make. See the target rocksdbjavastaticosx_ub in Makefile.

We might consider adding support in CMake. We don't use Conan. Can you provide the exact CMake command that you ran that will allow us to reproduce your problem please?

@shpala
Copy link
Contributor Author

shpala commented Jan 18, 2024 •
edited

@adamretter The code seems to successfully cross-compile when using Make, but fails when using Cmake.
The command I'm using is:
cmake -G "Unix Makefiles" -DCMAKE_SYSTEM_PROCESSOR=arm64 ..
The main CMakeLists.txt checks this variable in order to check for whether the compiler supports the "-march=armv8-a+crc+crypto" flag and consequently set the HAS_ARMV8_CRC variable.
Without setting HAS_ARMV8_CRC, the generated makefile won't compile the required crc32c_arm64.cc .

The reason this fails is that on macOS Cmake, calls to project() in CMakeLists.txt resets CMAKE_SYSTEM_PROCESSOR to the host CPU architecture.
For this reason it's more common to use CMAKE_OSX_ARCHITECTURES on macOS systems.
The main drawback of this, is that CMAKE_OSX_ARCHITECTURES might be a list of architectures, which we really don't want to support.

In the PR I provided, I modified CMakeLists.txt to check:
Only when running on Apple systems, if CMAKE_SYSTEM_PROCESSOR is different from CMAKE_OSX_ARCHITECTURES,
then set CMAKE_SYSTEM_PROCESSOR to be the same as CMAKE_OSX_ARCHITECTURES.
The build will return an error in cases when CMAKE_OSX_ARCHITECTURES will contain more than one value

shpala pushed a commit to shpala/rocksdb that referenced this issue Jan 18, 2024
@shpala
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
cb07a7d 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala pushed a commit to shpala/rocksdb that referenced this issue Jan 21, 2024
@shpala
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
71ec716 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala pushed a commit to shpala/rocksdb that referenced this issue Jan 22, 2024
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
5c23868 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala pushed a commit to shpala/rocksdb that referenced this issue Jan 26, 2024
@shpala
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
9ee960e 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala pushed a commit to shpala/rocksdb that referenced this issue Feb 23, 2024
@shpala
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
f6c4b37 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala pushed a commit to shpala/rocksdb that referenced this issue Feb 23, 2024
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
a4505d0 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala pushed a commit to shpala/rocksdb that referenced this issue Feb 23, 2024
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
2b520a9 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala added a commit to shpala/rocksdb that referenced this issue Feb 23, 2024
@shpala
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
aa692ba 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala added a commit to shpala/rocksdb that referenced this issue Feb 27, 2024
@shpala
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
111752e 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
@shpala
Copy link
Contributor Author

shpala commented Feb 27, 2024

The code is there, there's an open pull-request referenced to this issue.
@adamretter Do you know how I can promote this issue?

@adamretter
Copy link
Collaborator

@cmccrorie could you expedite this for @rhubner or @alanpaxton to review please the PR please?

shpala added a commit to shpala/rocksdb that referenced this issue Feb 28, 2024
@shpala
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
745c8d3 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
shpala added a commit to shpala/rocksdb that referenced this issue Feb 29, 2024
@shpala
Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64
f96fa2d 
by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target
whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call.
This is probably a Cmake bug that happens on macOS.
(facebook#12239)
kwadhwa18 pushed a commit to kwadhwa18/rocksdb that referenced this issue Mar 18, 2024
@shpala @kwadhwa18
Workaround for an issue with Cmake builds that happens when cross-com…
8193416 
…piling using Cmake on macOS x86_64 CPUs with target CPU arm64 (facebook#12240)

Summary:
This is a temporary workaround for Cmake bug that only sets the correct CMAKE_SYSTEM_PROCESSOR for cross-compilation when target CMAKE_SYSTEM_NAME differs from CMAKE_HOST_NAME:
https://gitlab.kitware.com/cmake/cmake/-/issues/25640

Fix cross-compilation on macOS x86_64 CPUs with target CPU arm64 by manually setting CMAKE_SYSTEM_PROCESSOR to the cross-compilation target whenever CMAKE_SYSTEM_PROCESSOR doesn't match CMAKE_OSX_ARCHITECTURES after project() call. This is probably a Cmake bug that happens on macOS.
Closes facebook#12239

The issue happens when RocksDB is built using the follwoing command:
cmake -G "Unix Makefiles" -DCMAKE_SYSTEM_PROCESSOR=arm64 ..
The build itself succeeds, but because Cmake wrongly sets CMAKE_SYSTEM_PROCESSOR to x86_64 instead of arm64 and causes crc32c_arm64.cc not to be compiled.
This in turn makes the project fails any linking with RocksDB:

```
Undefined symbols for architecture arm64:
"crc32c_arm64(unsigned int, unsigned char const*, unsigned long)", referenced from:
rocksdb::crc32c::ExtendARMImpl(unsigned int, char const*, unsigned long) in librocksdb.a(crc32c.cc.o)
```

Pull Request resolved: facebook#12240

Reviewed By: cbi42

Differential Revision: D54811365

Pulled By: pdillinger

fbshipit-source-id: 0958e3092806dadd2f61d582b7251af13a5f3f06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@shpala @adamretter