Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: switch macos builds to apple silicon and upgrade xcode #424

Merged
merged 3 commits into from
May 21, 2024

Conversation

galargh
Copy link
Contributor

@galargh galargh commented Oct 2, 2023

This PR migrated the macos builds in the repo to Apple Silicon executors since the Apple Intel ones are getting deprecated by January 2024. See https://discuss.circleci.com/t/macos-intel-support-deprecation-in-january-2024/48718.

FYI, the builds currently fail with:

# runtime/cgo
ld: warning: directory not found for option '-L/usr/lib/aarch64-linux-gnu/stubs'
# net
ld: warning: directory not found for option '-L/usr/lib/aarch64-linux-gnu/stubs'
# github.com/filecoin-project/filecoin-ffi/cgo
ld: warning: directory not found for option '-L/usr/lib/aarch64-linux-gnu/stubs'
ld: library not found for -lhwloc
clang: error: linker command failed with exit code 1 (use -v to see invocation)
FAIL    github.com/filecoin-project/filecoin-ffi [build failed]

I assume it's going to be easier to figure out for maintainers.

@galargh galargh marked this pull request as ready for review October 2, 2023 16:14
@vmx
Copy link
Contributor

vmx commented Oct 4, 2023

This is even more difficult. We currently create universal binaries on MacOS for x86 and aarch64. We need to check if it also works to create x86 version on the M1. I've opened #420 for tracking that.

@vmx
Copy link
Contributor

vmx commented Oct 4, 2023

I've fixed the original error, it was just about setting the LIBRARY_PATH. Though this should not be merged as building the universal binary on MacOS probably does not work yet.

@vmx
Copy link
Contributor

vmx commented Oct 6, 2023

This PR needs thorough review/testing by someone from the Lotus team. It should build the correct release artifacts, but I haven't really tested it. I'm neither that familiar with Lotus, nor do I own a Mac.

galargh and others added 2 commits May 14, 2024 21:17
It's now possible to build universal binaries on x86_64 as well as aarch64.

Closes #420.
@rvagg
Copy link
Member

rvagg commented May 14, 2024

@galargh @vmx I hope you don't mind that I rebased and force pushed this branch to get it updated.

Tried this successfully on my m2:

$ file libfilcrypto.a
./libfilcrypto.a: Mach-O universal binary with 2 architectures: [x86_64:current ar archive] [arm64]
./libfilcrypto.a (for architecture x86_64):	current ar archive
./libfilcrypto.a (for architecture arm64):	current ar archive

Though compiling lotus:

ld: warning: dylib (/opt/homebrew/lib/libhwloc.dylib) was built for newer macOS version (14.0) than being linked (13.3)

I have xcode 14 locally, I guess brew built hwloc for me, but the Circle config has 13.4.1. I'm just not sure how 13.3 got in here if I also built the FFI locally, I don't see an xcode or macos-version specifier in here anywhere. Maybe it's a rust thing? Hopefully the difference between 13.4 and 13.3 isn't going to make this a common warning.

But lotus still runs, and syncs the chain from a snapshot.

make unittests works, although I get the ld warning each time it runs something that touches ffi, it's pretty noisy. I get some unittest failures in storage/sealer involving lstat /dev/fd/4: bad file descriptor and also mentioning DEBUG ffiwrapper ffiwrapper/sealer_cgo.go:218 AddPiece: cannot close pieceData reader *nullreader.NullReader because it is not an io.Closer, which is sus, but I get the same errors if I run it from a normal build. So I don't think it's related.

I have an Intel mac in the house I could test the universal binary on just to be sure, but I don't have access to it at the moment, I'll try later in the week.

@vmx
Copy link
Contributor

vmx commented May 14, 2024

@rvagg I'm happy that you have a look. I have almost zero experience with MacOS (and no local machine to test on).

Copy link
Member

@rvagg rvagg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is working as tested both on my m2, and using assets compiled on my m2 used to compile and run lotus on an old intel Macbook 12" which is currently syncing the chain without too much trouble

good to go I'd say

@rvagg rvagg merged commit 03b9503 into master May 21, 2024
9 checks passed
@rvagg rvagg deleted the ci/macos-m1-medium-gen1 branch May 21, 2024 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants