Feature #17730: Ruby on macOS transitively links to ~150 dylibs

• Status: Closed
• Assignee: -
• Target version: -

[rickmark (Rick Mark)]

By using -framework Security and -framework Foundation Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50. This greatly reduces Ruby's surface area and dependencies on macOS. Further CoreFoundation is only used for one call in the entire codebase of CFStringNormalize(m, kCFStringNormalizationFormC); - if we can replace this, Ruby could work with only libSystem and libgmp.

[nobu (Nobuyoshi Nakada)]

-lobjc has been intentionally linked to link startup code with Objective-C support, by the request for extension libraries written in it.

[nobu (Nobuyoshi Nakada)]

• Tracker changed from Bug to Feature
• ruby -v deleted (3.0.0)
• Backport deleted (2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN)

• Status changed from Open to Closed

Applied in changeset git|dc7044eb48f475e5ac34e994e89115052574c451.

---

Remove unneeded dependencies on macOS [Feature #17730]

[rickmark (Rick Mark)]

Looks like we applied the Foundation -> CoreFoundation patch.

What about the Security framework? It's the source of the worst dependencies like Metal, CoreML, etc...

(I know my patch proposal moves HEAD to macOS 10.10 or later, but that seems reasonable these days)

We could alternatively use basically any other source for random (the only reason Security is linked).

OpenSSL, /dev/urandom, or arc4random would all be acceptable I suppose

[nobu (Nobuyoshi Nakada)]

The Security framework is linked only when it is available but CommonRandom is not.
It is the case that the minimum required macOS version is 10.7(Lion)..10.9(Mavericks).
I'm not sure whether other sources are available on these old versions.

[duerst (Martin Dürst)]

rickmark (Rick Mark) wrote:

Further CoreFoundation is only used for one call in the entire codebase of CFStringNormalize(m, kCFStringNormalizationFormC); - if we can replace this, Ruby could work with only libSystem and libgmp.

I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to m.unicode_normalize(:nfc).

[naruse (Yui NARUSE)]

duerst (Martin Dürst) wrote in #note-6:

rickmark (Rick Mark) wrote:

Further CoreFoundation is only used for one call in the entire codebase of CFStringNormalize(m, kCFStringNormalizationFormC); - if we can replace this, Ruby could work with only libSystem and libgmp.

I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to m.unicode_normalize(:nfc).

Why it uses CFStringNormalize(m, kCFStringNormalizationFormC); is because it needs to just the same conversion as HFS+/APFS, which is different from standard NFC.

[rickmark (Rick Mark)]

nobu (Nobuyoshi Nakada) wrote in #note-5:

The Security framework is linked only when it is available but CommonRandom is not.
It is the case that the minimum required macOS version is 10.7(Lion)..10.9(Mavericks).
I'm not sure whether other sources are available on these old versions.

From compiling recently it looks like Security.framework is always linked (see the static linkage results from the text files)

I can rework this patch to do what we would expect (use Security.framework on 10.7-10.9 and then use CoreCrypto for > 10.10)

Will open a new issue for that work

[rickmark (Rick Mark)]

naruse (Yui NARUSE) wrote in #note-7:

duerst (Martin Dürst) wrote in #note-6:

rickmark (Rick Mark) wrote:

Further CoreFoundation is only used for one call in the entire codebase of CFStringNormalize(m, kCFStringNormalizationFormC); - if we can replace this, Ruby could work with only libSystem and libgmp.

I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to m.unicode_normalize(:nfc).

Why it uses CFStringNormalize(m, kCFStringNormalizationFormC); is because it needs to just the same conversion as HFS+/APFS, which is different from standard NFC.

Yep - the normalization is called UTF-8-MAC on libicu. This implies we can use other code to do this. Also since CF is opensource we could just bring in the routine directly and avoid any linkage to core macOS frameworks other then libSystem.B. This would also allow other platforms to do proper normalization for macOS as well? I will cut a new ticket for this.

[rickmark (Rick Mark)]

• File ruby_final.txt ruby_final.txt added

rickmark (Rick Mark) wrote in #note-8:

nobu (Nobuyoshi Nakada) wrote in #note-5:

The Security framework is linked only when it is available but CommonRandom is not.
It is the case that the minimum required macOS version is 10.7(Lion)..10.9(Mavericks).
I'm not sure whether other sources are available on these old versions.

From compiling recently it looks like Security.framework is always linked (see the static linkage results from the text files)

I can rework this patch to do what we would expect (use Security.framework on 10.7-10.9 and then use CoreCrypto for > 10.10)

Will open a new issue for that work

Nevermind - I just recompiled ruby from head. Looks like the linking is correct now. Must have been Foundation brining in Security causing my confusion. Final linking output attached

[naruse (Yui NARUSE)]

rickmark (Rick Mark) wrote in #note-9:

naruse (Yui NARUSE) wrote in #note-7:

duerst (Martin Dürst) wrote in #note-6:

rickmark (Rick Mark) wrote:

Further CoreFoundation is only used for one call in the entire codebase of CFStringNormalize(m, kCFStringNormalizationFormC); - if we can replace this, Ruby could work with only libSystem and libgmp.

I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to m.unicode_normalize(:nfc).

Why it uses CFStringNormalize(m, kCFStringNormalizationFormC); is because it needs to just the same conversion as HFS+/APFS, which is different from standard NFC.

Yep - the normalization is called UTF-8-MAC on libicu. This implies we can use other code to do this. Also since CF is opensource we could just bring in the routine directly and avoid any linkage to core macOS frameworks other then libSystem.B. This would also allow other platforms to do proper normalization for macOS as well? I will cut a new ticket for this.

Ruby also has UTF-8-MAC, but HFS+/APFS's one is sometimes updated.
We first used our own encoding table for the usage, but finally we switched to use OS's to ensure the behavior is exactly same as the filesystem.

[mame (Yusuke Endoh)]

• Related to Feature #17741: Ruby links to `objc` for convenience - this should be moved into a native ext added