Bug #19403: Unable to Build Native Gems on Mac with Ruby 3.1.0+

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: 3.2.0
• Backport: 2.7: DONTNEED, 3.0: DONTNEED, 3.1: DONE, 3.2: DONE

[[e-mail] (James Couball)]

Unable to Build Native Gems on Mac with Ruby 3.1.0+

Problem

In Ruby 3.1 and later installed by rvm or asdf on a Mac, require 'mkmf' aborts
the script and reports the following error (the reported path is different depending
on the Ruby install manager and the version of Ruby):

mkmf.rb can't find header files for ruby at /Users/couballj/.asdf/installs/ruby/3.2.0/lib/ruby/include/ruby.h

You might have to install separate package for the ruby development
environment, ruby-dev or ruby-devel for example.

and RbConfig::CONFIG["rubyhdrdir"] is set to an incorrect path:

"/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Users/couballj/.asdf/installs/ruby/3.2.0/include/ruby-3.2.0"

Expected Behavior

Running require 'mkmf' should not abort and for my system/user using asdf installed
Ruby 3.2.0 RbConfig::CONFIG["rubyhdrdir"] should be set to:

"/Users/couballj/.asdf/installs/ruby/3.2.0/include/ruby-3.2.0"

Details

In Ruby 3.1 and later I can no longer build native gems (like nokoguru or redcarpet)
on my Mac.

I have tried this by installing Ruby with both rvm and asdf, both which have
the same results.

I receive the following error when trying to install (and build) these gems. For
example, here is the error message when I try to run gem install redcarpet:

mkmf.rb can't find header files for ruby at /Users/couballj/.asdf/installs/ruby/3.2.0/lib/ruby/include/ruby.h

You might have to install separate package for the ruby development
environment, ruby-dev or ruby-devel for example.

I found that this error could be triggered just by running a Ruby script that requires
mkmf. If test.rb contains this:

require 'mkmf'

Running this script:

ruby test.rb

Results in this output:

mkmf.rb can't find header files for ruby at /Users/couballj/.asdf/installs/ruby/3.2.0/lib/ruby/include/ruby.h

You might have to install separate package for the ruby development
environment, ruby-dev or ruby-devel for example.

Curiously, the reported directory does not exist. On my system, the path to ruby.h
is /Users/couballj/.asdf/installs/ruby/3.2.0/include/ruby-3.2.0/ruby.h

I found where in the Ruby code this error is reported in mkmf.rb line 233.

This code is trying to find ruby.h based on the value of RbConfig::CONFIG["rubyhdrdir"].
On my system, this value is:

"/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Users/couballj/.asdf/installs/ruby/3.2.0/include/ruby-3.2.0"

This value is a concatenation of ENV['SDKROOT'] (which on my system is /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk)
and the correct include directory path /Users/couballj/.asdf/installs/ruby/3.2.0/include/ruby-3.2.0.

When I run the same code with Ruby 2.7.7, RbConfig::CONFIG["rubyhdrdir"] is set to the correct path:

"/Users/couballj/.asdf/installs/ruby/3.2.0/include/ruby-3.2.0"

Possible Cause

I have found a possible cause in commit fd97862
in line 127 of mkconfig.rb. This change prepends ENV['SDKROOT'] to the include directory that
is used to build RbConfig::CONFIG["rubyhdrdir"].

While there may have been good reason for that change, in my case this change causes
require 'mkmf' to fail in my use case.

Work Around

I have been able to work around this issue by creating a symbolic link from
/Users /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Users to /Users
with the following command:

sudo ln -s /Users /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Users

This change will works for both rvm and asdf no matter what version of Ruby is
installed.

[[e-mail] (James Couball)]

I have also reported this in the asdf-ruby issue tracker asdf-vm/asdf-ruby#332.

[hsbt (Hiroshi SHIBATA)]

FYI: https://github.com/rbenv/ruby-build/discussions/2123#discussioncomment-4643615

[hsbt (Hiroshi SHIBATA)]

@[e-mail] (James Couball) Can you share your environmental variable of SDKROOT? In my environment, I couldn't look it.

$ env | rg SDKROOT
(nothing result)

[nobu (Nobuyoshi Nakada)]

I think it is fixed by 3a7367ccc319499127ead147e5a08f769e44208e.

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Closed
• Backport changed from 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN to 2.7: DONTNEED, 3.0: DONTNEED, 3.1: REQUIRED, 3.2: REQUIRED

I confirmed to fix installation error with SDKROOT environmental variable.

$ env | rg SDKROOT
SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

Before result with make install at ruby_3_1 branch.

(snip)
installing extension objects:       /Users/hsbt/.local/share/rbenv/versions/ruby_3_1/lib/ruby/vendor_ruby/3.1.0/arm64-darwin22
installing extension headers:       /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Users/hsbt/.local/share/rbenv/versions/ruby_3_1/include/ruby-3.1.0/arm64-darwin22
/Users/hsbt/Documents/github.com/ruby/ruby.github/lib/fileutils.rb:240:in `mkdir': Permission denied @ dir_s_mkdir - /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Users (Errno::EACCES)
        from /Users/hsbt/Documents/github.com/ruby/ruby.github/lib/fileutils.rb:240:in `fu_mkdir'
        from /Users/hsbt/Documents/github.com/ruby/ruby.github/lib/fileutils.rb:221:in `block (2 levels) in mkdir_p'

After result with 3a7367ccc319499127ead147e5a08f769e44208e:

(snip)
installing extension objects:       /Users/hsbt/.local/share/rbenv/versions/ruby_3_1/lib/ruby/vendor_ruby/3.1.0/arm64-darwin22
installing extension headers:       /Users/hsbt/.local/share/rbenv/versions/ruby_3_1/include/ruby-3.1.0/arm64-darwin22
installing extension scripts:       /Users/hsbt/.local/share/rbenv/versions/ruby_3_1/lib/ruby/3.1.0
(snip)

[[e-mail] (James Couball)]

Hello @hsbt (Hiroshi SHIBATA)! It looks like you solved the problem before I could respond :) Thank you!

For what it's worth, my value of SDKROOT is:

$ env | grep SDKROOT
SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

It is unclear to me ... will the change 3a7367c in be backported to 3.1 and 3.2?

[hsbt (Hiroshi SHIBATA)]

Yes. 3a7367ccc319499127ead147e5a08f769e44208e will be backported to 3.1 and 3.2.

[naruse (Yui NARUSE)]

• Backport changed from 2.7: DONTNEED, 3.0: DONTNEED, 3.1: REQUIRED, 3.2: REQUIRED to 2.7: DONTNEED, 3.0: DONTNEED, 3.1: REQUIRED, 3.2: DONE

ruby_3_2 3decf7df6f678ff6386c5c1888642d76f49dc147 merged revision(s) 3a7367ccc319499127ead147e5a08f769e44208e.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.7: DONTNEED, 3.0: DONTNEED, 3.1: REQUIRED, 3.2: DONE to 2.7: DONTNEED, 3.0: DONTNEED, 3.1: DONE, 3.2: DONE

ruby_3_1 485e0e46a58ff1a28eecd63b804b3dbea4b95b98 merged revision(s) 3a7367ccc319499127ead147e5a08f769e44208e.