Project

General

Profile

Actions

Bug #16651

closed

Extensions Do Not Compile on Mingw64 with mingw32-make

Added by cfis (Charlie Savage) over 1 year ago. Updated 2 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x64-mingw32]
[ruby-core:97248]

Description

When mkmf.rb creates a Makefile for an extension, it will generate something that looks like this:

srcdir = .
topdir = C/MSYS64/USR/LOCAL/ruby-2.7.0/include/ruby-2.7.0
hdrdir = $(topdir)
arch_hdrdir = C:/MSYS64/USR/LOCAL/ruby-2.7.0/include/ruby-2.7.0/x64-mingw32

Notice the topdir path is c/ without the ":" Its only the topdir that does this, all other paths in the makefile use the "c:/" style.

mkmf.rb intentionally does that, see line 1098:

      def mkintpath(path)
        # mingw uses make from msys and it needs special care
        # converts from C:\some\path to /C/some/path
        path = path.dup
        path.tr!('\\', '/')
        path.sub!(/\A([A-Za-z]):(?=\/)/, '/\1') <-------- This line
        path
      end

But this is wrong, and causes errors like this (this is compiling the debase gem but it doesn't matter what c extension you use):

make: *** No rule to make target 'C/MSYS64/USR/LOCAL/ruby-2.7.0/include/ruby-2.7.0/ruby.h', needed by 'breakpoint.o'.  Stop.

The fix is simple, just delete that line. The makefile should look like this:

srcdir = .
topdir = C:/MSYS64/USR/LOCAL/ruby-2.7.0/include/ruby-2.7.0
hdrdir = $(topdir)
arch_hdrdir = C:/MSYS64/USR/LOCAL/ruby-2.7.0/include/ruby-2.7.0/x64-mingw32

Note I'm not the first person to see this, but I've just been manually fixing it over the years. Would be good to really fix it.

https://github.com/oneclick/rubyinstaller2/issues/105
https://github.com/oneclick/rubyinstaller2/issues/47
https://github.com/tmm1/http_parser.rb/issues/55

Note some of those tickets put the blame on using mingw-make versus msys make. But on my system, neither work with the "c/" style path but both work with the "c:/" style path.

Updated by cfis (Charlie Savage) 9 months ago

This is still and issue with Ruby 2.7.2.

The offending line of code is now at line 1908 in mkmf.rb.

Updated by nobu (Nobuyoshi Nakada) 9 months ago

It would replace "C:/MSYS64/..." as "/C/MSYS64/...".
Where did a slash go?

Updated by cfis (Charlie Savage) 6 months ago

Sorry, missed your comment. The problem is /C/ is wrong.

This is still an issue with Ruby 3.0.0 and causes building extensions to fail.

Install msys2, install mingw64. Then try to build any extension. You'll get this error message:

* No rule to make target '/C/msys64/usr/local/ruby-3.0.0/include/ruby-3.0.0/ruby.h', needed by 'breakpoint.o'.  Stop.

If you comment out this line:

#path.sub!(/\A([A-Za-z]):(?=\/)/, '/\1')

Then the paths becomes:

C:/msys64/usr/local/ruby-3.0.0/include/ruby-3.0.0/ruby.h

And the problem is fixed. That is the correct path we want.

Updated by cfis (Charlie Savage) 6 months ago

Any thoughts?

Actions #6

Updated by xtkoba (Tee KOBAYASHI) 5 months ago

With RubyInstaller's I got the following result in the Command Prompt of Windows 10:

C:\Ruby30-x64\bin>irb
irb(main):001:0> Dir.pwd
=> "C:/Ruby30-x64/bin"
irb(main):002:0> File.exist?("C:/Ruby30-x64")
=> true
irb(main):003:0> File.exist?("C/Ruby30-x64")
=> false
irb(main):004:0> File.exist?("/C:/Ruby30-x64")
=> false
irb(main):005:0> File.exist?("/C/Ruby30-x64")
=> false
irb(main):006:0> File.exist?("c:/Ruby30-x64")
=> true
irb(main):007:0> File.exist?("c/Ruby30-x64")
=> false
irb(main):008:0> File.exist?("/c:/Ruby30-x64")
=> false
irb(main):009:0> File.exist?("/c/Ruby30-x64")
=> false
irb(main):010:0>

I suppose MSYS2 might be necessary for the notation like /c/ [1]. I might be talking nonsense because I don't know about MSYS2 in detail.

[1] https://www.msys2.org/wiki/How-does-MSYS2-differ-from-Cygwin/

Updated by cfis (Charlie Savage) 4 months ago

When running msys2 (ie, in an open console) the /c/ notation does work.

But it does not work in makefiles processed by mingw64. If you look at the makefiles that extconf.rb generates, they use paths like c:/abc/def except for this one case. And this one case breaks building extensions if you are using mingw make. Thus this should be changed (note that msys supports the c:/ format also).

Updated by xtkoba (Tee KOBAYASHI) 4 months ago

This issue is related to commit 389157d8cfab7b6e84cdcdb2863421bd2c93bd7f, which merged the patch posted as [ruby-core:21448]. The description of the patch reads that the character : in the path confuses make from MSYS because it is interpreted as a path separator. If this is still true, then we cannot simply leave : unremoved with a drive letter.

Updated by cfis (Charlie Savage) 4 months ago

Thanks for the background, that was very helpful. The difference is that I am using mingw64 make, not msys make. So:

  • msys make supports /C/msys64/usr/local/ruby-2.7.2/include/ruby-2.7.0 and C:/msys64/usr/local/ruby-2.7.2/include/ruby-2.7.0

  • mingw make only supports c:/msys64/usr/local/ruby-2.7.2/include/ruby-2.7.0

If you look at the generated makefiles (this is from gem install --platform ruby pg):

topdir = /C/msys64/usr/local/ruby-2.7.2/include/ruby-2.7.0
hdrdir = $(topdir)
arch_hdrdir = C:/msys64/usr/local/ruby-2.7.2/include/ruby-2.7.0/x64-mingw32

Notice the makefiles use both forms of the path.

Bottom line - the patch from 12 years ago is no longer useful. It is not needed for msys make and it breaks mingw-make. So can it be removed?

Thanks - Charlie

Actions #10

Updated by cfis (Charlie Savage) 4 months ago

  • Subject changed from Extensions Do Not Compile on Mingw64 to Extensions Do Not Compile on Mingw64 with mingw64-make

Updated by xtkoba (Tee KOBAYASHI) 4 months ago

I suppose arch_hdrdir also should have been canonicalized by mkintpath, because otherwise it would cause trouble with VPATH in Makefile which is defined as:

VPATH = $(srcdir):$(arch_hdrdir)/ruby:$(hdrdir)/ruby

And I believe that the notation of VPATH as above does not work well with mingw32-make, because it allows only one path.

It seems clear that mingw32-make is not supported in the current build system of Ruby. I doubt if the maintainers have tried to support it ever.

One way of supporting both make from MSYS and mingw32-make would be to stop using VPATH, although I have no idea whether this is possible. FYI, VPATH seems to cause another trouble (Bug #12179), and it would be desirable if we could throw it away.

Updated by cfis (Charlie Savage) 4 months ago

VPATH works fine with arch_hdrdir in the form c:/msys64/etc - Ruby has been doing that on Windows for many years.

With my proposed fix both msys make and mingw64 make work correctly.

Updated by nobu (Nobuyoshi Nakada) 4 months ago

cfis (Charlie Savage) wrote in #note-12:

VPATH works fine with arch_hdrdir in the form c:/msys64/etc - Ruby has been doing that on Windows for many years.

Does it work with different drives too?

Actions #14

Updated by nobu (Nobuyoshi Nakada) 4 months ago

  • Description updated (diff)

Updated by xtkoba (Tee KOBAYASHI) 4 months ago

I suppose it is by luck that arch_hdrdir of the form C:/foo/bar is not causing problems with make from MSYS. In fact, in my environment (not Windows) gem install rails && rails new foo succeeds when VPATH is modified such that it does not contain $(arch_hdrdir)/ruby, or not even $(hdrdir)/ruby. I wonder if any extension in the world requires these paths contained in VPATH.

The proposed modification might be OK for make from MSYS if the definition of VPATH can be modified so that it never contains any drive letters. Note that some extensions (e.g. sassc) add additional paths to VPATH. And assuming that the modification is applied, I doubt if mingw32-make will successfully build an extension which really depends on VPATH containing multiple paths.

cfis (Charlie Savage) Could you please try to gem install sassc with mingw32-make? If it really does not work with multiple paths in VPATH, then it should fail to build that extension. Unfortunately I do not have access to Windows machines connected to the Internet.

Updated by nobu (Nobuyoshi Nakada) 4 months ago

Building io-console with 2.7.2 from RubyInstaller:

$ ruby -v ../src/ext/io/console/extconf.rb
ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x64-mingw32]
checking for HAVE_RUBY_FIBER_SCHEDULER_H... no
checking for rb_scheduler_timeout()... no
creating Makefile

$ grep -e ^topdir -e ^arch_hdrdir Makefile
topdir = /C/Ruby27-x64/include/ruby-2.7.0
arch_hdrdir = C:/Ruby27-x64/include/ruby-2.7.0/x64-mingw32

$ make
compiling ../src/ext/io/console/console.c
linking shared-object io/console.so

It succeeds with no problem.

/C/ is not wrong in msys make, I'm not sure why the first description had the wrong path without the leading slash.

Updated by nobu (Nobuyoshi Nakada) 4 months ago

  • Backport changed from 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN to 2.6: REQUIRED, 2.7: REQUIRED, 3.0: REQUIRED
  • Subject changed from Extensions Do Not Compile on Mingw64 with mingw64-make to Extensions Do Not Compile on Mingw64 with mingw32-make

MinGW on MSys build is cross compilation.
No reason to use mingw32-make, which isn't for the build OS.

Actions #18

Updated by nobu (Nobuyoshi Nakada) 4 months ago

  • Status changed from Open to Closed

Applied in changeset git|799ea1d1540aca03542ee31647052343507a4b41.


mkmf.rb: convert also arch_hdrdir [Bug #16651]

Updated by cfis (Charlie Savage) 3 months ago

To answer the questions above:

  • gem install sassc works fine
  • using different drives, d:/ works fine

I don't understand the reason for this change - it just makes building on mingw64 with mingw-64 on windows more broken. If the format c:/ didn't work, then every native gem would have failed to compile because arch_hdrdir used that style. So clearly it does work.

Thus arch_hdrdir should not be changed, and instead topdir should have changed.

Updated by nagachika (Tomoyuki Chikanaga) 2 months ago

  • Backport changed from 2.6: REQUIRED, 2.7: REQUIRED, 3.0: REQUIRED to 2.6: REQUIRED, 2.7: REQUIRED, 3.0: DONE

ruby_3_0 5e21726cda22e3cb34127751aec7e9babb4308b3 merged revision(s) 799ea1d1540aca03542ee31647052343507a4b41.

Actions

Also available in: Atom PDF