Feature #17845
closedWindows Ruby - ucrt build?
Added by MSP-Greg (Greg L) over 3 years ago. Updated over 2 years ago.
Description
Currently, Windows Ruby is normally compiled two ways.
The first, mswin, is compiled using Microsoft's current Visual C compiler, and links to the universal runtime (ucrt).
The second, mingw, is compiled using MinGW gcc. This links to msvcrt, an older version of Microsoft's Visual C runtime.
Previously, all the MSYS2 MinGW packages linked to msvcrt. The MSYS2 project is now releasing build tools and packages linking to ucrt.
MSYS2 has provided ruby packages, and GitHub user @Biswa96 contributed https://github.com/msys2/MINGW-packages/pull/8518, allowing Ruby to compile with ucrt.
I tried the patch with ruby-loco, and it builds. There are some test issues, haven't had a chance to look at them yet.
Normally, using exe/dll/so files together that use different versions of the runtime is not a good idea. Building extension gems with ucrt may provide gems that can be used with mswin, but can't be compiled due to issues with Visual C (vs gcc). Also, packages from the vcpkg project may be compatible with Ruby ucrt.
The reason for this post is that Windows Ruby built with ucrt is essentially another platform, but the build 'looks like' a mingw build.
This is really a third platform choice. What should it be called? Things like CONFIG['RUBY_SO_NAME']
, RUBY_PLATFORM
, etc?
Files
ruby-mingw32-ucrt.patch (622 Bytes) ruby-mingw32-ucrt.patch | xtkoba (Tee KOBAYASHI), 06/07/2021 02:15 PM | ||
ruby-configure-mingw32-ucrt.patch (687 Bytes) ruby-configure-mingw32-ucrt.patch | xtkoba (Tee KOBAYASHI), 06/09/2021 12:27 AM | ||
ruby-configure-coroutine-mingw32.patch (351 Bytes) ruby-configure-coroutine-mingw32.patch | xtkoba (Tee KOBAYASHI), 06/11/2021 01:27 AM | ||
ruby-i386-mingw32-ucrt-undef.patch (398 Bytes) ruby-i386-mingw32-ucrt-undef.patch | xtkoba (Tee KOBAYASHI), 06/13/2021 06:15 PM | ||
ruby-date-mingw32-ucrt-timezone.patch (333 Bytes) ruby-date-mingw32-ucrt-timezone.patch | xtkoba (Tee KOBAYASHI), 06/13/2021 06:22 PM |
Updated by xtkoba (Tee KOBAYASHI) over 3 years ago
I wonder if there is a standard target triplet for MinGW + UCRT. I came up with x86_64-w64-mingw32-ucrt
, but when googled it hits only one page. I suppose something like x64-mingw32-ucrt
is acceptable for target_os
in configure
, as there is a preceding example for Android: aarch64-linux-android
vs aarch64-linux
(GNU/Linux), although I have no idea whether it is sufficient to just suffix target_os
.
Updated by MSP-Greg (Greg L) over 3 years ago
MinGW msvcrt
packages have a prefix of mingw-w64-x86_64
,
MinGW ucrt
packages have a prefix of mingw-w64-ucrt-x86_64
.
Updated by larskanis (Lars Kanis) over 3 years ago
I wonder if there is a standard target triplet for MinGW + UCRT.
The compiler triplet for MINGW with UCRT is still 'x86_64-w64-mingw32'. This is defined in MSYS2 here. It is not distinct from MSVCRT based MINGW because it differs only in a library (msvcrt.dll vs. ucrtbase.dll) and the dependent libraries are not part of the compiler triplet.
I did some testing with the patch used in MSYS2. It doesn't establish any new platform strings, so that RbConfig is mostly the same with MSVCRT and UCRT. Extension libraries linked to MSVCRT usually load into a UCRT based ruby at the first attempt, but they fail later on. FFI fails when using long double values and Nokogiri fails in some test case while calling free()
of msvcrt.dll. These are expected issues, when mixing MSVCRT and UCRT into one process.
So, I would propose a patch similar to this rather than the one in MSYS2. This sets
- CONFIG['arch'] = 'x64-mingw32' # equal in MSVCRT and UCRT
- CONFIG['sitearch'] = 'x64-ucrt' # vs. 'x64-msvcrt' in MSVCRT
- CONFIG['RUBY_SO_NAME'] = 'x64-ucrt-ruby310' # vs. 'x64-msvcrt-ruby310' in MSVCRT
- RUBY_PLATFORM = 'x64-mingw32' # equal in MSVCRT and UCRT
I'm uncertain if the use of UCRT should be reflected in RUBY_PLATFORM
. In any case it should still contain "mingw" since I've seen a lot of code checking for Windows by RUBY_PLATFORM=~/mingw|mswin/
.
Similarly Rubygems should use a different platform string for UCRT and MSVCRT based gems, since binary extensions are incompatible. This is still missing in the patch above. Maybe 'x64-mingw32-ucrt' as rubygems platform or 'x64-mingwucrt' or the platform of MSWIN?
My rough idea is to switch to UCRT with RubyInstaller-3.1. That means that all current releases keep using MSVCRT and that there will be no MSVCRT based RubyInstaller-3.1.
Updated by MSP-Greg (Greg L) over 3 years ago
Sorry, I meant to ping you immediately, but got distracted. Also, I saw your changes to ridk, but didn't see your patch file.
Anyway, I agree that there needs to be some way to distinguish between mingw and ucrt builds. And RubyGems/Bundler will need changes.
I haven't looked at runtime dependencies for a while. Question - wouldn't some (or most) extension gems compiled with ucrt be compatible with mswin builds, since there isn't a MSVC runtime mismatch?
I'll see what can be done to get ucrt usable with GH Actions...
Updated by xtkoba (Tee KOBAYASHI) about 3 years ago
The test TestEnv#test_huge_value
showed a difference between msvcrt and ucrt for x64-mingw32:
1) Failure:
TestEnv#test_huge_value [C:/CIFS/ruby-trunk/test/ruby/test_env.rb:472]:
Exception raised:
<#<Errno::EINVAL: Invalid argument - ruby_setenv(foo)>>
Backtrace:
C:/CIFS/ruby-trunk/test/ruby/test_env.rb:472:in `[]='
The test code is shown below. It seems that x64-mingw32 + ucrt behaves here as if /mswin/ =~ RUBY_PLATFORM
and not like msvcrt. To make this test work correctly we need a way to distinguish between the two.
def test_huge_value
huge_value = "bar" * 40960
ENV["foo"] = "bar"
if /mswin/ =~ RUBY_PLATFORM
assert_raise(Errno::EINVAL) { ENV["foo"] = huge_value }
assert_equal("bar", ENV["foo"])
else
assert_nothing_raised { ENV["foo"] = huge_value }
assert_equal(huge_value, ENV["foo"])
end
end
Updated by naruse (Yui NARUSE) about 3 years ago
- Tracker changed from Misc to Feature
- Target version set to 3.1
Updated by xtkoba (Tee KOBAYASHI) about 3 years ago
- File ruby-mingw32-ucrt.patch ruby-mingw32-ucrt.patch added
The attached patch is essentially a cleanup of that by @Biswa96 from GitHub. With this ./configure rb_cv_msvcrt=ucrt140
should work for Mingw-w64 with the default crt being UCRT [1].
[1] https://sourceforge.net/p/mingw-w64/mailman/message/36621319/
It would be desirable that the configure script is improved so that the checking for mingw32 runtime DLL can find the conftest executable linked against UCRT.
Updated by xtkoba (Tee KOBAYASHI) about 3 years ago
A patch is attached for the configure script to set rb_cv_msvcrt=ucrt
and RT_VER=140
when UCRT is used.
Updated by xtkoba (Tee KOBAYASHI) about 3 years ago
Pattern matching for target_os
in configure script should be permissive if we consider suffixing something onto "mingw32".
Updated by xtkoba (Tee KOBAYASHI) about 3 years ago
_WIN64
is not defined on i386-mingw32 + ucrt.
Updated by xtkoba (Tee KOBAYASHI) about 3 years ago
Or should extconf.rb
be improved to notice deprecation?
../../../ext/date/date_core.c:7803:17: warning: 'timezone' is deprecated: Only provided for source compatibility; this variable might not always be accurate when linking to UCRT. [-Wdeprecated-declarations]
of = (long)-timezone;
^
/usr/x86_64-w64-mingw32_ucrt/x86_64-w64-mingw32/include/time.h:257:32: note: 'timezone' has been explicitly marked deprecated here
_CRTIMP extern long timezone __MINGW_ATTRIB_DEPRECATED_UCRT;
^
/usr/x86_64-w64-mingw32_ucrt/x86_64-w64-mingw32/include/time.h:249:5: note: expanded from macro '__MINGW_ATTRIB_DEPRECATED_UCRT'
__MINGW_ATTRIB_DEPRECATED_MSG( \
^
/usr/x86_64-w64-mingw32_ucrt/x86_64-w64-mingw32/include/_mingw.h:182:58: note: expanded from macro '__MINGW_ATTRIB_DEPRECATED_MSG'
#define __MINGW_ATTRIB_DEPRECATED_MSG(x) __attribute__ ((__deprecated__(x)))
^
Updated by xtkoba (Tee KOBAYASHI) about 3 years ago
A separate ticket for -Wparentheses
warning issue: #17946
Updated by larskanis (Lars Kanis) about 3 years ago
Thank you @xtkoba (Tee KOBAYASHI) for working on this! I replaced my previous patches by the above patches in rubyinstaller2-packages.
One thing I noticed is that you disabled the call to _CrtSetReportMode(_CRT_ASSERT, 0)
in MINGW-UCRT. Actually gcc provides the function, if crtdbg.h
is included. However I don't know if it makes any functional difference to call it or not.
Or should
extconf.rb
be improved to notice deprecation?
IMHO it's OK to use defines and to keep extconf.rb
untouched.
Updated by xtkoba (Tee KOBAYASHI) about 3 years ago
On Mingw-w64 _CrtSetReportMode
is defined in crtdbg.h
as
#define _CrtSetReportMode(t,f) ((int)0)
thus I believe the definition in win32/win32.c
makes no difference although it may cause a -Wmacro-redefined
warning.
Updated by larskanis (Lars Kanis) about 3 years ago
The rubyinstaller-head nightly build is compiled with MINGW-UCRT since a few days. So all projects that use ruby-setup with version "head" or Appveyor with rubyinstaller-head download run on it now. Rubyinstaller-head already triggered a small issue in ffi and fxruby but other projects seems to run smooth.
The experience with rubyinstaller-head strengthens my plan to switch to UCRT with RubyInstaller-3.1.
Regarding CI runs: ruby-setup with version "mingw" is still on MSVCRT and https://github.com/MSP-Greg/setup-ruby-pkgs doesn't support UCRT builds so far.
@MSP-Greg (Greg L): Would you mind to add support to setup-ruby-pkgs? Do you want to switch your ruby-logo build to UCRT as well or add it as another architecture?
Updated by MSP-Greg (Greg L) about 3 years ago
@larkanis
Do you want to switch your ruby-logo build to UCRT as well or add it as another architecture?
I think adding as another architecture would be best? Maybe tag it as ruby-ucrt
?
Would you mind to add support to setup-ruby-pkgs?
I'll open an issue about adding the ucrt build tools to the Actions images so a full download isn't required.
Also, there's a lot of patches here, will they be added to ruby/ruby?
Re setup-ruby-pkgs
, maybe the best solution is to install ucrt
packages matching the listed mingw
package when the build is ruby-head
(or head
)?
Not sure whether to discuss here or in a setup-ruby-pkgs
issue. I think this affects setup-ruby
, as it adds the tools paths to ENV['Path']
... @Eregon (Benoit Daloze)?
Sorry, I've been following this, but didn't expect you'd convert your build so quickly. I'm fine with it though...
Updated by larskanis (Lars Kanis) about 3 years ago
I opened a pull request to ruby including the above patches: https://github.com/ruby/ruby/pull/4599
One remaining issue is about the gem platform. It is currently:
-
Gem::Platform.local.to_s
=> "x64-mingw32" # equal in MSVCRT and UCRT
BUT gems with C-extension aren't compatible between MINGW-MSVCRT and MINGW-UCRT. Loading fails due to different libruby names (due to ruby-configure-mingw32-ucrt.patch above). If libruby would be named equally, then C extensions would often work to some extend, but can fail with a segfault or deliver wrong results.
On the other hand the MSWIN platform isn't compatible to MINGW-UCRT. In theory gems compiled for MINGW-UCRT should run flawlessly on MSWIN, since they both share the same C runtime. But extensions compiled with MSWIN link to x64-vcruntime140-ruby310.dll
, so that either loading of the DLL fails or the entry addresses are different leading to error incompatible library version
. The other way around of using MSWIN gems in MINGW-UCRT doesn't work due to the additional dependency to vcruntime140.dll
, which isn't part of Windows nor MSYS2.
More details to the different Windows compiler platforms are here: https://www.msys2.org/docs/environments/
Since all three Windows platforms aren't compatible they should have distinct Rubygems platform strings. I propose x64-mingw32-ucrt
which fits into the naming scheme of MSWIN x64-mswin64-140
and the proposed x86_64-linux-musl
for Apline Linux. Or should we use the chance to drop the misleading "32" and call it x64-mingw-ucrt
?
Updated by larskanis (Lars Kanis) about 3 years ago
Thinking a bit more about the ABI incompatibilities convinced me more about the need to differentiate the ruby platforms on Windows. So I added another patch to https://github.com/ruby/ruby/pull/4599 which sets CONFIG['arch']
and RUBY_PLATFORM
to "x64-mingw-ucrt".
This allows easy differentiation between ABI incompatible platforms like MSWIN64 and MSVCRT-based MINGW32 and it also implicates a distinct rubygem platform which is also "x64-mingw-ucrt". See also the table in the pull request.
Although the term "mingw32" is the OS-part even for 64 bit systems, the "32" is misleading and confusing for many users. Therefore the new platform string drops the "32" from the OS part to just "mingw". This conforms to the common practice of windows platform testing per RUBY_PLATFORM=~/mswin|mingw/ .
Binaries are available here: https://github.com/oneclick/rubyinstaller2/releases/tag/rubyinstaller-head
I did some testing with several rails apps on rubyinstaller-head (x64-mingw-ucrt) and it did not show any regressions. There is only one issue with bundler: x64-mingw-ucrt is treated as platform "ruby" and "mri" in the Gemfile. This is probably conform to the understanding of most users, but it is against the specification of bundlers platform argument which classifies "mri" as NOT Windows.
Updated by larskanis (Lars Kanis) almost 3 years ago
IMHO the PR https://github.com/ruby/ruby/pull/4599 is ready to be merged. Are the platform strings OK as summarized in the PR? Are there any changes to be done to get it in?
The bundler issue described above is rather peripheral. I think it's best to discuss it with the bundler team directly.
Updated by MSP-Greg (Greg L) almost 3 years ago
GitHub Actions has added a Windows-2022 platform, and it only has MSYS2 packages installed, which provide most of the bash shell functionality. The MSYS2 packages do not include the MinGW build tool packages. This will require changes to ruby/setup-ruby to support the platform.
Rather than having every Windows Actions CI job pull build tool packages from the MSYS2 site, the idea of running an Actions CRON job (maybe once or twice daily) in another repo to gather the build tools, and package them in a zip (7z) file as a release. ruby/setup-ruby would download the appropriate file (mingw64 or ucrt64) and install the tools, which provides much faster CI. Personally, I think packaging all the tools needed to built Ruby would be best, and would be similar to Ubuntu & macOS. This install concept was actually used in Actions before MSYS2 was added to the images.
This would also mean that MSP-Greg/setup-ruby-pkgs wouldn't be needed for updates to build tools, as it's currently used for in many repos.
I'm mentioning this as the work needed to support Windows-2022 in Actions is extensive, and moving forward with Ruby ucrt64 builds would be helpful so that the Windows-2022 support could be done in one pass...
Updated by MSP-Greg (Greg L) almost 3 years ago
I said:
'This would also mean that MSP-Greg/setup-ruby-pkgs wouldn't be needed for updates to build tools, as it's currently used for in many repos.'
This would only be true if repos only run CI on the Window-2022 platform.
Updated by MSP-Greg (Greg L) over 2 years ago
This can be closed.
Updated by jeremyevans0 (Jeremy Evans) over 2 years ago
- Status changed from Open to Closed