Misc #21458: Test 'make install'?

• Status: Open
• Assignee: -

[MSP-Greg (Greg L)]

Can make install be tested? Two items come to mind:

1. Test that all gems can be required?
2. Check that all gem binstubs are installed and work.

Some of the above are being done in ruby-loco and https://github.com/ruby/ruby-dev-builder/blob/master/cli_test.rb. Both of these are cron jobs, they don't run on every commit.

A long time ago, I think this was discussed, but it was back when CI was much more limited. It was also mixed in with the idea of running tests from the 'install' folder, rather than the 'build' folder.

Maybe a few jobs could be added to check make install on the three main OS's (Ubuntu, macOS, Windows)?

Lastly, with minimal changes, tests can be run from the install folder, which is what ruby-loco has done for years.

Recently, two tests that I believe are skipped here started failing:

TestGemRequire#test_realworld_default_gem                 Line: 471  
Expected "" to not be empty.

TestGemRequire#test_realworld_upgraded_default_gem        Line: 492  
<"999.99.9"> expected but was
<"/ruby/tmp/test_rubygems_20250628-8920-sifyoe/test_realworld_upgraded_default_gem.rb:3:in '<main>': undefined method 'version' for nil (NoMethodError)">.

[MSP-Greg (Greg L)]

I'm asking about this because https://github.com/ruby/ruby-dev-builder/actions has been failing for five days, and ruby-loco for a similar time.

[Eregon (Benoit Daloze)]

MSP-Greg (Greg L) wrote:

Maybe a few jobs could be added to check make install on the three main OS's (Ubuntu, macOS, Windows)?

I agree, this is important to add, and means it's tested in a way end users use CRuby.

Maybe there are already some tests ran on a make install-ed Ruby?

[Eregon (Benoit Daloze)]

MSP-Greg (Greg L) wrote in #note-1:

I'm asking about this because https://github.com/ruby/ruby-dev-builder/actions has been failing for five days, and ruby-loco for a similar time.

Looking at https://github.com/ruby/ruby-dev-builder/actions/runs/15981482361/job/45076664975

Run bundle install
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies...
Fetching path 2.1.0
Installing path 2.1.0
Bundle complete! 3 Gemfile dependencies, 4 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

Run bundle exec rake --version
bundler: failed to load command: rake (/home/runner/.rubies/ruby-head/bin/rake)
/home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/runtime.rb:317:in 'Bundler::Runtime#check_for_activated_spec!': You have already activated json 2.12.2, but your Gemfile requires json 2.2.0. Since json is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports json as a default gem. (Gem::LoadError)
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/runtime.rb:25:in 'block in Bundler::Runtime#setup'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/spec_set.rb:233:in 'Array#each'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/spec_set.rb:233:in 'Bundler::SpecSet#each'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/runtime.rb:24:in 'Enumerable#map'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/runtime.rb:24:in 'Bundler::Runtime#setup'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler.rb:166:in 'Bundler.setup'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/setup.rb:32:in 'block in <top (required)>'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/ui/shell.rb:173:in 'Bundler::UI::Shell#with_level'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/ui/shell.rb:119:in 'Bundler::UI::Shell#silence'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/setup.rb:32:in '<top (required)>'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/cli/exec.rb:57:in 'Kernel#require_relative'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/cli/exec.rb:57:in 'Bundler::CLI::Exec#kernel_load'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/cli/exec.rb:23:in 'Bundler::CLI::Exec#run'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/cli.rb:452:in 'Bundler::CLI#exec'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/vendor/thor/lib/thor/command.rb:28:in 'Bundler::Thor::Command#run'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/vendor/thor/lib/thor/invocation.rb:127:in 'Bundler::Thor::Invocation#invoke_command'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/vendor/thor/lib/thor.rb:538:in 'Bundler::Thor.dispatch'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/cli.rb:35:in 'Bundler::CLI.dispatch'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/vendor/thor/lib/thor/base.rb:584:in 'Bundler::Thor::Base::ClassMethods#start'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/cli.rb:29:in 'Bundler::CLI.start'
	from /home/runner/.rubies/ruby-head/lib/ruby/gems/3.5.0+2/gems/bundler-2.7.0.dev/exe/bundle:28:in 'block in <top (required)>'
	from /home/runner/.rubies/ruby-head/lib/ruby/3.5.0+2/bundler/friendly_errors.rb:118:in 'Bundler.with_friendly_errors'
	from /home/runner/.rubies/ruby-head/lib/ruby/gems/3.5.0+2/gems/bundler-2.7.0.dev/exe/bundle:20:in '<top (required)>'
	from /home/runner/.rubies/ruby-head/bin/bundle:25:in 'Kernel#load'
	from /home/runner/.rubies/ruby-head/bin/bundle:25:in '<main>'
Error: Process completed with exit code 1.

This looks like a RubyGems/Bundler bug at first sight, could you report it there?
I wonder what is loading json 2.12.2 before Bundler can load the Gemfile.