Project

General

Profile

Actions

Feature #20187

closed

Bundled gems at Ruby 3.4

Added by hsbt (Hiroshi SHIBATA) 10 months ago. Updated 9 months ago.

Status:
Closed
Target version:
-
[ruby-core:116220]

Description

This ticket is for tracking bundled gems status at Ruby 3.4.

I added warnings for bundled gems of Ruby 3.3 at https://bugs.ruby-lang.org/issues/19351 and https://bugs.ruby-lang.org/issues/19776.

I started to extract(promote) target libraries to bundled gems. These are:

  • abbrev
  • base64
  • bigdecimal
  • csv
  • drb
  • getoptlong
  • mutex_m
  • nkf
  • observer
  • resolv-replace
  • rinda
  • syslog

Related issues 3 (2 open1 closed)

Related to Ruby master - Feature #19351: Promote bundled gems at Ruby 3.3Closedhsbt (Hiroshi SHIBATA)Actions
Related to Ruby master - Feature #20309: Bundled gems for Ruby 3.5Assignedhsbt (Hiroshi SHIBATA)Actions
Related to Ruby master - Feature #20859: Make Base64 to core classAssignedhsbt (Hiroshi SHIBATA)Actions
Actions #1

Updated by hsbt (Hiroshi SHIBATA) 10 months ago

  • Subject changed from Bundled gems at Ruby 3.3 to Bundled gems at Ruby 3.4

Updated by hsbt (Hiroshi SHIBATA) 10 months ago

I already done to extract mutex_m and getoptlong. Now I'm working base64 and bigdecimal.

Actions #3

Updated by hsbt (Hiroshi SHIBATA) 10 months ago

Actions #4

Updated by jeremyevans0 (Jeremy Evans) 10 months ago

  • Tracker changed from Bug to Misc
  • Backport deleted (3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN)

Updated by hsbt (Hiroshi SHIBATA) 9 months ago

  • Status changed from Closed to Open

@Eregon (Benoit Daloze) reverts my changes at ruby/spec without any discussion. We shouldn't test bundled gems in make check. But he disagree it.

The current task environment is not intentional behavior from https://github.com/ruby/ruby/pull/6130

I propose disable them again at https://github.com/ruby/ruby/pull/9960.

I think bundled gems should keep testing their code in their repository, not ruby repository.
We only test them with make test-bundled-gems for supporting development version of ruby interpreter.

Updated by Eregon (Benoit Daloze) 9 months ago

hsbt (Hiroshi SHIBATA) wrote in #note-6:

@Eregon (Benoit Daloze) reverts my changes at ruby/spec without any discussion.

There is some discussion at https://github.com/ruby/ruby/commit/44d74f22c8da3c13aa5363769418e2f5fd29f65a#r138276491.
From that I understood that running these specs was failing under some circumstances, and I fixed that.

What is the problem with my fix, which adds extract-gems as a dependency of test-spec?

We shouldn't test bundled gems in make check. But he disagree it.

Yes, I think we should not stop running these specs, because it is likely to introduce regressions.
Your commits stopped running (33062-32468) = 594 specs/tests (1.8%), while they were passing and they do pass on a proper Ruby installation of 3.4 dev.

Whether a gem is default gem or bundled gem does not matter when running without Bundler (as it needs to be, otherwise any script doing e.g. require "bigdecimal" would break).
And so IMO ruby/spec should run the same whether a gem is a default or bundled gem (since it doesn't use Bundler).
When a gem is no longer bundled then of course we need to stop testing it in ruby/spec, but this is not the case here.

The current task environment is not intentional behavior from https://github.com/ruby/ruby/pull/6130

As there is more and more bundled gems I think it is valuable to test them and have them available without extra steps.

I propose disable them again at https://github.com/ruby/ruby/pull/9960.

Is there a concrete problem that this would solve?
I am not keen to disable so many specs, if it's only to avoid an extra extract-gems for test-spec, which seems pretty fast.
If it solves a serious and concrete problem then I am OK that this gets merged.

I think bundled gems should keep testing their code in their repository, not ruby repository.

That is a fair point, I agree the specs/tests should ideally live in the same repository as the implementation.
However there are still older Ruby versions and alternative Rubies where the implementation is in stdlib or in their repositories, so in that case having the specs in ruby/spec is good.
This can even be the case after a gem becomes bundled, because alternative Rubies might have their own implementation of them, e.g. digest implemented as Ruby/Java instead of a C extension.

I think so far it was never discussed in details how to move these specs to the bundled gem repository.
It probably deserves its own ticket to figure it out.
My main concern there is it would be a lot of effort to keep running the specs in the many bundled gems repositories vs just running ruby/spec (would need to integrate this as part of test-bundled-gems but also for all other Ruby implementations, which is lots of work. Would also need a way to sync these spec files from bundled gems to ruby/ruby).
And then of course upstreaming all this would be a lot of work too.
I guess the first thing we need is ruby/mspec as a gem, so it could easily be reused by bundled gems, like test-unit-ruby-core.
And then we can try upstreaming the specs to one of the bundled gems and see how it goes.

We only test them with make test-bundled-gems for supporting development version of ruby interpreter.

Maybe we could make test-spec part of test-bundled-gems or so.
But there also seems to be no need since make test-spec works as-is on master.

I think it would remove a lot of workarounds if ruby/spec was always run on a complete ruby (i.e., like a real Ruby installation, so after make install or very similar layout). But I understand some people have concerns about that because e.g. make install is not so fast/convenient (discussed previously on some other ticket).

Updated by hsbt (Hiroshi SHIBATA) 9 months ago

  • Status changed from Open to Assigned

I changed test policy for this with nalsh's suggestion.

https://github.com/ruby/ruby/pull/9960#issuecomment-1943371439


We will separate ruby/spec to two groups.

  • core and library for default gems
  • library for bundled gems

I will make test-spec for core and default gems and disable to load .bundle directory. After that, I will create new mspec definition for bundled gems like bigdecimal etc. test-bundled-gems invoke that new definition.

Updated by hsbt (Hiroshi SHIBATA) 9 months ago

  • Status changed from Assigned to Closed
Actions #10

Updated by Eregon (Benoit Daloze) 8 months ago

Actions #11

Updated by hsbt (Hiroshi SHIBATA) 4 days ago

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0