Feature #21311: Namespace on read (revised)

• Status: Assigned
• Assignee: tagomoris (Satoshi Tagomori)
• Target version: 3.5

[tagomoris (Satoshi Tagomori)]

This replaces #19744

Concept

This proposes a new feature to define virtual top-level namespaces in Ruby. Those namespaces can require/load libraries (either .rb or native extension) separately from other namespaces. Dependencies of required/loaded libraries are also required/loaded in the namespace.

This feature will be disabled by default at first, and will be enabled by an env variable RUBY_NAMESPACE=1 as an experimental feature.
(It could be enabled by default in the future possibly.)

"on read" approach

The "on write" approach here is the design to define namespaces on the loaded side. For example, Java packages are defined in the .java files and it is required to separate namespaces from each other. It can be implemented very easily, but it requires all libraries to be updated with the package declaration. (In my opinion, it's almost impossible in the Ruby ecosystem.)

The "on read" approach is to create namespaces and then require/load applications and libraries in them. Programmers can control namespace separation at the "read" time. So, we can introduce the namespace separation incrementally.

Motivation

The "namespace on read" can solve the 2 problems below, and can make a path to solve another problem:

• Avoiding name conflicts between libraries
  • Applications can require two different libraries safely which use the same module name.
• Avoiding unexpected globally shared modules/objects
  • Applications can make an independent/unshared module instance.
• Multiple versions of gems can be required
  • Application developers will have fewer version conflicts between gem dependencies if rubygems/bundler will support the namespace on read. (Support from RubyGems/Bundler and/or other packaging systems will be needed)

For the motivation details, see [Feature #19744].

How we can use Namespace

# app1.rb
PORT = 2048
class App
  def self.port = ::PORT
  def val = PORT.to_s
end

p App.port # 2048

# app2.rb
class Number
  def double = self * 2
end

PORT = 2048.double
class App
  def self.port = ::PORT
  def val = PORT.double.to_s
end

p App.port # 4096

# main.rb - executed as `ruby main.rb`
ns1 = Namespace.new
ns1.require('./app1') # 2048
ns2 = Namespace.new
ns2.require('./app2') # 4096

PORT = 8080
class App
  def self.port = ::PORT
  def val = PORT.to_s
end

p App.port # 8080
p App.new.val # "8080"

p ns1::App.port # 2048
p ns1::App.new.val # "2048"

p ns2::App.port # 4096
p ns2::App.new.val # "8192"

1.double # NoMethodError

Namespace specification

Types of namespaces

There are two namespace types, "root" and "user" namespace. "Root" namespace exists solely in a Ruby process, and "user" namespaces can be created as many as Ruby programmers want.

Root namespace

Root namespace is a unique namespace to be defined when a Ruby process starts. It only contains built-in classes/modules/constants, which are available without any require calls, including RubyGems itself (when --disable-gems is not specified).

At here, "builtin" classes/modules are classes/modules accessible when users' script evaluation starts, without any require/load calls.

User namespace

User namespace is a namespace to run users' Ruby scripts. The "main" namespace is the namespace to run the user's .rb script specified by the ruby command-line argument. Other user namespaces ("optional" namespaces) can be created by Namespace.new call.

In user namespace (both main and optional namespaces), built-in class/module definitions are copied from the root namespace, and other new classes/modules are defined in the namespace, separately from other (root/user) namespaces.
The newly defined classes/modules are top-level classes/modules in the main namespace like App, but in optional namespaces, classes/modules are defined under the namespace (subclass of Module), like ns::App.

In that namespace ns, ns::App is accessible as App (or ::App). There is no way to access App in the main namespace from the code in the different namespace ns.

Constants, class variables and global variables

Constants, Class variables of built-in classes and global variables are also separated by namespace. Values set to class/global variables in a namespace are invisible in other namespaces.

Methods and procs

Methods defined in a namespace run with the defined namespace, even when called from other namespaces.
Procs created in a namespace run with the defined namespace too.

Dynamic link libraries

Dynamic link libraries (typically .so files) are also loaded in namespaces as well as .rb files.

Open class (Changes on built-in classes)

In user namespaces, built-in class definitions can be modified. But those operations are processed as copy-on-write of class definition from the root namespace, and the changed definitions are visible only in the (user) namespace.

Definitions in the root namespace are not modifiable from other namespaces. Methods defined in the root namespace run only with root-namespace definitions.

Enabling Namespace

Specify RUBY_NAMESPACE=1 environment variable when starting Ruby processes. 1 is the only valid value here.

Namespace feature can be enabled only when Ruby processes start. Setting RUBY_NAMESPACE=1 after starting Ruby scripts performs nothing.

Pull-request

https://github.com/ruby/ruby/pull/13226

[baweaver (Brandon Weaver)]

As a proof of concept this is a very valuable idea, and will give users a chance to experiment with it.

I wonder about the long-term ergonomics of this though, and if it may make sense to potentially introduce in Ruby 4 a new keyword for namespace that is stronger than module for wrapping:

namespace NamespaceOne
  require "./app1"
end

namespace NamespaceTwo
  require "./app2"
end

p NamespaceOne::App.port # 2048
p NamespaceOne::App.val # "2048"

p NamespaceTwo::App.port # 4096
p NamespaceTwo::App.val # "8192"

A require that is run inside of a namespace could serve the same function mentioned above, but could additionally provide an isolate environment for defining other code:

namespace Payrolls
  class Calculator; end
  private class RunTaxes; end
end

Payrolls::Calculator # can access
Payrolls:RunTaxes # raises violation error

namespace Payments
  class RecordTransaction; end
end

For Ruby 3.x I would agree that the proposed syntax is good for experimentation, but would ask that we consider making this a top-level concept in Ruby 4.x with a namespace keyword to fully isolate wrapped state.

[fxn (Xavier Noria)]

A few quick questions:

Assuming a normal execution context, nesting at the top level of a file is empty. Would it be also empty if the file is loaded under a namespace?

The description mentions classes and modules, which is kind of intuitive. They are relevant because they are the containers of constants. But, as we know, constants can store anything besides class and module objects. In particular, constants from the root namespace, recursively, can store any kind of object that internally can refer to any other object. There is a graph of pointers.

So, when a namespace is created, do we have to think that the entire object tree is deep cloned? (Maybe with CoW, but conceptually?) For example, let's imagine C::X is a string in the root namespace, and we create ns. Would ns::C::X.clear clear the string in both namespaces?

Global variables stay global I guess?

[tagomoris (Satoshi Tagomori)]

@baweaver I don't have strong opinion about adding namespace keyword, but having a block parameter on Namespace.new could provide similar UX without changing syntax.

NamespaceOne = Namespace.new do
  require "./app1"
end
p NamespaceOne::App.port #=> 2048

This looks a less smart but may not worst. Having Kernel#namespace could be an alternative idea.

NamespaceOne = namespace do
  require "./app1"
end

[tagomoris (Satoshi Tagomori)]

fxn (Xavier Noria) wrote in #note-2:

A few quick questions:

Assuming a normal execution context, nesting at the top level of a file is empty. Would it be also empty if the file is loaded under a namespace?

Yes. At that time, self will be a cloned (different) object from main in optional namespaces.

So, when a namespace is created, do we have to think that the entire object tree is deep cloned? (Maybe with CoW, but conceptually?)

Conceptually, yes. Definitions are deeply cloned. But objects (stored on constants, etc) will not be cloned (See below).

For example, let's imagine C::X is a string in the root namespace, and we create ns. Would ns::C::X.clear clear the string in both namespaces?

Yes. (I hope built-in classes/modules don't have such mutable objects, but those should have :-( )

Global variables stay global I guess?

Global variables are also separated by namespace. Imagine $LOAD_PATH and $LOADED_FEATURES that have different sets of load paths and actually loaded file paths, which should be different from each other namespace.
Providing protection for unexpected changes of global variables by libraries or other apps is a part of namespace concept.

[tagomoris (Satoshi Tagomori)]

• Description updated (diff)

[fxn (Xavier Noria)]

Thanks @tagomoris (Satoshi Tagomori).

Conceptually, yes. Definitions are deeply cloned. But objects (stored on constants, etc) will not be cloned (See below).

Let me understand this one better.

In Ruby, objects are stored in constants. Conceptually, a constant X storing a string object and a constant C storing a class object are not fundamentally different. Do you mean namespace creation traverses constant trees, clones only the values that are class and module objects, and keeps the rest of object references, which become shared between namespaces?

Even in the case of classes and modules, what happens to the objects in their ivars?

I do not know about builtin, but in the case of user-defined classes/modules, I don't think we can assume they do not mutate their state. We could have 2500 of them in the root namespace when the namespace is created.

[tagomoris (Satoshi Tagomori)]

fxn (Xavier Noria) wrote in #note-6:

In Ruby, objects are stored in constants. Conceptually, a constant X storing a string object and a constant C storing a class object are not fundamentally different. Do you mean namespace creation traverses constant trees, clones only the values that are class and module objects, and keeps the rest of object references, which become shared between namespaces?

For example, String is a built-in class and Class object value, stored as ::String constant. And in a namespace ns1, we can change String definition (for example, adding a constant String::X = "x").
But even in that case, the value of String is identical. ::String == ns1::String returns true.

That means, the value (VALUE in CRuby world) is identical and not copied when namespaces are created, but the backed class definition (struct rb_classext_t) are different and those are the CoW target.

Even in the case of classes and modules, what happens to the objects in their ivars?

Class ivars (instance variable tables of classes) are copied, but the ivar values are not copied. It's similar to constants (constant tables) of classes.

I do not know about builtin, but in the case of user-defined classes/modules, I don't think we can assume they do not mutate their state. We could have 2500 of them in the root namespace when the namespace is created.

In the namespace context, "builtin classes/modules" are classes and modules defined before any user-script evaluation. (I'll update the ticket description soon.)
The total number of those are, classes 685, modules 40 (and internal iclass 51). any user-defined classes/modules are not defined in the root namespace.

[tagomoris (Satoshi Tagomori)]

• Description updated (diff)

[fxn (Xavier Noria)]

any user-defined classes/modules are not defined in the root namespace.

Ah, that is key.

So, what happens in this script?

# main.rb

App = Class.new

ns1 = Namespace.new
ns1.require("./app1") # defines/reopens App

do App and ns1::App have the same object ID?

Or does the feature assume that if you want to isolate things that has to be the first thing before creating any constant, global variable, etc.?

[byroot (Jean Boussier)]

having a block parameter on Namespace.new could provide similar UX without changing syntax.

That wouldn't handle constant definitions correctly though. Similar to how people get tricked by Struct.new do today.

Foo = Struct.new(:bar) do
  BAZ = 1 # This is Object::BAZ
end

That's why I filed [Feature #20993], it would allow you to do:

module MyNamespace = Namespace.new
  BAZ = 1 # This is MyNamespace::BAZ
end

[Eregon (Benoit Daloze)]

@fxn The main and user namespaces are independent, though the main namespace can refer to user namespace via ns::SomeConstant.
So the App from main here is inaccessible in ns1, in fact all constants defined in the main namespace are inaccessible in user namespaces, see the end of https://bugs.ruby-lang.org/issues/21311#User-namespace.

[Eregon (Benoit Daloze)]

I think this addresses https://bugs.ruby-lang.org/issues/19744#note-74 by having a CoW copy of all builtin classes/modules in each namespace (including main namespace), nice.
From a quick read it sounds correct to me.
The semantics might be somewhat surprising in practice:

• e.g. String#start_with? is available in all namespaces but String#to_time is only available in the namespaces that load activesupport (clear if you know which methods are core but as we have seen from polls it is not always clear)
• core classes&modules are copy-on-write and shared references, but user-defined classes&modules are completely separate (except main namespace can reference anything from other namespace explicitly through ns::Foo and even store them), it's kind of a dual situation and a bit inconsistent. I think it's necessary semantically though, as a String from another namespace should still be obj.is_a?(String).
• Any user-defined class instance won't be is_a? in another namespace and this might be particularly confusing for stdlib/default gems/bundled gems, e.g. a Date or Pathname created in ns1 won't be is_a?(Date) in main, e.g. ns1::TODAY.is_a?(Date) # => false or ns1::Date.today # => false. Also Pathname('/') == ns1::Pathname('/') # => false. (all these examples run in the main namespace)

For the last point I suspect one might need a way to transition objects from a namespace to another somehow, which sounds hard.
Unless they truly need no communication at all between namespaces, but then different processes (or multiple interpreters in a process) might be a better trade-off (notably can run in parallel and stronger isolation).

[byroot (Jean Boussier)]

While I believe namespaces would be a good addition to Ruby, I'm not convinced this particular implementation of
namespaces is what Ruby needs.

First, I'm not convinced by the motivations:

Avoiding name conflicts between libraries: Applications can require two different libraries safely which use the same module name.

Is this a problem that happens on a regular basis? I believe Ruby has a pretty well established convention
for libraries to expose a single module with a name that correspond to their gem name.

Actual top level module name clashes are extremely rare in my experience.

Avoiding unexpected globally shared modules/objects

Here again, from my experience this is very rare, and usually accepted as a bug, and promptly fixed.

Do we have concrete cases of this being a peristent problem?

Multiple versions of gems can be required

I remember there was discussions about this in the past. Personally this is a feature it's quite strongly
against because it's extremely hard to reason about.

If you have library A using the gem G in version 1, and library B using the gem G in version 2,
and end up with A being passed a G-v2 object, you may end up in a world of hurt.

I understand this feature would be useful for bundler specifically to allow them to use gems internally
without conflicting with the application (a problem they currently solve by vendoring), but outside
of that I'm not convinced it's a desirable feature.

I get that it can happen that you end up in a sticky situation with two dependencies being essentially
incompatible because they require conflicting versions of another dependency, as it happened with the Faraday 2
transition a few years back, but I'm not convinced that working around the problem that way is a net positive.

Namespace monkey patches

This one isn't in your ticket, but from previous public talks I understand it is one?

Here again I'd like to question how big of a problem monkey patches really are.
It is true that 15 years ago, numerous popular gems would irresponsibly monkey patch core classes,
but I believe these days are long gone. Except for ActiveSupport (that gets a pass for being a framework)
very few gems ship with monkey patch.

A notable exception being "protocol" type of methods, such as to_json, to_yaml, to_msgpack, etc.

In addition, I routinely use monkey patches to backport a fix onto a gem while waiting for a fix to be merged
and published upstream. If monkey patches became scoped to namespaces, this would make this sort of "monkey patches"
way harder. So to me it's net negative.

Being able to namespace existing code

Again not listed in your motivations, but you explain pretty well that you want to be able to load arbitrary code
into a namespace, because you don't want to have to modify the existing libraries.

It makes sense, but is it really that big of a need? I personally see namespaces as a feature libraries can
use to write more robust and isolated code. Not as a feature applications can use to workaround libraries.

Other issues

Deduplication

Assuming this implementation of namespaces become largely used, it means some versions of some libraries would
be loaded dozens and dozens of time in the same process. IIRC in some previous public talks you mentioned
the possibility of deduplication, what's the status on this? Because without it, it's a big concern to me.

With Python/Java/Node namespacing systems it's an easily solved problem, because the file is essentially a
namespace objects, so you can just keep a map of file -> namespace_object, but here it seems way more involved.

What I think would be a positive

In order to not just be negative, I'll try to explain what I think would be helpful.

Local namespace

A common complaint I hear from less experienced / occasional Ruby users is they are having trouble figuring out where constants are comming from,
because of the single global namespace.
They prefer the Java/Python/Node style, where each file is more or less its own namespace, and at the top
of the file you list your imports.

I think translated in Ruby, it could be emulated by only allowing to reference constants from outside the namespace
in a fully qualified way:

class SomeClass
end

namespace MyLibrary
  p SomeClass # NameError

  SomeClass = ::SomeClass # This is basically an import

  p SomeClass # works
end

In other word, I think namespaces could be somewhat similar to BasicObject but for modules.

Overly public constants

Another common issue I witnessed is publicly exposed constants, that aren't meant to be public.

Being involved in a really big application, what people are trying to do to make that codebase more manageable
is to break it down in smaller components with the hope that a developer can more easily wrap their head around
a single component, that a component can be tested individually, etc.

This often fall appart because all constants are public by default, so other teams end up relying on APIs that
weren't meant to be used.

I think it would be helpful if namespaces constants were private by default and you had to explictly "export" (publicize)
them.

[Eregon (Benoit Daloze)]

(from description)

There is no way to access App in the main namespace from the code in the different namespace ns.

Right, although of course the main namespace can do ns1::MainApp = App and expose its class like that.

I wonder if there should be a way to get the Namespace object of the main namespace.
Then the main and user namespaces wouldn't have any difference besides the main namespace beind the default/starting namespace, as if the main script was executed under main_ns = Namespace.new; main_ns.require(main_script).

[Eregon (Benoit Daloze)]

Has the performance of Namespace been evaluated?
I would assume getting the current namespace to execute methods/procs is an overhead (the namespace is at least needed for constant accesses and for method lookup on builtin classes).

At least any shared code (so probably only methods/procs from the root namespace) doing constant lookup is likely slower as it needs to lookup from the current namespace / from the correct struct rb_classext_t.
Are constant inline caches disabled for such methods, if not how does it work and avoid invalidating those caches?

Same for method lookup on builtin classes, what about method lookup inline caches for method calls inside root namespace methods/procs?

[Eregon (Benoit Daloze)]

@byroot (Jean Boussier) makes a good point about use cases, I share the same concerns (and already did in https://bugs.ruby-lang.org/issues/19744#note-21 a while ago).
It seems easy to avoid these problems and these problems don't seem to come up frequently either.
I'm not sure adding such a big feature for these seemingly rather-niche issues is worth it.

From TruffleRuby's POV I am unsure it makes to implement Namespace there, when there is stronger isolation already available, more performant and with simpler semantics.

[Eregon (Benoit Daloze)]

• Related to Feature #19744: Namespace on read added

[fxn (Xavier Noria)]

The main and user namespaces are independent

@Eregon (Benoit Daloze) the description says

User namespace is a namespace to run users' Ruby scripts. The "main" namespace is the namespace to run the user's .rb script specified by the ruby command-line argument. Other user namespaces ("optional" namespaces) can be created by Namespace.new call.

The vocabulary is not very clear to me. What is a "script", is active_record.rb a script? If "User namespace is a namespace to run users' Ruby scripts" and also "The "main" namespace is the namespace to run the user's .rb script specified by the ruby command-line argument.", is main a user namespace?

I find the description also a bit hard to follow at times because it conflates constants and the objects they store. Classes and modules are value objects like any other value object. For example, they are not top-level or not top-level, constants that belong to Object are top-level. In some cases I can translate to what seems to be the intention, but to describe a feature like this I think it would be more clear that we are technically sharp.

I also see the concerns raised by @byroot (Jean Boussier).

[Dan0042 (Daniel DeLorme)]

byroot (Jean Boussier) wrote in #note-13:

I personally see namespaces as a feature libraries can use to write more robust and isolated code. Not as a feature applications can use to workaround libraries.

It's not about "working around" libraries. It's about loading entirely different and independent apps within the same process. All the motivations presented above are in service of that. Imagine a basic router that accepts requests and routes them to 2 namespaced apps A and B. A is a Rails 6 app, B is a Rails 7 app. Completely different worlds that do not ever interact so "end up with A being passed a G-v2 object" is not an issue in practice.

The 2 apps could be run as separate processes, but

1. The routing would have to be handled by something else (nginx?) and much less flexible than ruby
2. Concurrency is harder to control; if you want a limit of 10 concurrent requests, then you need 10 processes each of A and B, plus some external synchronization to ensure that of those 20 processes only 10 are ever active at one time.

I'm not sure if this can be achieved with TruffleRuby sub-interpreters or how you would go about it.

[fxn (Xavier Noria)]

My interpretation of the vocabulary is as following:

1. The interpreter boots in a root namespace.
2. The "state" when this process ends is somehow snapshotted into S or something, conceptually (plenty of details here, but I am talking only about the vocabulary).
3. Any other namespace is a user namespace. This is a concept, not a name.
4. When the interpreter starts interpreting "external" code, it creates a user namespace called main, and initializes it with S.
5. If the code in main spawns other namespaces, they are also user namespaces, and are initialized with S (for example, the constants and global variables in main are not inherited. In particular, the arrays stored in $LOADED_FEATURES, $LOAD_PATH, etc. are new array references whose initial items are as in root).
6. Such optional namespaces can in turn spawn namespaces, and they are initialized with S too.

So, I guess spawning does not create hierarchy. A namespace may not have a pointer to the namespace where it was born (or I don't see the need for it after creation). It is conceptually just one flat layer of independent user namespaces.

Is that a good model?

@tagomoris (Satoshi Tagomori) is that correct?

[fxn (Xavier Noria)]

If my interpretation is correct, the entry point of a library will be executed as many times as user namespaces require it, in the same operating system process.

That is a potential gotcha to have in mind too, since nowadays you can kind of assume that you'll be loaded at most once in the same process, and you may leverage that to run one-offs. With namespaces, you'd need to be idempotent by hand.

[byroot (Jean Boussier)]

Dan0042 (Daniel DeLorme) wrote in #note-19:

It's not about "working around" libraries. It's about loading entirely different and independent apps within the same process.

So that's not something I understood for the issue description. But if that's so, while I understand the appeal and see how it would be nice to have, I don't think it's useful enough to justify such a massive impact on the VM implementation. But that's just my opinion.

[peter.boling (Peter Boling)]

byroot (Jean Boussier) wrote in #note-13:

Avoiding name conflicts between libraries: Applications can require two different libraries safely which use the same module name.

Is this a problem that happens on a regular basis? I believe Ruby has a pretty well established convention
for libraries to expose a single module with a name that correspond to their gem name.

One use case for this is to benchmark libraries that do the same thing against each other. Very frequently libraries doing the same thing are forks of each other, and just as frequently are not re-namespaced.

As a concrete example, the memo_wise gem does this type of benchmarking, against all the other known "memoization" gems, including pitting its own latest release against its own unreleased HEAD. As a result there are many shared namespaces. This allows them to see if their changes are beneficial.

I wrote a tool in my gem gem_bench to assist them in that benchmarking. It loads a gem, and re-namespaces it, in a terrible, dirty, hacky way. It only works due to the relative simplicity of the libraries, and would not work in more complex cases. And that is the point I'm making here...

It isn't done much because it is damn hard to do. But it might be done more if we could easily load a bunch of otherwise conflicting tools and run them all at the same time. The benchmarking memo_wise has created from this is pretty cool. There are currently multiple open related pull requests, and we're actively working on it.

• https://github.com/panorama-ed/memo_wise/pulls
• https://github.com/panorama-ed/memo_wise?tab=readme-ov-file#benchmarks
• https://github.com/pboling/gem_bench

[byroot (Jean Boussier)]

peter.boling (Peter Boling) wrote in #note-23:

One use case for this is to benchmark libraries that do the same thing against each other. Very frequently libraries doing the same thing are forks of each other, and just as frequently are not re-namespaced.

I know it's going to read like I'm moving the goal post, but "very frequently" seem way too strong of a term here. This is rather rare.

But yes, benchmarking is indeed one use case. I myself often benchmark different versions of a single gem, for optimization purposes and this feature could be handy, but it's not like I can't do it today, and also benchmark-ips has the hold! and save! feature for exactly that purpose.

So I still think this is way too situational of a use case to justify such a massive change in the VM.

Because to me, in the end it's really about usefulness vs added implementation complexity.

Given the complexity of the implementation, I think it's very hard to justify unless it's envisioned as something that will be very largely adopted. You don't add 6k lines of C in the VM, and many extra indirections in performance sensitive codepaths just to make benchmarking a bit easier once in a while.

But perhaps this feature as designed would end up very useful and very used, I might just not see it yet. But I'd like to hear about common green path, day to day, use cases, not fringe needs.

[peter.boling (Peter Boling)]

Agreed on all points. I have little concept of the complexity of the implementation, and defer to you on that. I think there is value in considering that we can't entirely know how it might be used since we don't have it yet, and that it is a very powerful idea. The concurrent applications with a total request limit is an an interesting use case.

Dan0042 (Daniel DeLorme) wrote in #note-19:

Imagine a basic router that accepts requests and routes them to 2 namespaced apps A and B. A is a Rails 6 app, B is a Rails 7 app

Another potential use case is tools that dogfood themselves. I have a gem, kettle-soup-cover, a code coverage meta gem, which does a lot of work to remove constants, and reload files, so it can use itself while testing itself, getting code coverage on itself from itself. I had planned to release the code I'm using to manage this mess as a separate gem, because I see myself using it in other places. I've had to do the same thing multiple times in the past, and would prefer to have the solution library-fied and tested.

If I could load a version of the gem into a different namespace it would make the setup simpler.

• https://github.com/kettle-rb/kettle-soup-cover/blob/main/lib/kettle/change.rb

[ko1 (Koichi Sasada)]

FYI: you can try namespace PR on https://ruby.github.io/play-ruby/?run=14855772787&options=%7B%22arguments%22%3A%5B%5D%2C%22env%22%3A%7B%22RUBY_NAMESPACE%22%3A1%7D%7D&code=%0Adef+foo+%3D+%3Amain%0A%0Ans1+%3D+Namespace.new%0Ans1.require%28%27.%2Fa%27%29%0A%0Abegin%0A++p+A%0Arescue+NameError+%3D%3E+e%0A++p+e%0Aend%0A%0A%23---+a.rb%0A%0AA+%3D+%3Aa%0A%0Abegin%0A++foo%28%29%0Arescue+NoMethodError+%3D%3E+e%0A++p+e%0Aend%0A

Thank you katei-san and tagomori-san.

[fxn (Xavier Noria)]

I have compiled the branch and I am playing around a bit with miniruby to test my understanding of the feature.

Something that has caught my attention is that class and module names do not return constant paths. That is, the following snippet

A = Class.new
puts A.name

would print A normally, and something like #<Namespace:0x00000001009ef510>::A in a namespace.

I believe this proposal is more transparent than the previous one. However, the namespace is still not quite transparent.

You'd like that the gem being loaded "in a namespace" works as if it was in the main namespace. As it would happen in a subprocess forked at the start of the entrypoint. But, intuitively, I think it is going to be hard to achieve that level of isolation without introducing a new entity in the language with its own new set of rules.

[tagomoris (Satoshi Tagomori)]

I'm sorry that I skipped to make effort to describe the motivation and copied the motivation section from the past ticket (and it was not well described enough).
Let me explain the motivation in detail. I had some mixed points below:

Being able to namespace existing code and libraries

By using namespace, we can mount two, or more, dozens of different Rack applications on a Rack app server.
The use cases of this feature are:

• Mounting server-less applications (run on AWS Lambda, separated on processes in production) on a single app server for development
  • Apps may have conflicting classes (like User)
  • Apps may have conflicting dependencies
  • Apps need to load different set of environment variables
  • (This is my original motivation to development namespace)
• Mounting two different revisions of an application for useful deployments (older and newer commits of an app)
  • App server can implement in-process blue-green deployment
  • App server can check diffs of responses of two revisions from a (duplicated) request
  • App server can compare response time of two revisions from a (duplicated) request
• Mounting two different set of dependency versions of an application
  • This is a kind of blue-green deployments, but for dependencies without any app code diffs

The first one is to update my app server LFA (https://github.com/tagomoris/LFA). And I believe I would be very positive to try implementation of the second/third one if I were maintaining web apps written in Ruby.

Avoiding unexpected globally shared modules/objects

My case of this is Fluentd and Oj. Oj has a global set of configuration Oj.default_options. It can be set from anywhere, and it can break the behavior globally.
Many of Fluentd plugins use Oj, and the behavior of Oj may be changed by options set by different plugins.
It seems a kind of bug and should not happen, but Oj still doesn't have per-instance configuration APIs.

Multiple versions of gems can be required

Fluentd too. Plugins are different and independent software developed by many different maintainers, having different sets of dependencies, but those plugins can be loaded on a Fluentd process.
In my opinion, any other software having plugin systems should have same problem potentially.

Namespace monkey patches

It may not happen frequently now, but potentially happens, and (in my opinion) it prevents us to implement Ruby core feature in Ruby.
I'm unsure how it actually prevents Ruby committers, but if built-in Ruby code are protected from monkey patches by namespace, we can implement many things in Ruby.

For example, require/load mechanism is implemented in C, but most of those code are just doing path construction (string operations) and file system accesses. It's really obvious that Ruby is much better language than C in such purpose. And now, it can be JIT-ed!

I'm really unsure how monkey patch risk prevents committers to do those things, so this section can be ignorable :P

[tagomoris (Satoshi Tagomori)]

@fxn

So, I guess spawning does not create hierarchy. A namespace may not have a pointer to the namespace where it was born (or I don't see the need for it after creation). It is conceptually just one flat layer of independent user namespaces.

Is that a good model?

@tagomoris (Satoshi Tagomori) (Satoshi Tagomori) is that correct?

Correct.

If my interpretation is correct, the entry point of a library will be executed as many times as user namespaces require it, in the same operating system process.
That is a potential gotcha to have in mind too, since nowadays you can kind of assume that you'll be loaded at most once in the same process, and you may leverage that to run one-offs. With namespaces, you'd need to be idempotent by hand.

There should be no needs for library developers to consider such things.
Namespace is designed to isolate side effects of "the entry point of a library" (except for some cases like modifying ENV). If a library uses class variables or global variables, these are separated by namespace. If a library uses class instance variables, class instances are different between namespaces.
What can you imagine other things changed in library entry points?

That is, the following snippet
A = Class.new
puts A.name
would print A normally, and something like #Namespace:0x00000001009ef510::A in a namespace.

This depends on how we implement the classpath. Currently, in my opinion, it should NOT consider the current context namespace and always return classpath with namespace (like #<Namespace:0x00000001009ef510>::A). But it can be changed and it can show just A in the namespace.

[byroot (Jean Boussier)]

Thank you for expanding on the motivation, the design makes more sense to me now.

I'm still not convinced the tradeoff is worth it, but at least it makes sense.

Out of curiosity, have to ran the yjit-bench suite? I know it's a preview, but I wonder what the overhead currently is.

[peter.boling (Peter Boling)]

byroot (Jean Boussier) wrote in #note-24:

But I'd like to hear about common green path, day to day, use cases, not fringe needs.

Just was reminded of another gem I wrote that utilizes this pattern.

I use it two ways.

1. When I am writing a library that provides functionality to a Rails Model, but I don't need to test against an entire Rails app. I can mixin the functionality from the gem, and test it with an anonymous active record.

2. Inside a Rails app when I want to test a concern of the app in a vanilla model without setting up a real table in the actual test DB (as it uses a discrete sqlite inmemory DB). This helps me prove a concern works as intended, and that interference from elsewhere is breaking the behavior. For me this is a very common use case.

https://rubygems.org/gems/anonymous_active_record

In point of fact, the models created by this gem are not actually anonymous, because Class.new(ActiveRecord::Base) hasn't worked for a long time. I'm not sure if this proposal would work given that, but if I have time I want to test the theory.

[fxn (Xavier Noria)]

@tagomoris (Satoshi Tagomori) thanks for your replies, even more with such a busy thread :). Let me followup:

There should be no needs for library developers to consider such things.
Namespace is designed to isolate side effects of "the entry point of a library" (except for some cases like modifying ENV). If a library uses class variables or global variables, these are separated by namespace. If a library uses class instance variables, class instances are different between namespaces.
What can you imagine other things changed in library entry points?

This is more of a technical concern. It falls in the category of "which things do we assume today that would no longer hold with the new feature". Since $LOADED_FEATURES is as it was in the root namespace, not just the entrypoint, but all the required files of the namespaced code (maybe required by itself recursively) are going to be executed as many times as user namespaces load that entrypoint.

OK, generally, that happens once today.

So the difference would matter if those files have side-effects like writing to the file system, forking, writing to a database, subscribe to Kafka topics, or anything of that sort that escapes Ruby.

This depends on how we implement the classpath. Currently, in my opinion, it should NOT consider the current context namespace and always return classpath with namespace (like #Namespace:0x00000001009ef510::A). But it can be changed and it can show just A in the namespace.

Something I have in mind there (but probably not the only thing) is that there is a lot of code out there that passes klass.name around to be later const_getted. That klass.name string could even be written in external storage to be later constantized by a different process.

[Eregon (Benoit Daloze)]

Dan0042 (Daniel DeLorme) wrote in #note-19:

I'm not sure if this can be achieved with TruffleRuby sub-interpreters or how you would go about it.

It's easy with sub-interpreters, you would create a sub-interpreter instead of a namespace per app.
Contrary to namespaces, sub-interpreters do not inherit any state from the initial interpreter, they're just "a clean new interpreter".
At least with TruffleRuby sub-interpreters it's possible to pass objects between sub-interpreters, this works by passing a proxy, if any method is called on the proxy, it's executed in the sub-interpreter owning that object.
So you can pass strings or requests objects or whatever from the router to the apps.

[tenderlovemaking (Aaron Patterson)]

Eregon (Benoit Daloze) wrote in #note-12:

• Any user-defined class instance won't be is_a? in another namespace and this might be particularly confusing for stdlib/default gems/bundled gems, e.g. a Date or Pathname created in ns1 won't be is_a?(Date) in main, e.g. ns1::TODAY.is_a?(Date) # => false or ns1::Date.today # => false. Also Pathname('/') == ns1::Pathname('/') # => false. (all these examples run in the main namespace)

For the last point I suspect one might need a way to transition objects from a namespace to another somehow, which sounds hard.
Unless they truly need to communication at all between namespaces, but then different processes (or multiple interpreters in a process) might be a better trade-off (notably can run in parallel and stronger isolation).

I don't think it's a good idea to support communication between namespaces. You could get in to a situation where you load 2 incompatible versions of the same library, allocate an object in one version, then pass the object to a second version. is_a? checks could pass, but the shape of the data is wrong. I've heard this is a problem in Node.JS which already supports loading multiple versions of one library (though probably not a super common problem).

[fxn (Xavier Noria)]

Aaron that is a concern I have also in mind that I have not expressed yer.

The current proposal allows access to the user namespaces from the creating one And believe some use cases in this thread re dogfood need that communication.

But I think that'd leak in several ways. One of them allowing an arbitrary number of class and module names from having the same permanent name if set from the outside.

[Eregon (Benoit Daloze)]

tagomoris (Satoshi Tagomori) wrote:

Root namespace is a unique namespace to be defined when a Ruby process starts. It only contains built-in classes/modules/constants, which are available without any require calls, including RubyGems itself (when --disable-gems is not specified).

RubyGems has a lot of state and is a non-trivial amount of code.
It's very likely not safe to share that between namespaces, e.g., the list of loaded gems, which should be per-namespace and not per-process.
Basically any state (which is any mutable object, excluding modules and classes which are basically CoW by namespaces, but not the contents of their constants, @ivars, @@cvars) in the root namespace is leaking between namespaces.

What can you imagine other things changed in library entry points?

What @fxn said and in addition I recall some cases where a native function needs to be called once per process and not more.
Now given C extensions are loaded once per namespace that may be fine in some cases, but probably isn't in other cases where it's touching truly-process-global state, or state escaping the process.

I'm really unsure how monkey patch risk prevents committers to do those things, so this section can be ignorable :P

I guess the main thing is if some core method is implemented in Ruby, it needs to be careful when calling methods, e.g. if it calls if argument.is_a?(Integer), is_a? can be overwritten.
Primitive are already the solution for such cases and work well, with even less overhead than a regular method call, so I don't think this is a particularly good use-case for namespaces.

For example, require/load mechanism is implemented in C, but most of those code are just doing path construction (string operations) and file system accesses. It's really obvious that Ruby is much better language than C in such purpose. And now, it can be JIT-ed!

FWIW TruffleRuby does implement significant parts of require/load in Ruby, including the $LOADED_FEATURES cache.
Ruby code is certainly more readable than C code, but it also has warmup cost and require/load is done early so very warmup-sensitive.
I don't believe JITing such code with YJIT would be faster than the C implementation of it.
Using Ruby means a lot of extra abstractions compared to C, and that's not free.
Even TruffleRuby does not run Ruby code as fast as equivalent C code (it's strictly impossible in general due to the richer Ruby semantics).
All that said, I would love if CRuby would rewrite or document some of that logic, I find load.c particularly hard to follow and I don't think I'm alone on that.

[Eregon (Benoit Daloze)]

tagomoris (Satoshi Tagomori) wrote in #note-28:

Mounting server-less applications (run on AWS Lambda, separated on processes in production) on a single app server for development

Why not simply using different processes for that? CRuby is pretty fast to start (and server-less applications typically rather small). It also matches better what happens in production.
Reading the 2 main purposes at https://github.com/tagomoris/LFA it feels pretty niche to me, but I may be missing something.
I totally get the high value of being able to run those server-less applications locally, but why not just one process per server-less application?
What does the Ruby application server gains from being able to access all applications in the same process?

[fxn (Xavier Noria)]

Basically, I believe the idea is to have isolated execution contexts without forking, but is hard to achieve that by subclassing Module and tweeking things. Could be wrong, but it is my hunch. As I said, I believe this might need a new entity with new rules.

[Eregon (Benoit Daloze)]

Eregon (Benoit Daloze) wrote in #note-12:

Unless they truly need to communication at all between namespaces, but then different processes (or multiple interpreters in a process) might be a better trade-off (notably can run in parallel and stronger isolation).

I made a typo, and edited it on Redmine: to->no [communication].

tenderlovemaking (Aaron Patterson) wrote in #note-34:

I don't think it's a good idea to support communication between namespaces.

I'm not sure, but yeah it seems easy to use incorrectly.

is_a? checks could pass

It wouldn't, because for obj.is_a?(Foo), Foo is always different in different namespaces (unless explicitly assigned like ns1::Foo = Foo, and then it's truly the same class in both namespaces).

If there is no communication between namespaces then I don't see the value of namespaces, different processes or sub-interpreters (multiple interpreters in a process) seems a much better trade-off:

• the semantics are much simpler and better understood: "process-like isolation, nothing shared"
• no overhead on Ruby execution
• much simpler implementation

As a note it's also possible to have some form of communication for sub-interpreters.
That can be restricted to just primitive types (e.g. numbers and strings) or some core types to make it simpler/safer.

[matz (Yukihiro Matsumoto)]

Let's just merge and experiment. We won't know until we actually try it. Also, I nominate @tagomoris (Satoshi Tagomori) for committer.

Matz.

[hsbt (Hiroshi SHIBATA)]

@tagomoris (Satoshi Tagomori) Can you share your required/optional information to me? see https://github.com/ruby/ruby/wiki/Committer-How-To#required

[mame (Yusuke Endoh)]

I nominate @tagomoris (Satoshi Tagomori) (Satoshi Tagomori) for committer.

+1

[hsbt (Hiroshi SHIBATA)]

I added committer grants to @tagomoris (Satoshi Tagomori). Please let me know if you have extra permission or something.

[fxn (Xavier Noria)]

Congrats @tagomoris (Satoshi Tagomori) :).

Would you be open to consider having the permanent name of classes and modules be constant paths?

[tagomoris (Satoshi Tagomori)]

fxn (Xavier Noria) wrote in #note-44:

Congrats @tagomoris (Satoshi Tagomori) :).

Would you be open to consider having the permanent name of classes and modules be constant paths?

Thank you :D

I'm totally open to consider changing the current behavior around permanent classpath including namespace.
But I'll focus on fixing some minor issues then merging the branch into master as it is.
@fxn Could you open another independent ticket about it instead of continuing the discussion here?

[fxn (Xavier Noria)]

I'll do that, thank you!

[Eregon (Benoit Daloze)]

I think it would be good to compare this to "isolated execution contexts" which is the well-known approach to get such isolation, e.g.:

• CPython has sub-interpreters
• V8 (JavaScript) has isolates
• MRuby has mrb_state/mrb_open() (basically sub-interpreters)
• TruffleRuby and JRuby have sub-interpreters
• Scripting languages on the JVM have ScriptEngine/ScriptContext and GraalVM has Context
• Many use-cases which want to embed some other language often need such "isolated execution contexts"

They are all the same concept and well established.
I'll call this concept "sub-interpreters" for brevity.

Namespace on read is quite different and is not the same concept.
Here is a list of trade-offs I can think of:

• Namespace on read has no guarantee for isolation, so it is not possible to achieve some use cases which work fine with sub-interpreters
• Namespace on read loads everything in the same GC heap (cannot avoid it due to cross-namespace references). So with more apps it becomes slower. OTOH each sub-interpreter/isolate can have its own heap and GC.
• Namespace on read has complicated semantics, notably core classes are both shared (whatever was in the root namespace) and not shared (whatever is set/defined after). Sub-interpreter have the classical process-like isolation semantics.
• Implementation-wise, namespace is more complicated as there is subtle sharing going on CoW of core classes.
• Performance-wise, namespace has more overhead as explained in https://bugs.ruby-lang.org/issues/21311#note-15
• Sub-interpreters run in parallel. Namespaces do not. This is probably the biggest drawback. Ractor may help but Ractor is also incompatible with most gems/code and has lots of contention so doesn't scale linearly.

Implementing sub-interpreters is of course non-trivial work, but I would think not harder than implementing namespaces.

There is also an easy approximation of sub-interpreters: create N processes (or fork just after loading with --disable-gems) and use each process as a "sub-interpreter".
That runs in parallel, has separate heaps so parallel GC too and has the best isolation.
I don't think namespaces can beat that, except for the convenience of communication, but that can be done via pipes, sockets, etc.

IOW, sub-interpreters are similar to Namespace + Ractor but with proper isolation, full compatibility, linear scaling and clear semantics.

[Eregon (Benoit Daloze)]

In https://github.com/ruby/dev-meeting-log/blob/master/2025/DevMeeting-2025-05-08.md#discussion

mame: Are there any strong opponents?

I guess I am one.
There are many concerns from several people that have been raised here, many unanswered.
I think they should be answered before merging the PR.

It seems clear the performance should be checked before merging the PR, especially since the feature is not behind #ifdef.

There also doesn't seem to be a convincing use case that can't be done with just multiple processes.
At least that should be clearly shown, adding such complexity without a good use case seems a bad idea.

And the first paragraph https://bugs.ruby-lang.org/issues/21311#note-36 seems a clear bug.

In my opinion, to keep it short: we could have sub-interpreters (known to work well and have many advantages) or 2 half-working features like Namespace + Ractor (Ractor is incompatible with most gems, I don't think it's solvable, hence "half-working").
There are similarities between sub-interpreters and Namespace, e.g., it would be the same logic to copy C extensions to isolate them.
IOW, I think it would be far better to introduce sub-interpreters in CRuby than Namespace.

I understand wanting to merge soon to avoid conflicts but also there hasn't been a proper discussion on the advantages of Namespace vs sub-interpreters (which could have started long before the implementation was near-complete).
Given that TruffleRuby and JRuby and many other VMs have sub-interpreters, I think it's crucial to figure out if it's worth doing something different like Namespace (which can't run in parallel, has weaker isolation, has more overhead, etc, so seems worse in most aspects).

My concern is Namespace is going to be "another Ractor", i.e., basically a feature which is half though-out and doesn't actually work in practice for non-trivial cases.
Both might end up being CRuby-only features because no-GVL and sub-interpreters (which provides parallelism and full compatibility without removing the GVL BTW) are better alternatives.

[fxn (Xavier Noria)]

I am aligned with @Eregon (Benoit Daloze).

There are several important issues.

Another one that has not been mentioned (I think) is that the design trivially allows for two objects with the same object ID to be different entities. The Ruby programmer does not care about VALUE versus some internal C stuff we move around, if two objects have the same ID, they are the same object. But that is no longer the case.

Or, in a non-main user namespace, top-level constants do not belong to (the namespaced) Object. So, being evaluated within a namespace is, again, not transparent. Your code may work differently, while the goal is that it works as-is.

Obviously, there is a lot of work in this patch and I praise the effort to do this work. But, honestly, to me, it is at least premature to merge the patch with all these fundamental things open.

It is cool to make something experimental, but at least it has to be sound, in my view.

[fxn (Xavier Noria)]

Let me add.

This is not a case of "let's merge and polish". Problem is, it has to be seen if you can satisfy the goals with this approach at all. Because, you cannot be dazzled by the stated goals, you have to carefully check if the goals are actually satisfied by the solution.

And, personally, as I have said, I am skeptical that is achievable without introducing a fundamentally new concept in the language. (Happy to be proved wrong, though.)

[mame (Yusuke Endoh)]

This feature proposal is quite exceptional. Because this is a feature strongly driven by @matz (Yukihiro Matsumoto) himself, and Ruby is matz's language.

@tagomoris (Satoshi Tagomori) has been working closely with Matz for several years to design and implement it. Although there are still many incomplete or rough parts, the decision has already been made.

This feature will be soon merged with high priority. In fact, Matz even approved temporarily disabling some CI tests if necessary in order to proceed with the merge. (@ko1 (Koichi Sasada) is working hard right now to prevent that from happening.) Anyway, merging it takes precedence over discussion.

I am personally excited to see Ruby adding a bold new feature that brings real energy and discussion to the community. This is Ruby. Let's enjoy this moment and move forward together with excitement.

[tagomoris (Satoshi Tagomori)]

On my laptop, building fiddle fails and it prevents running make yjit-bench.

So I ran benchmark/app_fib.rb and got results (The result after several times continuous runs):

# Ruby 3.4.2
$ time ruby benchmark/app_fib.rb 

real	0m0.368s
user	0m0.296s
sys	0m0.019s
$ ruby -v
ruby 3.4.2 (2025-02-15 revision d2930f8e7a) +PRISM [arm64-darwin24]

# 3.5.0dev master HEAD
$ time build/exe/ruby benchmark/app_fib.rb 

real	0m0.301s
user	0m0.282s
sys	0m0.015s
$ build/exe/ruby -v
ruby 3.5.0dev (2025-05-10T06:59:40Z master 3c37d6ffcf) +PRISM [arm64-darwin24]

# 3.5.0dev namespace branch
$ time build/exe/ruby benchmark/app_fib.rb 

real	0m0.315s
user	0m0.294s
sys	0m0.018s
$ build/exe/ruby -v
ruby 3.5.0dev (2025-05-10T07:50:29Z namespace-on-read-.. bd4f57f96b) +PRISM [arm64-darwin24]

I haven't rebased on the current master HEAD yet, so it may be the reason of the diff between 0.294s and 0.282s.

And the result with RUBY_NAMESPACE=1:

$ time RUBY_NAMESPACE=1 build/exe/ruby benchmark/app_fib.rb 
build/exe/ruby: warning: Namespace is experimental, and the behavior may change in the future!
See doc/namespace.md for know issues, etc.

real	0m0.302s
user	0m0.286s
sys	0m0.013s

I can't understand why it's quicker than the one without namespace...
(ISeq inline cache works well even with namespace, so it's not surprising to me if the result with RUBY_NAMESPACE=1 is not slower than without.)

[byroot (Jean Boussier)]

So I ran benchmark/app_fib.rb

Fibonnaci isn't a very interesting benchmark here. It's just the same method being called over and over.

yjit-bench contains some macro benchmark that are more likely to have inline cache misses hence will have to go through the extra layers of indirections introduced by the patch.

[kou (Kouhei Sutou)]

building fiddle fails

Could you share the error message?

[fxn (Xavier Noria)]

@mame (Yusuke Endoh)

I am personally excited to see Ruby adding a bold new feature that brings real energy and discussion to the community. This is Ruby. Let's enjoy this moment and move forward together with excitement.

Absolutely.

I would have waited a little bit more to pass a minimal of community scrutiny before merging as experimental. Which is what is happening against the clock in the bug tracker right now. Some of us are experienced in this corner of the language and can detect inconsistencies just by reading the spec or writing couple of scripts. Others have performance and complexity trade-offs in their radar. Others may see the implications in internal caches, etc.

Open Source, sharing and discussing.

The feature itself is cool on paper. I want to stress that I am not against the feature, and that I absolutely respect the merge and any decisions @matz (Yukihiro Matsumoto) does, as it could not be otherwise.

I am a mathematician by training. If presented with something, I'll try to look for the holes and inconsistencies. And that is my way to contribute to this. Please do not take my QA as something against the feature or personal ❤️.

[tagomoris (Satoshi Tagomori)]

kou (Kouhei Sutou) wrote in #note-54:

building fiddle fails

Could you share the error message?

I couldn't find the log when fiddle is built during make yjit-bench...
Anyway, I'll retry yjit-bench on Linux.

[tagomoris (Satoshi Tagomori)]

make yjit-bench is now failing even on master HEAD.

bin/rails aborted!
LoadError: cannot load such file -- cgi/cookie (LoadError)

I saw commits to re-organize cgi.
Hmm, I can wait for the fix and will make benchmark outputs if others don't make changes which conflict with the namespace change.
Otherwise, I hope merging the change immediately and will check the performance degradation later.

[byroot (Jean Boussier)]

bin/rails aborted!
LoadError: cannot load such file -- cgi/cookie (LoadError)

Yes, the cgi changes broke Rails. You can add gem "cgi" to the benchmark's Gemfile to workaround that.

Both Rails nightly CI and Shopify nightly CI are still broken from the recent Set change and got broken more from the CGI change. It has been weeks we haven't seen ruby-head pass, and are still trying to play catch up.

[byroot (Jean Boussier)]

I opened a PR on yjit-bench with workarounds for both the fiddle issue and the CGI issue: https://github.com/Shopify/yjit-bench/pull/363

I'll see if I can benchmark your branch.

[byroot (Jean Boussier)]

Trying to run benchmarks with RUBY_NAMESPACE=1 crashes during init:

opt/rubies/head-namespaces/bin/ruby: [BUG] Segmentation fault at 0x0000000000000008
ruby 3.5.0dev (2025-05-11T00:30:16Z namespace-on-read-.. 6629795fff) +PRISM [arm64-darwin24]

-- Crash Report log information --------------------------------------------
   See Crash Report log file in one of the following locations:             
     * ~/Library/Logs/DiagnosticReports                                     
     * /Library/Logs/DiagnosticReports                                      
   for more details.                                                        
Don't forget to include the above Crash Report log file in bug reports.     

-- Control frame information -----------------------------------------------
c:0001 p:0000 s:0003 E:000380 DUMMY  [FINISH]


-- Threading information ---------------------------------------------------
Total ractor count: 1
Ruby thread count for this ractor: 1

-- Machine register context ------------------------------------------------
  x0: 0x0000000000000000  x1: 0x0000000000000f9f  x2: 0x000000016b156668
  x3: 0x0000000000000000  x4: 0x0000000000000005  x5: 0x00000000de600168
  x6: 0x0000000000002ce0  x7: 0x0000000000000001 x18: 0x0000000000000000
 x19: 0x0000000120026a40 x20: 0x0000000000000000 x21: 0x0000000000000f9f
 x22: 0x00000001051be000 x23: 0x00000001051c3668 x24: 0x0000000142e05900
 x25: 0x0000000000000008 x26: 0x00000001051c3638 x27: 0x000000016b1574c0
 x28: 0x0000000000000009  lr: 0x0000000104d7d420  fp: 0x000000016b156650
  sp: 0x000000016b1565f0

-- C level backtrace information -------------------------------------------
/opt/rubies/head-namespaces/bin/ruby(rb_vm_bugreport+0xb6c) [0x104f315f8]
/opt/rubies/head-namespaces/bin/ruby(rb_bug_for_fatal_signal+0x100) [0x104d5efd4]
/opt/rubies/head-namespaces/bin/ruby(sigsegv+0x84) [0x104e897e8]
/usr/lib/system/libsystem_platform.dylib(_sigtramp+0x38) [0x18de56de4]
/opt/rubies/head-namespaces/bin/ruby(object_id+0x80) [0x104d7d420]
/opt/rubies/head-namespaces/bin/ruby(object_id+0x80) [0x104d7d420]
/opt/rubies/head-namespaces/bin/ruby(rb_initialize_main_namespace+0xe4) [0x104ddaa20]
/opt/rubies/head-namespaces/bin/ruby(ruby_opt_init+0x120) [0x104e7f524]
/opt/rubies/head-namespaces/bin/ruby(ruby_process_options+0x1370) [0x104e7e31c]
/opt/rubies/head-namespaces/bin/ruby(ruby_options+0xb0) [0x104d69844]
/opt/rubies/head-namespaces/bin/ruby(main+0x64) [0x104ca8d54]

[byroot (Jean Boussier)]

headline benchmarks (in non-ideal conditions, I was on battery, etc) and whithout RUBY_NAMESPACE=1:

master: ruby 3.5.0dev (2025-05-11T03:09:26Z master 49742414f6) +PRISM [arm64-darwin24]
namespace: ruby 3.5.0dev (2025-05-11T00:30:16Z namespace-on-read-.. 6629795fff) +PRISM [arm64-darwin24]

--------------  -----------  ----------  --------------  ----------  -----------------  ----------------
bench           master (ms)  stddev (%)  namespace (ms)  stddev (%)  namespace 1st itr  master/namespace
activerecord    148.5        1.2         152.0           3.6         0.986              0.977           
chunky-png      427.8        1.1         424.6           0.3         1.027              1.007           
erubi-rails     625.6        1.7         659.8           1.4         0.997              0.948           
hexapdf         1112.4       1.3         1126.5          0.9         0.943              0.988           
liquid-c        25.9         1.8         27.0            2.0         0.767              0.960           
liquid-compile  25.8         2.4         25.9            2.3         1.000              0.997           
liquid-render   67.8         1.4         71.9            1.5         0.961              0.944           
lobsters        444.4        1.3         446.7           1.4         1.004              0.995           
mail            75.6         13.2        70.7            5.7         0.947              1.068           
psych-load      1095.8       0.5         1093.4          0.2         1.028              1.002           
railsbench      984.1        1.9         1029.9          2.7         0.970              0.956           
rubocop         80.0         2.1         81.8            2.5         0.932              0.977           
ruby-lsp        85.9         0.8         88.0            1.0         0.960              0.976           
sequel          27.6         2.7         27.7            1.1         0.992              0.997           
--------------  -----------  ----------  --------------  ----------  -----------------  ----------------
Legend:
- namespace 1st itr: ratio of master/namespace time for the first benchmarking iteration.
- master/namespace: ratio of master/namespace time. Higher is better for namespace. Above 1 represents a speedup.

The overhead on the interpreter seem to be about 3-5%. I'll try to fix the branch to get RUBY_NAMESPACE=1 not to crash and run the benchmarks again in a better environment.

[byroot (Jean Boussier)]

I submitted a fix for the crash here: https://github.com/tagomoris/ruby/pull/5

But now it's crashing in require

/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:61: [BUG] Segmentation fault at 0x0000000000000339
ruby 3.5.0dev (2025-05-11T00:30:16Z namespace-on-read-.. 6629795fff) +PRISM [arm64-darwin24]

-- Crash Report log information --------------------------------------------
   See Crash Report log file in one of the following locations:             
     * ~/Library/Logs/DiagnosticReports                                     
     * /Library/Logs/DiagnosticReports                                      
   for more details.                                                        
Don't forget to include the above Crash Report log file in bug reports.     

-- Control frame information -----------------------------------------------
c:0030 p:0017 s:0140 e:000136 METHOD /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:61
c:0029 p:0005 s:0132 e:000131 BLOCK  /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:51 [FINISH]
c:0028 p:---- s:0128 e:000127 CFUNC  :map
c:0027 p:0025 s:0124 e:000122 METHOD /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:51
c:0026 p:0007 s:0118 e:000117 METHOD /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:209
c:0025 p:0492 s:0113 e:000112 BLOCK  /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflections.rb:70
c:0024 p:0017 s:0109 e:000108 METHOD /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:267
c:0023 p:0007 s:0104 e:000103 CLASS  /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflections.rb:12
c:0022 p:0013 s:0101 e:000100 TOP    /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflections.rb:11 [FINISH]
c:0021 p:---- s:0098 e:000097 CFUNC  :require
c:0020 p:0023 s:0093 e:000092 METHOD <internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136 [FINISH]
c:0019 p:---- s:0087 e:000086 CFUNC  :require
c:0018 p:0005 s:0082 e:000081 TOP    /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/methods.rb:3 [FINISH]
c:0017 p:---- s:0079 e:000078 CFUNC  :require
c:0016 p:0023 s:0074 e:000073 METHOD <internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136 [FINISH]
c:0015 p:---- s:0068 e:000067 CFUNC  :require
c:0014 p:0005 s:0063 e:000062 TOP    /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/dependencies/autoload.rb:3 [FINISH]
c:0013 p:---- s:0060 e:000059 CFUNC  :require
c:0012 p:0023 s:0055 e:000054 METHOD <internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136 [FINISH]
c:0011 p:---- s:0049 e:000048 CFUNC  :require
c:0010 p:0011 s:0044 e:000043 TOP    /Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support.rb:27 [FINISH]
c:0009 p:---- s:0041 e:000040 CFUNC  :require
c:0008 p:0023 s:0036 e:000035 METHOD <internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136 [FINISH]
c:0007 p:---- s:0030 e:000029 CFUNC  :require
c:0006 p:0005 s:0025 e:000024 TOP    /Users/byroot/.gem/rubies/head-namespaces/gems/activerecord-7.2.2.1/lib/active_record.rb:26 [FINISH]
c:0005 p:---- s:0022 e:000021 CFUNC  :require
c:0004 p:0023 s:0017 e:000016 METHOD <internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136 [FINISH]
c:0003 p:---- s:0011 e:000010 CFUNC  :require
c:0002 p:0023 s:0006 e:000005 EVAL   benchmark.rb:6 [FINISH]
c:0001 p:0000 s:0003 E:000d50 DUMMY  [FINISH]

-- Ruby level backtrace information ----------------------------------------
benchmark.rb:6:in '<main>'
benchmark.rb:6:in 'require'
<internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136:in 'require'
<internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136:in 'require'
/Users/byroot/.gem/rubies/head-namespaces/gems/activerecord-7.2.2.1/lib/active_record.rb:26:in '<top (required)>'
/Users/byroot/.gem/rubies/head-namespaces/gems/activerecord-7.2.2.1/lib/active_record.rb:26:in 'require'
<internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136:in 'require'
<internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136:in 'require'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support.rb:27:in '<top (required)>'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support.rb:27:in 'require'
<internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136:in 'require'
<internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136:in 'require'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/dependencies/autoload.rb:3:in '<top (required)>'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/dependencies/autoload.rb:3:in 'require'
<internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136:in 'require'
<internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136:in 'require'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/methods.rb:3:in '<top (required)>'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/methods.rb:3:in 'require'
<internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136:in 'require'
<internal:/opt/rubies/head-namespaces/lib/ruby/3.5.0+0/rubygems/core_ext/kernel_require.rb>:136:in 'require'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflections.rb:11:in '<top (required)>'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflections.rb:12:in '<module:ActiveSupport>'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:267:in 'inflections'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflections.rb:70:in 'block in <module:ActiveSupport>'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:209:in 'uncountable'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:51:in 'add'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:51:in 'map'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:51:in 'block in add'
/Users/byroot/.gem/rubies/head-namespaces/gems/activesupport-7.2.2.1/lib/active_support/inflector/inflections.rb:61:in 'to_regex'

-- Threading information ---------------------------------------------------
Total ractor count: 1
Ruby thread count for this ractor: 1

-- Machine register context ------------------------------------------------
  x0: 0x0000000000000331  x1: 0x0000000000000000  x2: 0x0000000000000f9f
  x3: 0x000000016b4544b8  x4: 0x000000016b454664  x5: 0x0000000000000000
  x6: 0x0000000000000001  x7: 0x0000000000000028 x18: 0x0000000000000000
 x19: 0x0000000000000331 x20: 0x000000015600fc00 x21: 0x000000011e59eec0
 x22: 0x0000000000000000 x23: 0x0000000000000f9f x24: 0x000060000098f660
 x25: 0x0000000000000001 x26: 0x000000016b454448 x27: 0x0000000000000001
 x28: 0x00000001049d16dc  lr: 0x00000001049d16fc  fp: 0x000000016b454390
  sp: 0x000000016b454340

-- C level backtrace information -------------------------------------------
/opt/rubies/head-namespaces/bin/ruby(rb_vm_bugreport+0xb6c) [0x104c2d5f8]
/opt/rubies/head-namespaces/bin/ruby(rb_bug_for_fatal_signal+0x100) [0x104a5af40]
/opt/rubies/head-namespaces/bin/ruby(sigsegv+0x84) [0x104b857e8]
/usr/lib/system/libsystem_platform.dylib(_sigtramp+0x38) [0x18de56de4]
/opt/rubies/head-namespaces/bin/ruby(class_classext_foreach_i+0x20) [0x1049d16fc]
/opt/rubies/head-namespaces/bin/ruby(class_classext_foreach_i+0x20) [0x1049d16fc]
/opt/rubies/head-namespaces/bin/ruby(rb_st_foreach+0xa4) [0x104b8d0c0]
/opt/rubies/head-namespaces/bin/ruby(rb_class_classext_foreach+0x4c) [0x1049d1698]
/opt/rubies/head-namespaces/bin/ruby(rb_gc_mark_children+0x1bc) [0x104a7ad90]
/opt/rubies/head-namespaces/bin/ruby(gc_mark_stacked_objects_incremental+0xac) [0x104a84710]
/opt/rubies/head-namespaces/bin/ruby(gc_continue+0xd0) [0x104a82b88]
/opt/rubies/head-namespaces/bin/ruby(newobj_cache_miss+0x180) [0x104a827b0]
/opt/rubies/head-namespaces/bin/ruby(newobj_of+0x170) [0x104a77548]
/opt/rubies/head-namespaces/bin/ruby(str_duplicate+0xb4) [0x104b9d924]
/opt/rubies/head-namespaces/bin/ruby(rb_reg_initialize_str+0xc4) [0x104b4d994]
/opt/rubies/head-namespaces/bin/ruby(rb_reg_new_ary+0x1e8) [0x104b4dc1c]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x5df4) [0x104c05b90]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(invoke_block_from_c_bh+0x368) [0x104c27274]
/opt/rubies/head-namespaces/bin/ruby(rb_yield+0xa8) [0x104c0e2fc]
/opt/rubies/head-namespaces/bin/ruby(rb_ary_collect+0xd4) [0x1049adeb4]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x1d60) [0x104c01afc]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(rb_ensure+0xb8) [0x104a66ee4]
/opt/rubies/head-namespaces/bin/ruby(load_iseq_eval+0x280) [0x104acb488]
/opt/rubies/head-namespaces/bin/ruby(require_internal+0x99c) [0x104ac90ac]
/opt/rubies/head-namespaces/bin/ruby(rb_require_string_internal+0x58) [0x104ac8320]
/opt/rubies/head-namespaces/bin/ruby(rb_f_require+0x44) [0x104ac81f0]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_call_alias+0x70) [0x104c19b38]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x2228) [0x104c01fc4]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(vm_call0_body+0x470) [0x104c2482c]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_call_kw+0xd8) [0x104c0c660]
/opt/rubies/head-namespaces/bin/ruby(rb_call_super_kw+0x1bc) [0x104c0cbf0]
/opt/rubies/head-namespaces/bin/ruby(rb_ensure+0xb8) [0x104a66ee4]
/opt/rubies/head-namespaces/bin/ruby(rb_namespace_user_loading_func+0x80) [0x104ad7b0c]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x2228) [0x104c01fc4]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(rb_ensure+0xb8) [0x104a66ee4]
/opt/rubies/head-namespaces/bin/ruby(load_iseq_eval+0x280) [0x104acb488]
/opt/rubies/head-namespaces/bin/ruby(require_internal+0x99c) [0x104ac90ac]
/opt/rubies/head-namespaces/bin/ruby(rb_require_string_internal+0x58) [0x104ac8320]
/opt/rubies/head-namespaces/bin/ruby(rb_f_require+0x44) [0x104ac81f0]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_call_alias+0x70) [0x104c19b38]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x2228) [0x104c01fc4]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(vm_call0_body+0x470) [0x104c2482c]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_call_kw+0xd8) [0x104c0c660]
/opt/rubies/head-namespaces/bin/ruby(rb_call_super_kw+0x1bc) [0x104c0cbf0]
/opt/rubies/head-namespaces/bin/ruby(rb_ensure+0xb8) [0x104a66ee4]
/opt/rubies/head-namespaces/bin/ruby(rb_namespace_user_loading_func+0x80) [0x104ad7b0c]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x2228) [0x104c01fc4]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(rb_ensure+0xb8) [0x104a66ee4]
/opt/rubies/head-namespaces/bin/ruby(load_iseq_eval+0x280) [0x104acb488]
/opt/rubies/head-namespaces/bin/ruby(require_internal+0x99c) [0x104ac90ac]
/opt/rubies/head-namespaces/bin/ruby(rb_require_string_internal+0x58) [0x104ac8320]
/opt/rubies/head-namespaces/bin/ruby(rb_f_require+0x44) [0x104ac81f0]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_call_alias+0x70) [0x104c19b38]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x2228) [0x104c01fc4]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(vm_call0_body+0x470) [0x104c2482c]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_call_kw+0xd8) [0x104c0c660]
/opt/rubies/head-namespaces/bin/ruby(rb_call_super_kw+0x1bc) [0x104c0cbf0]
/opt/rubies/head-namespaces/bin/ruby(rb_ensure+0xb8) [0x104a66ee4]
/opt/rubies/head-namespaces/bin/ruby(rb_namespace_user_loading_func+0x80) [0x104ad7b0c]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x2228) [0x104c01fc4]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(rb_ensure+0xb8) [0x104a66ee4]
/opt/rubies/head-namespaces/bin/ruby(load_iseq_eval+0x280) [0x104acb488]
/opt/rubies/head-namespaces/bin/ruby(require_internal+0x99c) [0x104ac90ac]
/opt/rubies/head-namespaces/bin/ruby(rb_require_string_internal+0x58) [0x104ac8320]
/opt/rubies/head-namespaces/bin/ruby(rb_f_require+0x44) [0x104ac81f0]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_call_alias+0x70) [0x104c19b38]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x2228) [0x104c01fc4]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(vm_call0_body+0x470) [0x104c2482c]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_call_kw+0xd8) [0x104c0c660]
/opt/rubies/head-namespaces/bin/ruby(rb_call_super_kw+0x1bc) [0x104c0cbf0]
/opt/rubies/head-namespaces/bin/ruby(rb_ensure+0xb8) [0x104a66ee4]
/opt/rubies/head-namespaces/bin/ruby(rb_namespace_user_loading_func+0x80) [0x104ad7b0c]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x2228) [0x104c01fc4]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(rb_ensure+0xb8) [0x104a66ee4]
/opt/rubies/head-namespaces/bin/ruby(load_iseq_eval+0x280) [0x104acb488]
/opt/rubies/head-namespaces/bin/ruby(require_internal+0x99c) [0x104ac90ac]
/opt/rubies/head-namespaces/bin/ruby(rb_require_string_internal+0x58) [0x104ac8320]
/opt/rubies/head-namespaces/bin/ruby(rb_f_require+0x44) [0x104ac81f0]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_call_alias+0x70) [0x104c19b38]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x2228) [0x104c01fc4]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(vm_call0_body+0x470) [0x104c2482c]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_call_kw+0xd8) [0x104c0c660]
/opt/rubies/head-namespaces/bin/ruby(rb_call_super_kw+0x1bc) [0x104c0cbf0]
/opt/rubies/head-namespaces/bin/ruby(rb_ensure+0xb8) [0x104a66ee4]
/opt/rubies/head-namespaces/bin/ruby(rb_namespace_user_loading_func+0x80) [0x104ad7b0c]
/opt/rubies/head-namespaces/bin/ruby(vm_call_cfunc_with_frame_+0xf0) [0x104c1e8cc]
/opt/rubies/head-namespaces/bin/ruby(vm_exec_core+0x2228) [0x104c01fc4]
/opt/rubies/head-namespaces/bin/ruby(rb_vm_exec+0x184) [0x104bfe838]
/opt/rubies/head-namespaces/bin/ruby(rb_ec_exec_node+0x98) [0x104a6607c]
/opt/rubies/head-namespaces/bin/ruby(ruby_run_node+0x44) [0x104a65f9c]
/opt/rubies/head-namespaces/bin/ruby(main+0x68) [0x1049a4cc4]

[byroot (Jean Boussier)]

Alright, so I can't benchmark with RUBY_NAMESPACE=1 because I keep running into that crash and can't figure it out.
In the process of trying to find a fix I found multiple other bugs:

• https://github.com/tagomoris/ruby/pull/7
• https://github.com/tagomoris/ruby/pull/6
• https://github.com/tagomoris/ruby/pull/5

I still ran the benchmark again on a better machine (still not tuned for benchmarking though), and the results appear consistent.

Interpreter:

master: ruby 3.5.0dev (2025-05-11T03:09:26Z master 49742414f6) +PRISM [arm64-darwin24]
last_commit=Revert "Fix redefinition of `clock_gettime` and `clock_getres`"
namespace-off: ruby 3.5.0dev (2025-05-11T08:53:41Z byroot-namespace 20e6177509) +PRISM [arm64-darwin24]

--------------  -----------  ----------  ------------------  ----------  ---------------------  --------------------
bench           master (ms)  stddev (%)  namespace-off (ms)  stddev (%)  namespace-off 1st itr  master/namespace-off
activerecord    153.4        1.7         154.5               1.1         0.996                  0.993               
chunky-png      423.8        1.0         421.2               1.3         1.007                  1.006               
erubi-rails     664.6        1.1         699.1               1.2         0.887                  0.951               
hexapdf         1132.1       3.7         1150.5              1.1         0.972                  0.984               
liquid-c        26.5         4.2         27.7                3.7         0.745                  0.957               
liquid-compile  27.0         3.0         27.0                2.5         0.998                  0.999               
liquid-render   70.5         2.7         72.9                1.7         0.914                  0.967               
lobsters        476.1        0.8         483.7               1.2         1.004                  0.984               
mail            78.0         14.0        69.9                3.4         0.904                  1.115               
psych-load      1103.7       0.6         1107.9              0.6         0.977                  0.996               
railsbench      1058.7       0.8         1073.4              1.3         0.978                  0.986               
rubocop         83.7         1.8         86.7                3.0         0.986                  0.965               
ruby-lsp        89.8         2.2         89.6                1.2         1.005                  1.002               
sequel          28.2         2.5         28.7                2.7         0.981                  0.980               
--------------  -----------  ----------  ------------------  ----------  ---------------------  --------------------
Legend:
- namespace-off 1st itr: ratio of master/namespace-off time for the first benchmarking iteration.
- master/namespace-off: ratio of master/namespace-off time. Higher is better for namespace-off. Above 1 represents a speedup.

YJIT:

master: ruby 3.5.0dev (2025-05-11T03:09:26Z master 49742414f6) +YJIT +PRISM [arm64-darwin24]
last_commit=Revert "Fix redefinition of `clock_gettime` and `clock_getres`"
namespace-off: ruby 3.5.0dev (2025-05-11T08:53:41Z byroot-namespace 20e6177509) +YJIT +PRISM [arm64-darwin24]

--------------  -----------  ----------  ------------------  ----------  ---------------------  --------------------
bench           master (ms)  stddev (%)  namespace-off (ms)  stddev (%)  namespace-off 1st itr  master/namespace-off
activerecord    65.3         1.7         69.2                1.9         0.914                  0.942               
chunky-png      236.2        2.3         230.5               1.4         0.998                  1.025               
erubi-rails     321.8        1.5         357.1               2.2         0.830                  0.901               
hexapdf         593.2        3.7         600.0               3.2         0.980                  0.989               
liquid-c        19.5         3.2         20.4                3.2         0.965                  0.953               
liquid-compile  19.0         3.5         19.2                3.4         0.936                  0.989               
liquid-render   28.6         2.7         28.8                4.1         1.014                  0.991               
lobsters        317.0        2.4         326.0               2.3         0.926                  0.972               
mail            53.2         17.3        53.3                15.7        1.005                  0.998               
psych-load      651.3        0.9         666.8               0.5         0.978                  0.977               
railsbench      536.3        1.5         550.5               2.0         1.005                  0.974               
rubocop         52.9         6.6         53.2                5.6         0.995                  0.994               
ruby-lsp        47.7         1.8         49.6                4.0         0.990                  0.962               
sequel          23.1         2.7         23.7                3.1         0.914                  0.974               
--------------  -----------  ----------  ------------------  ----------  ---------------------  --------------------
Legend:
- namespace-off 1st itr: ratio of master/namespace-off time for the first benchmarking iteration.
- master/namespace-off: ratio of master/namespace-off time. Higher is better for namespace-off. Above 1 represents a speedup.

[tagomoris (Satoshi Tagomori)]

@byroot (Jean Boussier) Thank you for working on the feature. I'll merge the patches soon (had merged one already).
About the crash with RUBY_NAMESPACE=1, I'll work on it ASAP after merging the branch once.

I know that the 3-5% performance degradation is not small, especially for the very large workload (like Shopify). But, for now, I want to merge the branch once and then try to find performance problems I added (except for the case Matz say "stop!"). I can't maintain my branch only by myself (with so many continuous rebases) for a long time.

[fxn (Xavier Noria)]

Some of the issues I have noted seem to be addressable and have been added to the TODO. (I have written tickets for them.)

But there is one that seems fundamental and worries me. Since this ticket is alive, I'd like to raise awareness of it.

Let's do this thought experiment, in which we imagive c.rb defines a class C with a class method x:

ns = Namespace.new
ns.require('c')
ns::C.x(Object)

Now, there are two conflicting options here:

1. If the method receives the passed object reference as usual (remember, Object is a constant that evaluates to an object), then within x the value returned by Object in the namespace and the method argument have the same object ID, but different state (constants may give different lists, for example).

2. Alternatively, the method invocation internally swaps the passed object reference with the one with the same ID in the namespace (so to speak), and x does not receive the object passed by the caller. You have a method call that does not pass what you pass.

Both (1) and (2) break the current semantics of Ruby. ((2) is what happens, though I do not know if that is deliberate or a side-effect of the implementation.)

It is the combination of isolation + constant access outside the namespace + method call semantics that I find difficult to reconcile (does not mean it is impossible with newly introduced rules).

[tagomoris (Satoshi Tagomori)]

@fxn I'm unsure I correctly understand what you mean... Let me check your point.

You said:

• The current namespace implementation behaves like (2), not (1).
• The passed value to ns::C.x (in the callee namespace side) will be a different (copied) object from Object in the caller namespace.

Is that correctly what you mean?

My answer: Namespace behaves like (1), not (2).

https://github.com/tagomoris/ruby/blob/namespace-on-read-classext/doc/namespace.md#builtin-classes-referred-via-namespace-objects
As I noted on the section above, Object is the Object class object in any namespaces. When it is patched (added a constant, added a method, etc) in a namespace, the backed rb_classext_t (it's not a object itself, it's a set of "state" in your vocabulary) will be copied. But the Object-class object itself is not copied, just be passed.

(1) breaks the current semantics of Ruby. Yes, namespace is a new idea to change the semantics.

[fxn (Xavier Noria)]

@tagomoris (Satoshi Tagomori) From the point of view of the Ruby programmer, I have an object, I pass it, and the callee receives a different object. To me, that is (2). Somehow, the object itself has mutated.

We are not talking about constant lookup in the namespace. Or about the value the Object constant evaluates to in a namespace.

We are talking about object references crossing namespaces via method calls.

I send you an object, and you get a different object.

As I said in the ticket, that object could be in some attribute three levels down in the object tree of your user-level class instance.

In the namespace examples, sure

ns::String.foo # NoMethodError

but there is a constant lookup there that we all understand retrieves a different object (in an observable meaning) from within the namespace.

[fxn (Xavier Noria)]

@tagomoris (Satoshi Tagomori) when I said that (1) would break Ruby semantics I meant that in the same namespace you would have two objects with the same ID and different state. That does not happen as far as I can tell.

[tagomoris (Satoshi Tagomori)]

fxn (Xavier Noria) wrote in #note-68:

@tagomoris (Satoshi Tagomori) when I said that (1) would break Ruby semantics I meant that in the same namespace you would have two objects with the same ID and different state. That does not happen as far as I can tell.

That should not happen, and if it happens, it's a bug. Could you write and show the script to reproduce it?

[fxn (Xavier Noria)]

That should not happen, and if it happens, it's a bug. Could you write and show the script to reproduce it?

Exactly, as far as I can tell it does not happen, which to me it means we do not have (1) in place.

It was a way to say that, from my perspective, we have (2) in place.

Maybe you have the C code in your head, I am looking at it as a Ruby programmer would. I have one object reference, and I pass it in method call, no namespace-level constant resolution involved in the callee.

See what I mean?

[tagomoris (Satoshi Tagomori)]

fxn (Xavier Noria) wrote in #note-70:

Maybe you have the C code in your head, I am looking at it as a Ruby programmer would. I have one object reference, and I pass it in method call, no namespace-level constant resolution involved in the callee.

See what I mean?

No, I can't.

What I understand as a Ruby programmer (me), Class object is just an object like other objects, referred either constant resolution or passing value (as a method argument). We can check the object equality by object id.
And, - this part is the new paradigm Namespace introduced -, an object, crossing namespaces from A to B, is just an object (because object id is not changed before/after), but may have different sets of definitions (constants, ivars, methods, etc) in A and B.

Actually, it's not so big change because we have refinements. When using X happens, objects can have different sets of methods. Namespace expands it to constants, class ivars, include/extend and others.

[fxn (Xavier Noria)]

Example:

# main.rb

class C
  attr_reader :target

  def initialize(target)
    @target = target
  end
end

ns = Namespace.new
ns.require_relative 'foo'

c = C.new(Object)
ns::Foo.new.m(c)

# foo.rb

class Foo
  def m(c)
    (c.target)::C # NameError
  end
end

That raises.

The object tree of my c instance has somehow been altered in the method call.

I believe I understand what you have in mind, eh? Only I believe the docs should explain this so that users can understand these consequences, and wanted to make sure everyone is aware of this.

We can continue in the ticket if you will :).

[fxn (Xavier Noria)]

wanted to make sure everyone is aware of this

What I want all in the thread to realize is that if somewhere in the tree of your object, some code relies on a core extension, that object is automatically not cross-namespace-safe, and you may not even know it.

Together with the design allowing to mix instances of different versions of the same gem, makes this particular aspect of the feature questionable, for me.

If this behavior stays, I'll volunteer a doc patch.

[tagomoris (Satoshi Tagomori)]

• Status changed from Open to Closed
• Assignee set to tagomoris (Satoshi Tagomori)
• Target version set to 3.5

I've merged the change into master. So, let me close this issue.
Please open another ticker for further problems, issues, bugs, etc.

[byroot (Jean Boussier)]

Please open another ticker for further problems, issues, bugs, etc.

Right now RUBY_NAMESPACE=1 make -j btest fails, are you actively working on adding a CI step that run with RUBY_NAMESPACE=1 or should I open another ticket to track this? I would have assume it would have been part of this ticket.

[tagomoris (Satoshi Tagomori)]

• Status changed from Closed to Open

byroot (Jean Boussier) wrote in #note-75:

Please open another ticker for further problems, issues, bugs, etc.

Right now RUBY_NAMESPACE=1 make -j btest fails, are you actively working on adding a CI step that run with RUBY_NAMESPACE=1 or should I open another ticket to track this? I would have assume it would have been part of this ticket.

I had a plan to add CI configuration to add RUBY_NAMESPACE=1 checks and I thought it would not be a part of this. But yes, it should be. I'll close this ticket after adding checks.

[peterzhu2118 (Peter Zhu)]

As another datapoint on Linux (Ubuntu 24.04, AMD Ryzen 9800X3D), it looks like it is slower (around 0-5%) but also has higher memory usage by about 5-10%.

pre-namespace: ruby 3.5.0dev (2025-05-11T03:09:26Z master 49742414f6) +PRISM [x86_64-linux]
namespace: ruby 3.5.0dev (2025-05-12T16:59:58Z master c6528548d0) +PRISM [x86_64-linux]
last_commit=[ruby/erb] Use cgi/escape instead of deprecated cgi/util
--------------  ------------------  ----------  ---------  --------------  ----------  ---------  -----------------  -----------------------
bench           pre-namespace (ms)  stddev (%)  RSS (MiB)  namespace (ms)  stddev (%)  RSS (MiB)  namespace 1st itr  pre-namespace/namespace
activerecord    165.9               0.2         54.8       171.4           0.1         64.1       0.958              0.968                  
chunky-png      434.4               0.1         55.2       429.1           0.1         54.4       1.012              1.012                  
erubi-rails     695.5               0.2         110.6      735.4           1.4         154.5      0.991              0.946                  
hexapdf         1345.6              0.7         225.3      1366.4          0.6         252.7      0.972              0.985                  
liquid-c        32.6                0.8         40.4       33.0            1.3         32.9       0.991              0.989                  
liquid-compile  30.4                1.5         31.4       30.1            1.4         32.2       1.012              1.008                  
liquid-render   86.4                0.4         31.1       88.7            0.3         34.6       0.969              0.974                  
lobsters        592.0               0.1         276.9      596.0           0.5         298.9      1.023              0.993                  
mail            80.6                4.8         49.4       79.2            4.3         49.3       1.021              1.018                  
psych-load      1145.3              0.2         73.2       1151.4          0.3         72.8       1.007              0.995                  
railsbench      1453.8              0.5         105.3      1459.9          0.6         113.1      0.975              0.996                  
rubocop         91.7                0.3         82.6       93.1            0.3         85.0       1.005              0.985                  
ruby-lsp        89.6                0.5         64.8       90.0            0.4         67.1       1.009              0.996                  
sequel          27.8                0.4         30.6       27.8            0.7         31.3       0.981              1.000                  
--------------  ------------------  ----------  ---------  --------------  ----------  ---------  -----------------  -----------------------

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned

[jhawthorn (John Hawthorn)]

Sorry, I didn't get a chance to review this before it was merged. I really don't think adding this level of indirection to RCLASS_EXT access and similar is a good idea.

Let's look at rb_class_get_superclass as an example. This should be approximately the same code inserted every time RCLASS_SUPER is used. Here's how it looks before namespaces were merged:

❯ llvm-objdump -d --symbolize-operands --x86-asm-syntax=intel --no-show-raw-insn --no-leading-addr --disassemble-symbols=rb_class_get_superclass -S ./miniruby

./miniruby:     file format elf64-x86-64

Disassembly of section .text:

<rb_class_get_superclass>:
;     return RCLASS(klass)->super;
                mov     rax, qword ptr [rdi + 0x10]
; }
                ret

This is how something this common in the VM should ideally look. Just one instruction (a couple would be fine) to read the data we need and returning it. No branching, no locking, easily inlineable or taught to the JIT compiler.

Here's how rb_class_get_superclass looks now:

❯ llvm-objdump -d --symbolize-operands --x86-asm-syntax=intel --no-show-raw-insn --no-leading-addr --disassemble-symbols=rb_class_get_superclass -S ./miniruby

./miniruby:     file format elf64-x86-64

Disassembly of section .text:

<rb_class_get_superclass>:
; {
                push    rbx
                sub     rsp, 0x10
                mov     rbx, qword ptr fs:[0x28]
                mov     qword ptr [rsp + 0x8], rbx
                mov     rbx, rdi
;     if (RCLASS_PRIME_CLASSEXT_READABLE_P(obj)) {
                cmp     qword ptr [rdi + 0x10], 0x0
                je       <L0>
;     ns = rb_current_namespace();
                call     <rb_current_namespace>
;     if (!ns || NAMESPACE_BUILTIN_P(ns)) {
                test    rax, rax
                je       <L0>
                cmp     byte ptr [rax + 0x78], 0x0
                je       <L1>
<L0>:
;     return RCLASS_EXT_PRIME(obj);
                lea     rax, [rbx + 0x18]
<L3>:
;     return RCLASS_SUPER(klass);
                mov     rax, qword ptr [rax + 0x8]
; }
                mov     rdx, qword ptr [rsp + 0x8]
                sub     rdx, qword ptr fs:[0x28]
                jne      <L2>
                add     rsp, 0x10
                pop     rbx
                ret
                nop     dword ptr [rax + rax]
<L1>:
;     st_table *classext_tbl = RCLASS_CLASSEXT_TBL(obj);
                mov     rdi, qword ptr [rbx + 0x10]
;     if (classext_tbl) {
                test    rdi, rdi
                je       <L0>
;         if (rb_st_lookup(classext_tbl, (st_data_t)ns->ns_object, &classext_ptr)) {
                mov     rsi, qword ptr [rax]
                mov     rdx, rsp
                call     <rb_st_lookup>
                test    eax, eax
                je       <L0>
;             return (rb_classext_t *)classext_ptr;
                mov     rax, qword ptr [rsp]
;     if (ext)
                test    rax, rax
                jne      <L3>
                jmp      <L0>
<L2>:
; }
                call     <__stack_chk_fail@plt>

It's tempting to assume that because we're not taking the branches with the more expensive operation there isn't a lot of cost, however because we might be calling all of these other methods, the C compiler has to be pessimistic and generate code assuming the method calls were taken each time. So not only is our one instruction now 14 instructions with two conditional branches, any of the hundreds of locations this is inlined, the surrounding code the C compiler generates will be worse.

I don't share the optimism that this can be made efficient with more work. In fact I expect this to become even slower as bugs are fixed. For example even this huge disassembly is not correct because it does not hold the VM lock where it should and will almost certainly segfault under Ractors.

I really think we need an implementation where T_CLASS again refers to a unique implementation (rather than needing to look up the implementation for the current namespace), and RCLASS_EXT is once against just pointer arithmetic. Otherwise I don't see possible how with this implementation it's possible to have an efficient JIT compiler (unless it disables itself when there are namespaces), an efficient implementation of Ractors (we can't write any lock-free access to class-related contents, because classes cannot be accessed without a lock and hash lookup), or even a fast interpreter (the 5% slowdown we already see here).

[mame (Yusuke Endoh)]

jhawthorn (John Hawthorn) wrote in #note-79:

Sorry, I didn't get a chance to review this before it was merged. I really don't think adding this level of indirection to RCLASS_EXT access and similar is a good idea.

Since I was the one who originally suggested the approach of adding indirection to classext, I'd like to respond briefly.

First, the ability to override constant and method definitions per namespace was a specification given by Matz. To meet this requirement, @tagomoris (Satoshi Tagomori) initially pursued an implementation based on refinements.

As you know, refinements are not widely used in practice. I think there are several reasons for this, but a major one is that they are treated as second-class citizens in terms of implementation and optimization. I wanted to make sure namespaces wouldn't fall into the same trap.

So I proposed a more straightforward approach: giving each namespace its own classext. This idea was well received by @ko1 (Koichi Sasada). He believed that, in most cases, caching would allow us to avoid repeated access to RCLASSEXT, and that while some some overhead remain, we could aoid significant performance degredation. (In fact, the overhead appears to be only about 5%.)

This is how we arrived at the current implementation direction.

That said, this is ultimately an implementation detail. If there is another design that can meet Matz's specification without relegating namespaces to second-class status (i.e., being fast only when unused), it would absolutely be acceptable.

[byroot (Jean Boussier)]

(In fact, the overhead appears to be only about 5%.)

That's when namespaces aren't used. Since creating a namespace start some sort of de-optimization phase, I assume the impact is way worse with RUBY_NAMESPACE=1, but I couldn't measure that because there are still too many bugs.

[Eregon (Benoit Daloze)]

mame (Yusuke Endoh) wrote in #note-51:

This feature proposal is quite exceptional. Because this is a feature strongly driven by @matz (Yukihiro Matsumoto) himself, and Ruby is matz's language.

@tagomoris (Satoshi Tagomori) has been working closely with Matz for several years to design and implement it. Although there are still many incomplete or rough parts, the decision has already been made.

I took some time before replying here to try to find the right words.
Actually I think it is obvious what the 3 major problems are with this, but I will spell them out.

It is great to hear matz was closely involved in the design of this feature.

1. However the design of such a big feature needs to be presented in advance, so it's not just the thoughts of 1/2/a few people but the Ruby community can participate, give feedback and help refine the feature design to make it better.
   Features are always better when more people are involved, a recent example is Prism (involved many people in the design, and it is widely used now) vs RubyVM::AST (involved very few people, used very little due to design flaws & impractical).
2. Alternatives should be considered early, to avoid "reinventing the wheel in a suboptimal way". The obvious alternative here is sub-interpreters, which is widely used (https://bugs.ruby-lang.org/issues/21311#note-47). In fact CRuby even had a mvm branch for this a long time ago. @tagomoris (Satoshi Tagomori) Was this considered, what advantages do namespaces have compared to sub-interpreters?
3. The motivation needs to be clear and based on real-world use cases.

This feature missed all 3 rather catastrophically:

The current updated design was presented only days before it was merged. I find this irresponsible, especially when there are still so many problems with it. No wonder many people are reacting negatively to this.
Granted, the previous design was explained in #19744.
I found a critical flaw with that design (https://bugs.ruby-lang.org/issues/19744#note-74) and as a result it seems the feature design was reconsidered to address that.
That part is good, however, this is only one of several serious concerns raised by many people in #19744.

The biggest concern for any big feature should be what does it solve, is the motivation good enough to warrant the added complexity, performance overhead, etc?
Let's look at the motivations from https://bugs.ruby-lang.org/issues/21311#note-28:

Mounting server-less applications (run on AWS Lambda, separated on processes in production) on a single app server for development

I can't see any significant benefit compared to just running each server-less application in its own process, as I said in https://bugs.ruby-lang.org/issues/21311#note-37.
Maybe I'm missing some, then let's point those out, but clearly the advantages should be established way before merging the feature, i.e. early in the design process.
There are many drawbacks to use namespaces for this use case, notably no proper isolation and no parallelism.
I doubt any advantage can outweigh that. Also running things so differently than in production is bound to have issues (e.g. passes locally but fails in production).

Mounting two different revisions of an application for useful deployments (older and newer commits of an app)

Can do that with multiple processes, more compatible and even necessary if there is process-global state relied on (e.g. CWD).
There can be a process receiving external requests, passing it to the two variants and comparing them, nothing new, it has been done many times.
As far as I see the only advantage of namespaces for this is slightly easier communication but not fully compatible, lot more complexity, no parallelism, no proper isolation.

Mounting two different set of dependency versions of an application

Basically the same as above regarding trade-offs of using namespaces for this.

Let's look at https://bugs.ruby-lang.org/issues/21311#Motivation too (already discussed in https://bugs.ruby-lang.org/issues/21311#note-13):

Avoiding name conflicts between libraries

Too rare to matter, this is a solved problem (a gem should only declare one top-level constant corresponding to the gem name), doesn't need any new solution.

Avoiding unexpected globally shared modules/objects

This is a problem (e.g. Oj global state/configuration) but namespaces are a poor solution for it: they duplicate the memory and load multiple copies of Oj (also making copies of the native library on the filesystem BTW).
The proper solution there is like what the JSON gem did: provide an API to have that state locally (JSON::Coder or using a JSON::State instance explicitly).
Also namespaces just wouldn't solve this if gems A and B depend on Oj and modify the Oj global state incompatibly, gems A and B would have to be modified just to use namespaces which is just not gonna happen
(i.e. I don't think RubyGems or Bundler can do that transparently, gems A & B will refer to Oj with Oj, not ns::Oj. Loading A & B in namespaces just makes the same problem but for gems depending on A or B).

Multiple versions of gems can be required

This has been discussed extensively in #19744 and it seems pretty unhealthy for the Ruby ecosystem.
I also don't see anyone going to decide to use namespaces to workaround some issue in some gem, forking the gem repository + fix on a branch is far more sensible and less risky than starting to use namespaces in your application to workaround some gem issue.

@matz (Yukihiro Matsumoto) What real-world use cases do you see for Namespace?

[tenderlovemaking (Aaron Patterson)]

Eregon (Benoit Daloze) wrote in #note-82:

@matz (Yukihiro Matsumoto) What real-world use cases do you see for Namespace?

One thing I can think of is running multiple Rails applications in one process. Right now Rails applications are basically singletons. Of course we could rearchitect Rails to not be a singleton, but loading the app in a namespace sounds much easier. Of course today those apps couldn't run in parallel in one process, but if we can keep improving Ractors, then I could see a world with a Ractor webserver that is able to serve multiple Rails apps all in one process.

This is just my idea though.

[tenderlovemaking (Aaron Patterson)]

• Related to Feature #21335: Namespaces should be present in the backtrace added

[Eregon (Benoit Daloze)]

What are the advantages of running multiple Rails apps in one process instead of one process per Rails app?

That seems similar to the "Mounting multiple server-less applications in one process" motivation, and so I see the same disadvantages, mainly:

• no proper isolation so one app can crash the other, or something (e.g. a native extension) depends on process-global state and it just doesn't work
• parallelism with Ractor seems unlikely given how hard it is to make gems Ractor-compatible and would need to adapt Ractors to Namespaces otherwise no chance for gems to even load (e.g. can't set an @ivar on a class in non-main Ractor, can't set a constant with a non-shareable value in non-main Ractor, etc).
• not saving any significant amount of memory compared to multiple processes since namespaces basically duplicate everything and share very little.

[tenderlovemaking (Aaron Patterson)]

Eregon (Benoit Daloze) wrote in #note-85:

What are the advantages of running multiple Rails apps in one process instead of one process per Rails app?

One advantage is operational simplicity, only one process to boot / monitor. Namespaces could also be used as a stepping stone for migrating a Rails app to an engine.

I think namespaces would also be helpful when testing legacy code that depends on singletons. We have many tests in the Rails codebase that use fork for isolation, and I think namespaces could help there.

[fxn (Xavier Noria)]

I have written a blog post trying to present this feature in a digested way and provide a mental model for all Ruby programmers.

Hope it is all factually correct, and it transmits faithfully the design behind namespaces. In particular, hope the mental model of RPC to think about cross-namespace communication is a good one.

Please let me know if there's anything to be revised!

[vo.x (Vit Ondruch)]

fxn (Xavier Noria) wrote in #note-87:

I have written a blog post trying to present this feature in a digested way and provide a mental model for all Ruby programmers.

Thanks. Very nice overview.

However, given I have not paid close attention to the whole discussion (sorry), can you also elaborate how does it look with RubyGems? Do I need to require "rubygems" to be able to load e.g. bundled gems libraries?

[bughit (bug hit)]

Technically, the reference is the same, the object ID is the same, but the builtin object state is per-namespace now.

If this is referring to instance state, like ivars, this is a corruption of the concept of an object. If the state is different it can't be the same object. It would be like two string references being the "same" object but holding different string values, an absurdity.

[fxn (Xavier Noria)]

bughit (bug hit) wrote in #note-89:

If this is referring to instance state, like ivars, this is a corruption of the concept of an object. If the state is different it can't be the same object. It would be like two string references being the "same" object but holding different string values, an absurdity.

That part of the post is talking exclusively about the builtin class and module objects.

The rule is the following: Within a namespace, two objects with the same ID are the same. Which gives you a consistent view of the world within that namespace. If they are not, it is a bug (but see also my last remarks in this comment).

However, across namespaces, builtin objects have the same ID. Take this example

# foo.rb
module Foo
  def self.m(klass)
    klass.object_id # => 16, same as in main, it is the same reference
    klass::X # NameError
  end
end

# main.rb
Object.object_id # => 16, same as in foo.rb, it is the same reference

X = 1

ns = Namespace.new
ns.require('foo')
ns::Foo.m(Object) # NameError

You pass a class object in the method call, but it is a builtin class object. These have special rules. So, that same reference in ns has the original "reset" state, which does not have X among its constants. (The constants in a class or module object are part of their state.)

I actually raised this point during the conversation, and we have two ways to look at this:

1. This feature is new, and as I have also said before, it needs a new entity with new rules. This is a new rule.
2. @tagomoris (Satoshi Tagomori) made the good point that in today's Ruby, refinements already allow two references to have observable different properties in the same program depending on the execution context.

[fxn (Xavier Noria)]

vo.x (Vit Ondruch) wrote in #note-88:

Thanks. Very nice overview.

Thank you.

However, given I have not paid close attention to the whole discussion (sorry)

Nah, no prob, it has been a long one with so many angles.

can you also elaborate how does it look with RubyGems? Do I need to require "rubygems" to be able to load e.g. bundled gems libraries?

I am not 100% sure of what is the vision there, I'll let @tagomoris (Satoshi Tagomori) or @mame (Yusuke Endoh) reply this one.

But, since namespaced code starts with a blank slate, you cannot think about use cases in terms of "I'll run this individual test in a namespace" or somesuch, because in the namespace there is nothing. You'd need to start at Bundler require.

The only way out to not running Bundle require in each namespace that I can think of is that the driver namespace passes an executor of some sort to the test. That is, your foo.rb starts clean, but you inject the already loaded stuff via some indirection. I have not done this but I think it would be doable.

We need to learn the patterns.

[vo.x (Vit Ondruch)]

But, since namespaced code starts with a blank slate

I was more wondering where is the cut line, because it might not be obvious. E.g. in the case of RubyGems, the overridden require is not something one thinks about these days.

Maybe all the preludes are question to me.

We need to learn the patterns.

I actually look forward to this.

[Eregon (Benoit Daloze)]

vo.x (Vit Ondruch) wrote in #note-88:

can you also elaborate how does it look with RubyGems? Do I need to require "rubygems" to be able to load e.g. bundled gems libraries?

Each user namespace will automatically load its own copy of RubyGems when created, that's the only way to be correct and compatible: #21324

[vo.x (Vit Ondruch)]

Eregon (Benoit Daloze) wrote in #note-93:

vo.x (Vit Ondruch) wrote in #note-88:

can you also elaborate how does it look with RubyGems? Do I need to require "rubygems" to be able to load e.g. bundled gems libraries?

Each user namespace will automatically load its own copy of RubyGems when created, that's the only way to be correct and compatible: #21324

Just to be clear, I don't agitate for loading RubyGems, just wonder where the line is. Having clean pristine environment is very unique opportunity and if the namespace have to be created, then doing additional require "rubygems" is not a big deal.

[bughit (bug hit)]

fxn (Xavier Noria) wrote in #note-90:

1. This feature is new, and as I have also said before, it needs a new entity with new rules. This is a new rule.

There are things that new features should not do, like redefining and corrupting the concept of a object.

2. @tagomoris (Satoshi Tagomori) made the good point that in today's Ruby, refinements already allow two references to have observable different properties in the same program depending on the execution context.

My understanding is refinements can alter behavior (methods) of an object for a certain scope. That could be surprising (for the same instance) but easier to justify since it's understood that there's a level of indirection for behavior, methods don't live in the instance. Divergent instance state on the other hand (ivars or other private data like the char array of a string) on the same instance is complete nonsense. The example I used to illustrate the absurdity of this is two string references to the "same" string object but having different string values. Now perhaps this effect will be limited to built-in class objects only, but it's still conceptual corruption and my string example becomes justifiable and consistent with this feature.

[fxn (Xavier Noria)]

bughit (bug hit) wrote in #note-95:

The example I used to illustrate the absurdity of this is two string references to the "same" string object but having different string values.

I am not sure I follow. The edge case in which things have the same ID but different state cross-namespace are the builtin classes and modules, not their instances.

The same string reference may respond to blank? in some namespaces and not respond to blank? in others.

Or do you have something else in mind?

[bughit (bug hit)]

fxn (Xavier Noria) wrote in #note-96:

I am not sure I follow. The edge case in which things have the same ID but different state cross-namespace are the builtin classes and modules, not their instances.

This argument would work for other languages where classes are magical templates that can be endowed with all kinds of special traits. But this is ruby where a class is an object, an instance of Class and is obliged to follow the rules of objects. Since it is an object, it should not have this bizarre trait that objects are not supposed to have. An object identity and the instance state are in a one-to-one relationship. Its called instance state or instance variables for a reason. Its specific to the instance/identity. The alternative is insane: two string references to the "same" string object but having different string values. Though this feature does not allow this specific manifestation of this absurdity, it allows the general idea behind it.

[fxn (Xavier Noria)]

@bughit I totally see what you mean, and I had myself this very objection up in the thread.

While my post had the aim of spreading the word and make namespaces accessible to a broader audience, I have personally still not reached a definitive opinion about these details.

Now we have only one namespace, and in the new paradigm, things are still as they are today per namespace (mod the leaks, but that will go away, I think). The rules get tricky when we are in cross-namespace territory. But this is a new territory! We need to experiment and understand the new model.

Think about it, can today a single program have objects with the same ID that are not equal? Yes! Absolutely it can!

That may happen when you fork, and we do not perceive anything weird here because we are so used to think about processes as isolates that being the same program does not suggest an inconsistency to us. Even though in the same program the processes can communicate and exchange object IDs.

My intuition is that we'll start to think about a lot of stuff "per namespace" from now on. Like globals, per namespace.

From my perspective, that is not right or wrong in an absolute sense of the word. There is no one single definition of OOP or FP. Heck, the only thing that exists for real are CPU instructions, everything else is an abstraction we invent. How many languages allow you to modify something as fundamental as Object or Kernel? Not so many! Ruby has a very unique culture.

In the end, a programming language is a formal language, and it is up to the language designers to set the rules. You can define INTERCAL if you wish, the rules are yours and you can depart from what is considered standard as much as you want and give shape to your own variant of a theme.

What we all expect is that the rules are consistent.

Not saying you have to agree with any of these points, please feel free to strongly disagree, but maybe that opens a bit the way we all can look at it and adapt to the new concepts.

[bughit (bug hit)]

fxn (Xavier Noria) wrote in #note-98:

Think about it, can today a single program have objects with the same ID that are not equal? Yes! Absolutely it can!

Object id in this conversation is a proxy for identity, so the correct question is: Can today a single program have an object that is value unequal to itself. The current answer is no, and allowing it to be "yes" by making possible divergent instance state/variables for different references to the same object, seems like a dangerous, cavalier and unjustified corruption of a foundational concept in the language.

That may happen when you fork, and we do not perceive anything weird here because we are so used to think about processes as isolate

When you fork, you end up with two distinct and independent objects. That they have the same object id is incidental. It could also happen through random chance, across time and space and it would be just as irrelevant. We are discussing a situation where the same object id means there is actually only one object.

[bughit (bug hit)]

bughit (bug hit) wrote in #note-99:

Can today a single program have an object that is value unequal to itself.

I mean by the default and intuitive definition of value equality which is comparing instance state. Obviously you can define a contrived non-comparison which would just return false for self.

[fxn (Xavier Noria)]

bughit (bug hit) wrote in #note-99:

Object id in this conversation is a proxy for identity, so the correct question is: Can today a single program have an object that is value unequal to itself. The current answer is no, and allowing it to be "yes" by making possible divergent instance state/variables for different references to the same object, seems like a dangerous, cavalier and unjustified.

Could we speak source code? Which is one concrete situation that bothers you?

[fxn (Xavier Noria)]

For everyone following, we are talking about stuff like:

# main.rb
module Kernel
  @a = 1
end

p Kernel.object_id
p Kernel.instance_variable_defined?(:@a)

ns = Namespace.new
ns.require_relative('foo')
ns::Foo.m(Kernel)

# foo.rb
class Foo
  def self.m(obj)
    p obj.object_id
    p obj.instance_variable_defined?(:@a)
  end
end

That prints

8
true
8
false

and we see the difference because we have visibility over two namespaces at the same time. If we only lived within a namespace, it would not be observable, as far as I can tell.

Communication cross-namespaces is, I think, where the new paradigm may get tricky. And you may like or not like how this works.

[bughit (bug hit)]

fxn (Xavier Noria) wrote in #note-101:

Could we speak source code? Which is one concrete situation that bothers you?

Well, you posted the code, not that it needed demonstrating as it was already admitted. Two references to the same object showing divergent instance state (ivars). This is not just a one-off special case hack, it's messing with the core general conception of an object.

[fxn (Xavier Noria)]

As of today, I also lean on the side of not feeling comfortable with this particular aspect of namespaces.

Isolation is easy to explain and understand. While new, I believe it is intuitive 👍.

However, this so open cross-namespace reference passing, constant access, ..., I think it may be confusing and has controversial (non-intuitive?) implications like this one we are discussing.

For example, without trying, can people following the thread say what this would print?

module Kernel
  @a = 1
end

ns = Namespace.new
p ns::Kernel.instance_variable_defined?(:@a)

[rubyFeedback (robert heiler)]

First, I would like to thank mame for the explanation in regards to the namespace discussion, or rather the pre-discussion in regards to design decision(s). Probably not everyone knew about the prior discussion(s), so it seems useful to know about it.

I would also like to thank Satoshi Tagomori for the idea(s) and effort. It is actually really difficult to come up with good proposals and implementation(s). I myself wanted to have some kind of "namespace-like" feature or functionality in Ruby, but I have not been able to come up with any good proposal. It's actually not so trivial - I have a few rough ideas, but when it comes to detail, as well as the actual implementation of those details and ideas, there are things I was not certain about, and I then gave up on it. So it is great that others manage to come up with their own ideas.

That is not to say I am all in favour of the proposal here (but, neither am I in disfavour really); I actually think it is good that ideas are put forward in motion, even if these may be different ideas, or may not be "fleshed out" in detail, so to speak, in every regard. To me this is not necessarily a huge problem, because I think bugs as well as specification and documentation can be detailed at a later time too, and I think we had this with JIT/yjit/mjit/whatever-the-jit too.

Eregon (and others) is (or may be) a bit skeptical and asked for some use cases; tenderlove provided one or two. I think I can also provide one or two, based on another thought process actually. I understand that these are not directly related to the namespace-suggestion here, so I am not saying that what I write may convince Eregon or anyone else, but I do think that these additional ideas may perhaps provide Eregon with another perception.

Again, this is not to say that I say the proposal here is the one I would have had in mind, but I think if we look at things at a broader perspective - including refinements, and why so few people use it, despite refinements being a good idea - we may come a bit closer to useful ideas.

So perhaps, just three pointers, and I will leave it at that:

(1) Many years ago, on IRC, freenode, on a #mono or c-sharp channel, a C-sharp user did not like that in ruby we don't have separate "namespaces", e. g. everyone can duck-patch ("monkey-patch") things. While I think this is a useful feature (ruby being dynamic), I can somewhat understand the criticism, even if I do not fully share it. This was many many years ago, before ruby added refinements by the way. (In regards to refinements, one thing that always stopped me from using refinements was the syntax; the "use" clause in particular. But I digress.)

(2) On rubygems, we can have only one gem with a certain name, such as "configuration". I would use a module for this, e. g. "module Configuration", but of course this module would potentially clash with someone else who used the same module. And I could not host this on rubygems because a gem called configuration already exists. There are some workarounds for this, such as renaming the project, but ultimately I want to have the toplevel name "Configuration", as that makes the most sense to me: this is for a project I thought about, where all configure-options would be stored in individual .yml files, and the user could save the content of the .yml files dynamically (and reload them dynamically, too, including query-methods, such as for instance "use_colours?" as method, becomes use_colours.yml, as a file, and the content can be the boolean false or true - and so forth). On github we can have multiple projects with the same name, so github has a bit more flexibility than rubygems. So, my thought process here was that we could perhaps add "meta-information" to classes and modules, such as "module Configuration; author_name = 'Jim the tiny'; version = '1.0.0'; of course the latter is usually done via VERSION, as many projects use that, but I thought this could reside within a class or module as such, as meta-information). (In my ideas, I would not have added a separate "namespace" concept, but extended classes and modules instead, but no matter the implementation details, I think it is more useful to think about the underlying use cases here.)

For instance, ruby recently reserved "module Ruby", if I remember correctly:

https://rubygems.org/gems/ruby

Probably to be used by ruby itself. (This may have future use cases, e. g. we could
perhaps add things to it at a later time, or group certain things inside of it, but
I digress.)

If we would have meta-tag information for modules and classes, this could then allow
us to decide what to do with things we load or require. For instance, we could
decide that "module Configuration" would be reserved rather than extended, if someone
else would use it. This would NOT be the default, but people could decide that for
their own projects - a bit like with refinements. Now, I understand that this can
become a bit complicated (and was one reason why I did not try to propose something
like that), but if you ignore the implementation details, then I think some ruby
developers may find it useful to be able to disallow changes made, to certain
classes and modules, at their own discretion, for their own projects. So, not to
handicap other users of ruby, but for their own stuff, a bit like refinements.

I can not say how useful such restrictions would be, but I think there may be a
rationale for that. I can not say how many people would actually use it (see
refinements), but we have a somewhat similar situation with @@class_variables -
some people use it, some don't. (I don't actually; I found that it is easier to
not worry about them and just stick to @instance_variables, including on the
toplevel. Often I have just a singleton-like variable there, such as module Foo;
@use_colours = true; and go from there.)

The idea here would also be to allow rubygems to be more open about the same
name, just like github. Then we could perhaps do: "gem install fancypants --author
tenderlove" or whatever the syntax should be, if such a gem called fancypants were
to exist. (Could also be specified in a gemspec file of course.)

Of course one drawback would be that it is a bit more complicated, so one also has
to evaluate how useful this could be. But ignoring the complexity, I kind of feel
somewhat strongly about wanting to retain certain SIMPLE names, such as "module
Configuration" or "module Colours" (or module Colors; which already exists by the
way) and so forth.

(3) Last but not least, related to this partially, is the idea to add a new way
to "import" ruby code. We already have require() (including require_relative() and
load() and I think these have to remain how they are, due to backwards compatibility,
and matz also probably wants to keep this part of ruby simple. So a new way could
perhaps be used; I am not very creative, so "import" seemed a good name, even if e. g.
python already uses it. So what would be the idea behind "import"? Well, we would keep
this extensible for the future; and it could allow for more flexibility in loading code.
For instance, "importing" into a specific namespace, and using another name, too. So if
we have:

module Foobar

Via import-functionality, this could become "module Barfoo". Of course this is already
possible, if we assign Barfoo = Foobar; and then Foobar = nil I guess, but I found the
idea more elegant to bundle together useful things into a new import-like functionality.
And it could also offer auto-load like functionality - I don't actually mean to behave
like autoload, but to be able to "import" ruby code where it is irrelevant WHERE the
.rb file resides. Right now we often have to hardcode the path via a .rb file, and if
that is moved, downstream code can break. My idea would be to have "import" also help
us in this case, and automatically work, no matter where the .rb file is.

The idea behind import is a tiny bit related to what Satoshi Tagomori showed in regards
to:

ns1.require('./app1') # 2048

(Though it seems to just use require, which is a bit different to my idea, but again,
that's for other discussions.)

Anyway. I don't really have an extremely strong opinion about the proposal here per se
either way; at the same time I kind of think that ruby could benefit from more thoughts in regards to larger projects, many .rb files, same-named modules and classes and so forth. This may not convince Eregon in regards to the proposal here, but I actually think that we should discuss things. Perhaps there could be a developer meeting in due time, at a later time, some months from now on, when many of the bugs were resolved, to see which ideas may work and which ones may not work. I don't know how open Satoshi Tagomori and matz are to amendments (I don't have any, mind you) but I also think it is perfectly fine to see how things go forward. IMO ruby could need some more "top-level" or "fine-tuned" control here. And making good proposals and implementing them, is a lot of work, so I think it is great that Satoshi Tagomori puts in time and effort here.