Feature #14758: Add a first-class support for isolated bounded packages / modules / contexts

• Status: Open
• Assignee: -
• Target version: -

[grzesiek (Grzegorz Bizon)]

While one of the core principles of Ruby is to extend the language in a way it is a most useful and convenient tool that a software developer can have in their toolbox, lack of a first-class isolation on module level can cause some serious problems when project grows beyond some size.

This is especially visible in large projects, where most of the code lives in the lib/ directory, and there are tens or hundreds of modules there. Ideally we would like to make these modules isolated and hide complexity behind facades. Currently it is not possible to isolate such modules, because a developer can still reach beyond boundary of a bounded context, and use MyModule::InternalClass directly.

It is very difficult to enforce boundaries, currently it requires a lot of work to implement complex static analysis rules.

Would it make sense to add support for first-class package, context or boundary, that would be a regular module but would not allow referencing inner constants from outside?

context MyModule
  class MyIsolatedClass
    # ...
  end

  def self.build
    MyIsolatedClass.new
  end
end

MyModule::MyIsolatedClass # raises context violation
MyModule.build # => Returns an instance of MyModule::MyIsolatedClass

I'm pretty sure that I failed at finding similar feature proposal that has been already submitted, in that case sorry for that!

Please let me know what do you think about that! Thanks in advance! <3

[grzesiek (Grzegorz Bizon)]

• Description updated (diff)

[dsferreira (Daniel Ferreira)]

Hi Grzegorz,

My proposal regarding the implementation of Internal Interfaces (internal access modifier) https://bugs.ruby-lang.org/issues/9992
was pondered when thinking around those terms but applies to internal classes/modules that can not be set as private.

The idea of some kind of isolation at the namespace level.

What I use to do to hide internal modules and classes all together is to use:

module Foo
  class Bar
  end

  private_constant :Bar
end

a = Foo::Bar.new
# => NameError: private constant Foo::Bar referenced

As a Best Practice I use the top module as the gem API and let all internal classes and modules as private.
This poses some challenges but by using dependency injection we can easily overcome them.

Also I only test the API.
I don't unit test internal modules or classes.
That way I'm free to redesign the internal architecture without the need to touch the tests and without breaking anything.
Hope this helps.

P.S.

Would love to hear your thoughts on the Internal Interface proposal!

[dsferreira (Daniel Ferreira)]

Sorry. I just corrected the typo:
private_constant :Foo
should read:
private_constant :Bar

[shevegen (Robert A. Heiler)]

I am mostly neutral to somewhat against it; not because of the
additional information, but because of the restrictions put on
ruby hackers.

I think Daniel already summed some part of it up via:

Also I only test the API.
I don't unit test internal modules or classes.

Which I think makes sense. The internal part should not be relevant,
neither for testing, nor anyone else to access it (why would anyone
access internals? Did you provide means aka an API to access the
functionality? Because if you did not, then you could do so -
and if you did, I don't understand why anyone would WANT to access
the internals, anyway?)

The part of the proposal here, though:

MyModule::MyIsolatedClass # raises context violation

goes against ruby's philosophy in my opinion since you attempt to
restrict what people can do. Ruby still has .send() which I love;
.public_send() was added but .send() remains as-is and I am pretty
sure that .send() will always be there - it's sort of the smalltalk
influence on ruby here.

Why would you want to prevent others from accessing any part of
ruby code? Is it a real problem? Does something break if others
do? In all the explanations given, I have never read an explanation
as to WHY it should be restricted exactly.

You can always document for people to not do so, and instead
provide common API entry points.

What I do is to usually define top-level module-methods,
such as:

module Foobar

   # code here

And then call it like:

Foobar.do_this_or_that

where internally classes can be called through these methods.

I also do this in large projects, at the least for the more
important classes. And these usually are callable from the
commandline via "--options" like:

foobar --clean-up-directories
foobar --clean-directories
foobar --cdirs

like aliases.

Also note that ruby's core philosophy is to let people (ruby
hackers) do what they want to. That is one reason why there
is no strict separation between public/private, and why
we always had .send(), which is similar to smalltalk. Python
does not have a strong public/private distinction either by
the way.

All these proposals ultimately always try to restrict what ruby
hackers can do, for no good reason. I am not against providing
additional cues, mind you - for example, I'd like to have a way
to specify extra information to denote on the toplevel
"namespaces", so that we can distinguish whether something is
"owned" by ruby core (like top-level Kernel), as opposed to
other modules. From this context I'd also like to redefine
refinements (the proposal for them), but it's a bit complicated
and not trivial to write a good proposal for change. It's much
easier to make smaller changes than large ones.

Also in the linked issue:

::Foo::Bar.new.baz    => error: internal method being called outside ::Foo namespace

To me, it goes against the old ruby philosophy. But I think it is
not really good to comment against it because, either way, it will
be matz who decides, and you only have to convince matz, not me nor
anyone else. And he once said that you can persuade him with arguments
but keep in mind that others also feel strongly about losing flexibility
that they used to have. :)

[dsferreira (Daniel Ferreira)]

Hi Robert,

I agree that we should not change current ruby behaviour.
I see the proposal as an extra functionality not a behaviour change.

Regarding allowing ruby hackers to do all they want:

• Ruby allows developers to find their way to accomplish what they want. 100% freedom is good in my point of view.
• You mentioned Object#send as an example
• private_method vs public_method is another example
• For sure we can find a way to force "internal interface" access for ruby hackery joy. :-)

The problem with enterprise code is that sometimes it is hard to do proper code reviews and force best practices.
I have that experience.
In big teams with very big projects ruby code can become unmanageable.
That is one of the reasons big corporations still use JAVA very extensively.
Putting barriers to bad practices helps a lot in this management.

In the mean time my advise would be to try to implement a better code review process if possible.
(Lots of times it is not)
Maybe requiring two reviewers instead of just one?
Teams, specially when they are new to ruby tend to do all kinds of architecture mistakes.
Why? Because ruby just allows it.
And when the project managers are only concerned with delivery times that is what you get.
Once you need more than the standard Rails MVC files structure things will tend to be done in the wild land.
Enter lib/ dir and all you will find is caos.
Do you use service classes?
SOLID patterns?
It may help as well...

shevegen (Robert A. Heiler) wrote:

Also in the linked issue:

Can you please comment on the specific issue ticket?
I just linked it because I think it's related to the subject in question here.

[grzesiek (Grzegorz Bizon)]

Thank you for your replies Daniel, Robert!

Also I only test the API. I don't unit test internal modules or classes.

I guess that this is usually a bad idea, but it depends on what you are using your tests for. When I have to develop a complex module I usually follow TDD because TDD is a design tool for me, not a testing tool per se. But I would like to avoid diving into discussions about TDD / testing here.

Daniel, internal modifiers are interesting, but I think this feature would not work for me.

I would like to make isolation a default strategy for a module, I don't want to remember about specifying isolation modifier each time I create a constant. I think that package / context / boundary isolation would be mostly useful in case of complex modules, that consist of many internal constants.

Robert, thanks for the comment about that this might be against Ruby's philosophy of not restricting users. But then Encapsulation, one of the core principles of Object Oriented Paradigm, would also be against Ruby's philosophy, is that correct?

I personally believe that hiding complexity is not against Ruby principles. Programming in general is about building abstractions, and hiding complexity behind facades, and restricting access to internals is working in favor of making code conceptually manageable, which should help a programmer. And this is probably top priority of Ruby language - to make Ruby a more efficient language.

Why would you want to prevent others from accessing any part of ruby code? Is it a real problem? Does something break if others do? In all the explanations given, I have never read an explanation as to WHY it should be restricted exactly.

"All non-trivial abstractions, to some degree, are leaky."

People are going to use internal code of a module, because it is more simple than thinking about building / extending abstractions. Extending an abstraction involves thinking in terms of building intention-revealing interfaces and a good design, and this is much more complex than just leaking an abstraction. I would like to have a mechanism that will tell developers that in this particular case, this is not a good idea. But if they decide to do that anyway, there should certainly be away of doing that.

Also note that ruby's core philosophy is to let people (ruby hackers) do what they want to.

Sure, I'm not against doing that, but I would like to have a mechanism that will reveal my intention of hiding internal constants better. I'm not aware of any existing mechanism like this at the moment.

In big teams with very big projects ruby code can become unmanageable. That is one of the reasons big corporations still use JAVA very extensively. Putting barriers to bad practices helps a lot in this management.

I've seen that as well. Ruby code can deteriorate quickly because there is no good way of hiding complexity. But please note that I'm not talking about creating barriers here, I'm talking about passing knowledge, and intention of an author of a module to people who are going to work on it in the future. I believe this makes things more simple in the end, instead of more complex, and is a win-win solution for everyone.

[dsferreira (Daniel Ferreira)]

grzesiek (Grzegorz Bizon) wrote:

I guess that this is usually a bad idea, but it depends on what you are using your tests for. When I have to develop a complex module I usually follow TDD because TDD is a design tool for me, not a testing tool per se. But I would like to avoid diving into discussions about TDD / testing here.

I use TDD.
One thing doesn't imply another.
Think about it:
You set your test for the scenario on the public interface method.
Red light.
You create your code logic to get the green light.
Add tests for other scenarios.
Get green light for the other scenarios.
Get 100% coverage.
You can now refactor the code logic at your own will redefining your architecture and tweaking performance without thinking ever about tests.
And you know that if your test suite passes and you still have 100% coverage all your code is good.

;-)

[grzesiek (Grzegorz Bizon)]

dsferreira (Daniel Ferreira) wrote:

I guess that this is usually a bad idea, but it depends on what you are using your tests for. When I have to develop a complex module I usually follow TDD because TDD is a design tool for me, not a testing tool per se. But I would like to avoid diving into discussions about TDD / testing here.

I use TDD.
One thing doesn't imply another.

Sure, but this way you don't use it as a design tool for internals of your module. I believe that people simply use TDD for different things. But let's don't clutter this thread with discussions about TDD, please :)