Bug #15787: LoadError by EPERM on read-only volume

• Status: Feedback
• Assignee: -
• Target version: -
• ruby -v: ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-haiku]
• Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
• Tags: maybenotbug, lib, haiku

[extrowerk (Zoltán Mizsei)]

On Haiku the package management just virtually extracts/populates the files, and as it doesn't have write-overlay feature, the populated files are read-only.

Issue: Ruby has to maintain a list of installed gems in a single file, in rubygems.rb. Ruby stats this file, and bails out if it is read-only:

~ » ruby --version
ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-haiku]
~ » ruby
Traceback (most recent call last):
        1: from <internal:gem_prelude>:2:in `<internal:gem_prelude>'
<internal:gem_prelude>:2:in `require': Operation not supported -- /boot/system/lib/ruby/2.6.0/rubygems.rb (LoadError)
~ » ls -la /boot/system/lib/ruby/2.6.0/rubygems.rb
-r--r--r-- 1 user root 36970 márc.  6 10:01 /boot/system/lib/ruby/2.6.0/rubygems.rb
~ » uname -a
Haiku shredder 1 hrev53091 Apr 22 2019 22:17:21 x86_64 x86_64 Haiku

The 2.2.x branch was not affected, but since 2.3 every version have this problem. Happens here: https://github.com/ruby/ruby/blob/trunk/ruby.c#L2098

Either it has to create this file somewhere in non-packaged (this is the writeable folderstructure) and hope it won't get out of sync when pkgman (package updater software) updates Ruby packages, or it has to be fixed to enumerate them directly by examining the directories.
The fact that this file is located in a packaged tree also hints that Ruby will probably want to also use packaged location for manual gems installs, which obviously won't work either.

Question: is it possible to force/instrument ruby to use a user-specific "rubygems.rb" if the system one read only?

Probably this problem exists on other platforms too, where the user doesn't have write-right to this file. How is it handled?

Thank You!

[extrowerk (Zoltán Mizsei)]

• Description updated (diff)

[shevegen (Robert A. Heiler)]

This is indeed unfortunate but I guess the code could be changed. Keep in mind that the ruby core
team probably has not that much experience with haiku - most will use linux or mac OSX, some
windows or one of the BSDs.

Your problem description reminds me a bit of the situation of gems, to some extent, e. g.
where we can use --userinstall to install into the home directory (in particular debian
has this tendency to change target locations, e. g. /var/*).

You mentioned that the behaviour in 2.2.x was different (by the way, have you been the haiku
users who tried to get ruby to work in the past on haiku?), so I would classify this as a
regression (since it worked before), although I think none of this was deliberate - there are
not that many heroic haiku users out there. For --userinstall, typically the home directory
is writable for the user, so that should work fine.

Is this more related to how rubygems works, though?

Perhaps your issue is related to both rubygems, and ruby.c; I believe that the ruby core team
and the rubygems team will be sympathetic to the described problem, in particular because
there is indeed no trivial way to change the read-only situation. And I think the ruby team
wants to have ruby work on as many different operating systems as is reasonable possible, so
it seems reasonable to assume that there is an interest in resolving the problem at hand.

I think what might help most is if you could suggest what changes you would consider to be
best, in particular to (a) have it work for future versions of haiku and (b) to not break
any other ruby installations (on other operating systems). A simple hackish work around
for locations could always be an environment variable, but I am not sure how well the
ruby team knows haiku to know what will work to resolve this; this is why I think the ruby
team may prefer if you could give some recommendations.

What happens when ruby is upgraded on haiku, say versions 2.6.3 to version 2.7.0 or
like that?

[extrowerk (Zoltán Mizsei)]

shevegen (Robert A. Heiler) wrote:

This is indeed unfortunate but I guess the code could be changed. Keep in mind that the ruby core
team probably has not that much experience with haiku - most will use linux or mac OSX, some
windows or one of the BSDs.

I know, this is why i am here :) Our ruby port is way too old and needs to be updated, but for that this issue have to be solved first.

You mentioned that the behaviour in 2.2.x was different (by the way, have you been the haiku
users who tried to get ruby to work in the past on haiku?), so I would classify this as a
regression (since it worked before), although I think none of this was deliberate - there are
not that many heroic haiku users out there. For --userinstall, typically the home directory
is writable for the user, so that should work fine.

Yep, in 2.2.x there were no problem on Haiku, according the 2.2.10 sources (in ruby.c) it was not checked if the file is RO. So for us it is a regression, but maybe it is required or logically needed.
Beside other HaikuPorts team members i also worked on ruby in the past ( https://github.com/haikuports/haikuports/pulls?utf8=%E2%9C%93&q=is%3Apr+author%3Aextrowerk+ruby+ ), but i only use it for jekyll, so forgive me if i make logical mistakes or doesn't understand the importance of some code-blocks, maybe they are required in production.

I know about --userinstall, this is the workaround what i see everywhere on the internet as workaround, but it doesn't solves the problem for us, as ruby checks the rubygems.rb before starting to doing anything and it bails out even with --userinstall on Haiku.

Perhaps your issue is related to both rubygems, and ruby.c; I believe that the ruby core team
and the rubygems team will be sympathetic to the described problem, in particular because
there is indeed no trivial way to change the read-only situation. And I think the ruby team
wants to have ruby work on as many different operating systems as is reasonable possible, so
it seems reasonable to assume that there is an interest in resolving the problem at hand.

We are also definetely interested to have an up to date ruby port in our package manager.

I think what might help most is if you could suggest what changes you would consider to be
best, in particular to (a) have it work for future versions of haiku and (b) to not break
any other ruby installations (on other operating systems). A simple hackish work around
for locations could always be an environment variable, but I am not sure how well the
ruby team knows haiku to know what will work to resolve this; this is why I think the ruby
team may prefer if you could give some recommendations.

I tried to fix it earlier ( https://github.com/haikuports/haikuports/pull/700 ), it just disables the check, but the HaikuPorts team was not amused, so we never merged this change, however it worked for me.

Consider the following:

• some programs using ruby and needs some ruby gems installed. To the package_daemon be able to fullfill the dependencies, we have to package the gems as we do for the python modules, as it cannot just "gem install" things (version constraints, know-working versions, etc), so if a user installs the program it will guaranteed to work. Thus we will install some gems in system folders, they will be read-only, of course.
• the user must be able to install gems trough "gem" too. They will be installed in the home folder.

The system and user gems needs to be enumerated at start, at least both location have to be searched for the imported gems. For this we can use the vendor/sites folders AFAIK.

Now 2 questions:

• what if the same gem installed as system and as user too? Maybe even different versions? Does ruby picks automatically the never one? I am pretty sure it is handled somehow already in ruby.
• Does rubygems.rb needs to be writeable at all?

If it needs to be writeable, then is it possible to add a user-specific "rubygems.rb" to override/extend the system one? If so, how is it handled on other platforms?
If not, then i don't know, why does ruby checks for it.

What happens when ruby is upgraded on haiku, say versions 2.6.3 to version 2.7.0 or
like that?

I am not sure if i understand this question. Currently the system gems installed to " /boot/system/lib/ruby/gems/2.6.0 " which is read only, while the user gems installed to "$HOME/.gems/ruby/2.6.0" (i plan to change it later, on Haiku there should be no dot-folders in the home, and nothing should place any configuration/settings files directly into $HOME, it makes a mess. The plan is to move them to either /boot/system/non-packaged/lib/ruby/gems/2.6.0 or to /boot/home/config/non-packaged/lib/ruby/gems/2.6.0 , as we do it for python. This folders are writeable.
Then if ruby gets upgraded it won't find the gems installed for the previous version, but so we can even provide different ruby versions in the package manager, as we do it for python (python2 and python3)

I hope i answered your question. If something still unclear, feel free to ask me.

[wishdev (John Higgins)]

There is no such requirement as rubygems.rb in that folder being written to. I run locked down lxc containers on a daily basis that have no such writing ability (read only roots) and ruby works just fine. That folder is also not generally writeable on my systems for a standard user either even if the root is not read only.

I'm also generally unaware (and I could be very wrong) but there is no "list" of installed gems anywhere that I can find.

I don't believe you are barking up the correct tree here.

The file you mention is required to be readable because it is the core of the rubygems module - but there is nothing written to that file at all.

[extrowerk (Zoltán Mizsei)]

wishdev (John Higgins) wrote:

The file you mention is required to be readable because it is the core of the rubygems module - but there is nothing written to that file at all.

Well, it's trying to lock the file, which is readonly, anyway, so locks are useless. So if ruby doesn't needs to write to this file then why is it locked?

It is possible that i don't see something obvious here, btw.

[extrowerk (Zoltán Mizsei)]

Ok, i consulted with PulkoMandy, one of the main Haiku developer, he meant it is not possible to unset "O_NONBLOCK or O_NDELAY because we don't allow it" on packaged file-system. This is why this check fails.
We make it sure the file opened without O_NONBLOCK or O_NDELAY from the beginning.

I made a smal ltestprogram, which tries to do this, and it works on normal files, but fails on our packaged files.

So i assumed it correctly, it is a limitation what needs to be handled here.

[extrowerk (Zoltán Mizsei)]

• File ruby-changes.patch ruby-changes.patch added

Here is my patchset against the current master. Please let me know if anything else required.
Patch tested working on Haiku.

[nobu (Nobuyoshi Nakada)]

• Related to Bug #11559: ビジーループの thread と YAML.parse を組み合わせたときの実行時間が 2.2.3 で遅くなっている added

[nobu (Nobuyoshi Nakada)]

• Related to deleted (Bug #11559: ビジーループの thread と YAML.parse を組み合わせたときの実行時間が 2.2.3 で遅くなっている)

[nobu (Nobuyoshi Nakada)]

• Related to Bug #11060: load(fifo) blocks whole process added

[nobu (Nobuyoshi Nakada)]

• Subject changed from rubygems.rb on read-only volume to LoadError by EPERM on read-only volume
• Status changed from Open to Feedback

Sorry, I have wrongly got this as a rubygems specific error by just reading the subject.
Opening in non-blocking mode is not to block the whole process by loading from a fifo, then reset it to get rid of problems on the DATA stream.
Would't just ignoring EPERM suffice?

[extrowerk (Zoltán Mizsei)]

nobu (Nobuyoshi Nakada) wrote:

Sorry, I have wrongly got this as a rubygems specific error by just reading the subject.
Opening in non-blocking mode is not to block the whole process by loading from a fifo, then reset it to get rid of problems on the DATA stream.
Would't just ignoring EPERM suffice?

Sorry, but because the missing knowledge i cannot give you a correct and helpful answer, but i would gladly test it if you can propose a better fix for this.
Thanks!

[hsbt (Hiroshi SHIBATA)]

• Tags set to haiku, lib, maybenotbug