Feature #22222: Expose a C API equivalent of `RubyVM::InstructionSequence.load_from_binary`

• Status: Open
• Assignee: -
• Target version: -

[byroot (Jean Boussier)]

Currently the only API to deserialize instruction sequence is RubyVM::InstructionSequence.load_from_binary(String).

It works fine, but isn't ideal for gems like bootsnap, as it needs to allocate a fairly large string and memcpy into it.

I would like to be able to directly pass a char *ptr and size_t len pair, so that I can directly pass a internal buffer or even mmapped file.

Proposed API:

VALUE rb_iseq_load_from_binary(const char *ptr, size_t len)

Pull request: https://github.com/ruby/ruby/pull/18136

[nobu (Nobuyoshi Nakada)]

rb_str_new_static does not memcpy (of course you have the responsibility to keep the String).

[byroot (Jean Boussier)]

@nobu (Nobuyoshi Nakada) yes I know, but as you mention, it's meant to point at static C strings, not mmaped files or stack buffers, so it doesn't work for Bootsnap.

[byroot (Jean Boussier)]

• Description updated (diff)

[nobu (Nobuyoshi Nakada)]

The only requirements for rb_str_new_static are that the caller is responsible for freeing the allocated memory, and that the memory is not freed while the object is still in existence.
You can think of it as requiring the same level of care as when passing a buffer to a system call.
The term “static” essentially means that the memory is stable from the perspective of this function.

[byroot (Jean Boussier)]

Yes, I get that, but my point is that these constraints are very hard to ensure in my case.

[Eregon (Benoit Daloze)]

I guess one concrete issue here with using rb_str_new_static() is the mmaped file/stack buffer will be freed soon after the RubyVM::InstructionSequence.load_from_binary(String) call, but that String object might live longer, there is no "GC this object now" (AFAIK).
And potentially something could get that String via ObjectSpace.each_object, and then it'd segfault if the underlying buffer has been freed.