Feature #13252: C API for creating strings without copying

• Status: Assigned
• Assignee: ko1 (Koichi Sasada)
• Target version: -

[tenderlovemaking (Aaron Patterson)]

Hi,

I'd like to have a C API that allows me to create String objects without copying the underlying char *. Basically a C API similar to the rb_str_new_static, but have the GC free the underlying char * when the object dies. The use case is that at work we have C libraries that allocate char * and we want to pass those to Ruby land without copying the buffer. Here is an example of what we're doing now:

https://github.com/arthurnn/memcached/commit/1886546944b420dc6953096ba1f5eae772001e31#diff-f508f9b8263ea397534b2b3f8efed987R147

I'd like it if there was a public API for doing something like this.

Thank you!

P.S. I am sure I can't be the first to ask for this, but I couldn't find a similar issue in RedMine, so if this has been answered I apologize.
P.P.S. I've added a patch for kind of what I want.

[nobu (Nobuyoshi Nakada)]

• Description updated (diff)

It is not guaranteed that ruby_xfree can free a pointer allocated by other than ruby_xmalloc and so on.
You can't mix bare malloc and ruby_xfree.

[shyouhei (Shyouhei Urabe)]

Nobuyoshi Nakada wrote:

It is not guaranteed that ruby_xfree can free a pointer allocated by other than ruby_xmalloc and so on.
You can't mix bare malloc and ruby_xfree.

Agreed. I heard that DLLs can have their own malloc() implementation on Windows. Even on POSIX variants, memory regions (e.g. mmap()-ed ones) are not always guaranteed to be free()-able.

This proposed C APIs are too easy to be misused when publicized. And misuse of them directly results in SEGV. I don't think it's a wise idea.

[normalperson (Eric Wong)]

[e-mail] wrote:

Nobuyoshi Nakada wrote:

It is not guaranteed that ruby_xfree can free a pointer allocated by other than ruby_xmalloc and so on.
You can't mix bare malloc and ruby_xfree.

Agreed. I heard that DLLs can have their own malloc() implementation on Windows. Even on POSIX variants, memory regions (e.g. mmap()-ed ones) are not always guaranteed to be free()-able.

This proposed C APIs are too easy to be misused when publicized. And misuse of them directly results in SEGV. I don't think it's a wise idea.

One place we can use this in C Ruby is ruby_getcwd() on GNU libc.

Yes, it's dangerous if misused. Perhaps having a way to plug-in
and redefine alloc/free/realloc functions would be safe on a
per-T_STRING basis.

We can maybe use FL_USER{3,4,5}, and STR_NOFREE flags for
non-embed strings and allow up to 14 non-standard plug-in
*alloc implementations for users to define.

(4 bits; 16 implementations possible, 2 of which are built-in:
1: normal malloc 2: no-free, replacing existing STR_NOFREE cases)

[nobu (Nobuyoshi Nakada)]

normalperson (Eric Wong) wrote:

We can maybe use FL_USER{3,4,5}, and STR_NOFREE flags for
non-embed strings and allow up to 14 non-standard plug-in
*alloc implementations for users to define.

FL_USER{3,4,5} are for RSTRING_EMBED_LEN_MASK.

[shyouhei (Shyouhei Urabe)]

• Status changed from Open to Assigned

[shyouhei (Shyouhei Urabe)]

• Assignee set to ko1 (Koichi Sasada)