Project

General

Profile

Feature #13252

C API for creating strings without copying

Added by tenderlovemaking (Aaron Patterson) about 2 years ago. Updated about 2 years ago.

Status:
Assigned
Priority:
Normal
Target version:
-
[ruby-core:79763]

Description

Hi,

I'd like to have a C API that allows me to create String objects without copying the underlying char *. Basically a C API similar to the rb_str_new_static, but have the GC free the underlying char * when the object dies. The use case is that at work we have C libraries that allocate char * and we want to pass those to Ruby land without copying the buffer. Here is an example of what we're doing now:

https://github.com/arthurnn/memcached/commit/1886546944b420dc6953096ba1f5eae772001e31#diff-f508f9b8263ea397534b2b3f8efed987R147

I'd like it if there was a public API for doing something like this.

Thank you!

P.S. I am sure I can't be the first to ask for this, but I couldn't find a similar issue in RedMine, so if this has been answered I apologize.
P.P.S. I've added a patch for kind of what I want.


Files

out.diff (1.66 KB) out.diff tenderlovemaking (Aaron Patterson), 02/25/2017 12:19 AM

History

Updated by nobu (Nobuyoshi Nakada) about 2 years ago

  • Description updated (diff)

It is not guaranteed that ruby_xfree can free a pointer allocated by other than ruby_xmalloc and so on.
You can't mix bare malloc and ruby_xfree.

Updated by shyouhei (Shyouhei Urabe) about 2 years ago

Nobuyoshi Nakada wrote:

It is not guaranteed that ruby_xfree can free a pointer allocated by other than ruby_xmalloc and so on.
You can't mix bare malloc and ruby_xfree.

Agreed. I heard that DLLs can have their own malloc() implementation on Windows. Even on POSIX variants, memory regions (e.g. mmap()-ed ones) are not always guaranteed to be free()-able.

This proposed C APIs are too easy to be misused when publicized. And misuse of them directly results in SEGV. I don't think it's a wise idea.

Updated by normalperson (Eric Wong) about 2 years ago

shyouhei@ruby-lang.org wrote:

Nobuyoshi Nakada wrote:

It is not guaranteed that ruby_xfree can free a pointer allocated by other than ruby_xmalloc and so on.
You can't mix bare malloc and ruby_xfree.

Agreed. I heard that DLLs can have their own malloc() implementation on Windows. Even on POSIX variants, memory regions (e.g. mmap()-ed ones) are not always guaranteed to be free()-able.

This proposed C APIs are too easy to be misused when publicized. And misuse of them directly results in SEGV. I don't think it's a wise idea.

One place we can use this in C Ruby is ruby_getcwd() on GNU libc.

Yes, it's dangerous if misused. Perhaps having a way to plug-in
and redefine alloc/free/realloc functions would be safe on a
per-T_STRING basis.

We can maybe use FL_USER{3,4,5}, and STR_NOFREE flags for
non-embed strings and allow up to 14 non-standard plug-in
*alloc implementations for users to define.

(4 bits; 16 implementations possible, 2 of which are built-in:
1: normal malloc 2: no-free, replacing existing STR_NOFREE cases)

Updated by nobu (Nobuyoshi Nakada) about 2 years ago

normalperson (Eric Wong) wrote:

We can maybe use FL_USER{3,4,5}, and STR_NOFREE flags for
non-embed strings and allow up to 14 non-standard plug-in
*alloc implementations for users to define.

FL_USER{3,4,5} are for RSTRING_EMBED_LEN_MASK.

#5

Updated by shyouhei (Shyouhei Urabe) about 2 years ago

  • Status changed from Open to Assigned

Updated by shyouhei (Shyouhei Urabe) about 2 years ago

  • Assignee set to ko1 (Koichi Sasada)

Also available in: Atom PDF