Bug #12757: Wrong overflow check in rb_str_set_len()

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 2.1: UNKNOWN, 2.2: REQUIRED, 2.3: REQUIRED

[rhenium (Kazuki Yamaguchi)]

string.c:

void
rb_str_set_len(VALUE str, long len)
{
    long capa;
    const int termlen = TERM_LEN(str);

    str_modifiable(str);
    if (STR_SHARED_P(str)) {
        rb_raise(rb_eRuntimeError, "can't set length of shared string");
    }
    if (len + termlen - 1 > (capa = (long)rb_str_capacity(str))) {
        rb_bug("probable buffer overflow: %ld for %ld", len, capa);
    }
    STR_SET_LEN(str, len);
    TERM_FILL(&RSTRING_PTR(str)[len], termlen);
}

The overflow check len + termlen - 1 > (capa = (long)rb_str_capacity(str)) is wrong, as the capa does not include the space for termlen. This can cause false-positive [BUG] for String with multi-byte termlen when setting the length to the number equal to the capacity.

For example, the following code that internally calls rb_str_set_len() causes the [BUG]:

str = String.new(capacity: 100, encoding: "UTF-32BE")
IO.pipe { |r, w|
  w.write("x"*100)
  r.read(100, str)
}

• Status changed from Open to Closed

Applied in changeset r56148.

---

string.c: fix buffer overflow check condition in rb_str_set_len()

• string.c (rb_str_set_len): The buffer overflow check is wrong. The
  space for termlen is allocated outside the capacity returned by
  rb_str_capacity(). This fixes r41920 ("string.c: multi-byte
  terminator", 2013-07-11). [ruby-core:77257] [Bug #12757]

• test/-ext-/string/test_set_len.rb (test_capacity_equals_to_new_size):
  Test for this change. Applying only the test will trigger [BUG].