Bug #22239: Multiple buffer clear errors in IO

• Status: Open
• Assignee: -
• Target version: -
• Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN

[YO4 (Yoshinao Muramatsu)]

While testing my changes on the Windows platform to use an encoding converter for newline conversion for "r", I discovered a failure in test_reopen. This corresponds to case (3) below.

I asked an LLM to investigate similar cases and related code that raised concerns, and was able to identify several issues, so I am reporting them here. I'll submit a PR later.
I used Hy3 (High) via opencode GO to perform this survey and create the test code. (For transparency—this is not an advertisement.)

case(1)
In io.c, free_io_buffer only clears ptr and left off, len and capa as they were.
Since clear_readconv releases the character buffer through it, READ_CHAR_PENDING() kept reporting pending characters after the buffer had been released.
As a result, every operation that clears the code converter (IO#rewind, IO#seek, IO#pos=, IO#flush, IO#binmode, IO#tell, IO#set_encoding, ...) left the IO in a state where byte oriented reads raise IOError.
Also IO#eof? answers false at the end of file and IO#sysseek refuses to work.

File.binwrite("t", "foo")
open("t", "rt") do |f|
  f.ungetc(f.getc)
  f.rewind
  f.getbyte # => 'IO#getbyte': byte oriented read for character buffered IO (IOError)
end

case(2)
IO#reopen(io) only flushes the write buffer when the IO is writable, and never unreads the read buffer.
The buffer happens to be dropped by the io_seek after dup2, but only when the other stream is readable and seekable, so a read-write IO keeps stale bytes otherwise.

File.binwrite("t", "0123456789")
f = File.open("t", "r+b")
f.getbyte
IO.pipe do |r, w|
  w.write("ABC"); w.close
  f.reopen(r)
  f.read # => "123456789ABC" instead of "ABC"
end

case(3)
IO#reopen(name) discards the byte buffer but keeps the code converter and the character buffer, so characters left over from the previous stream leak into the reopened one.

File.binwrite("t", "foo")
open(File::NULL, "rt") do |f|
  f.ungetc("a")
  f.reopen("t")
  f.read # => "afoo" instead of "foo"
end

case(4)
IO#reopen(io) relies on flush_before_seek to drop the buffers, but io_unread returns at once when rbuf is empty and therefore never reaches clear_codeconv.
With characters pending in cbuf and nothing in rbuf they leak into the reopened stream.

File.write("t", "foo\n")
open(File::NULL, "rt") do |f|
  f.ungetc("a")
  open("t", "rt") {|f2| f.reopen(f2)}
  f.gets # => "afoo\n" instead of "foo\n"
end

case(5)
Unlike IO#rewind, IO#seek and IO#pos= do not clear the character buffer.
As io_unread returns at once when rbuf is empty, an ungotten character survives the repositioning while it does not in binary mode.

File.write("t", "foo\n")
open("t", "rt") do |f|
  f.ungetc("a")
  f.seek(2, :SET)
  f.getc # => "a" instead of "o"
end

[YO4 (Yoshinao Muramatsu)]

GH PR created. https://github.com/ruby/ruby/pull/18276