Bug #19703: text/binary mode of parent process fd is not properly inherited on windows

• Status: Open
• Assignee: -
• Target version: -
• Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

[YO4 (Yoshinao Muramatsu)]

Windows OS can't inherit parent process fds greater than 2, but C runtime library support this.
https://learn.microsoft.com/en-us/cpp/c-runtime-library/spawn-wspawn-functions#environment-of-the-spawned-process

So I made test tool below.
This tool opens NUL device several times and create child ruby process that dumps fds' translation mode.

// fdinherit.c : test file descprotor inheritance for windows C runtime
#include <stdio.h>
#include <stdlib.h>
#include <process.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <share.h>
#include <fcntl.h>
#include <io.h>

int main(int argc, char* argv[])
{
    int fh, count = 0;
    errno_t err;
    // child process: read fds and check if it includes "\r" and prints
    // '.' : read as binary
    // 't' : read as text
    // 'F' : open error (no fd?)
    char rb1[] =
        "print '012'; 3.upto("; char rb2[] = ") { |n|"
          "print ((IO.for_fd(n, 'rb').read.include?(13.chr) ? '.' : 't') rescue 'F');"
          "puts if (n % 10) == 9"
        "}";
    char upto[] = "-4294967295";

    // fd max number to test
    if (argc >= 2) count = atoi(argv[1]);
    if (count < 3) count = 7;

    // create crlf file (current directory)
    system("echo;>crlf.txt");

    // open fds to inherit
    // even fds are text mode, odd fds are binary mode
    for (int i = 3; i <= count; i++) {
        err = _sopen_s(&fh, "crlf.txt", _O_RDONLY | ((i & 1) ? _O_BINARY : _O_TEXT), _SH_DENYNO, _S_IREAD | _S_IWRITE);
        if (err > 0) printf("%d %d\n", err, fh);
    }

    // on child side, +1'd fd dump will fail (prints 'F')
    sprintf_s(upto, sizeof(upto), "%d", count + 1);
    // on windows, arguments are joined into a string. so below has bad manners, but let me take a few corners.
    _spawnlp(_P_WAIT, "ruby.exe", "ruby.exe", "-e", "\"", rb1, upto, rb2, "\"", NULL);

    _close(fh);
}

As an example, this would display something like this

>fdinherit.exe
012.t.t.F
>fdinherit.exe 16
012.t.t.t.
t.t.t.tF
>

0,1,2 are stdin, stdout, stderr
fds of 3,5,7,... are inherited as binary mode
fds of 4,6,... are inherited as text mode

And I found a few things to point out.
When I modify ruby part like "IO.for_fd(n, 'r')" / "IO.for_fd(n, 'rt')" / "IO.for_fd(n)",
It get result like 012tttttF, so translation mode are interprited wrong.
The odd thing is that the conversion mode is interpreted as is when "IO.for_fd(n, 'rb')", which I think is the expected behavior when "IO.for_fd(n, 'r')".

This tool and ruby.exe each work with a combination of msvcrt 32/64bit and ucrt64, so the C runtime interface seems stable.
My understanding is that MRI does not state that it supports inheritance of fds larger than 3 in windows, but it seems to work partially.
Are there any chances for improvement?

For those who want to follow up: it seems that fds are not inherited via cmd.exe /c