Project

General

Profile

Bug #12280

IO.copy_stream(IO, IO) fails with "pread() not implemented"

Added by Eregon (Benoit Daloze) over 3 years ago. Updated over 3 years ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:74938]

Description

IO.copy_stream on Windows with IO having an offset fails with:
NotImplementedError: pread() not implemented

This seems suboptimal, should a clearer be reported?
Is it a platform limitation?

History

Updated by normalperson (Eric Wong) over 3 years ago

eregontp@gmail.com wrote:

IO.copy_stream on Windows with IO having an offset fails with:
NotImplementedError: pread() not implemented

This seems suboptimal, should a clearer be reported?

Not sure. Maybe include a bug report address to get pread
implemented in Windows? :)

Is it a platform limitation?

Yes, it's a platform limitation.

lseek+read can be used to workaround the limitation in
single-threaded programs only (and only as long as the FD is not
shared across fork; maybe FD sharing across fork isn't possible
in Windows...)

But I guess an error message is better than a racy workaround
which can corrupt data with concurrent access.

Updated by Eregon (Benoit Daloze) over 3 years ago

Eric Wong wrote:

eregontp@gmail.com wrote:

IO.copy_stream on Windows with IO having an offset fails with:
NotImplementedError: pread() not implemented

This seems suboptimal, should a clearer be reported?

Not sure. Maybe include a bug report address to get pread
implemented in Windows? :)

Is it a platform limitation?

Yes, it's a platform limitation.

lseek+read can be used to workaround the limitation in
single-threaded programs only (and only as long as the FD is not
shared across fork; maybe FD sharing across fork isn't possible
in Windows...)

But I guess an error message is better than a racy workaround
which can corrupt data with concurrent access.

Thanks for the quick reply!

For me, having a message like:
"NotImplementedError: IO.copy_stream with offset is not supported on Windows since it is not atomic"
would be a good enough fix for this.

Updated by naruse (Yui NARUSE) over 3 years ago

Benoit Daloze wrote:

For me, having a message like:
"NotImplementedError: IO.copy_stream with offset is not supported on Windows since it is not atomic"
would be a good enough fix for this.

IO.copy_stream is a utility method.
It works as expected and fast if there's a suitable API like sendfile.

On this precondition, IO.copy_stream should work even on Windows.
You can simply acquire GVL or fine grained lock to achieve multithread safety.

Anyway you can provide pread with ReadFile.
You may know, on Windows read() is our own function and the real name is rb_w32_read.
You can implement rb_w32_pread with ReadFile in win32/win32.c.

Updated by shyouhei (Shyouhei Urabe) over 3 years ago

+1 to add a fallback implementation when pread is unavailable.

Updated by normalperson (Eric Wong) over 3 years ago

naruse@airemix.jp wrote:

Benoit Daloze wrote:

For me, having a message like:
"NotImplementedError: IO.copy_stream with offset is not supported on Windows since it is not atomic"
would be a good enough fix for this.

IO.copy_stream is a utility method.
It works as expected and fast if there's a suitable API like sendfile.

On this precondition, IO.copy_stream should work even on Windows.
You can simply acquire GVL or fine grained lock to achieve multithread safety.

Can open files be shared across processes in Windows?
If so, thread-safety is not enough; pread provides atomicity across
forked processes (and also send_io/recv_io) in *nix; not just threads.

Anyway you can provide pread with ReadFile.
You may know, on Windows read() is our own function and the real name is rb_w32_read.
You can implement rb_w32_pread with ReadFile in win32/win32.c.

Anyways, I'll leave Windows devs to implement if it is possible
and safe :)

Also available in: Atom PDF