Project

General

Profile

Actions

Feature #13726

closed

PATCH: Windows builds - fractional second file times

Added by MSP-Greg (Greg L) over 6 years ago. Updated over 6 years ago.

Status:
Closed
Target version:
-
[ruby-core:81946]

Description

At present, I believe Windows builds have integer seconds resolution on File times.

Attached are three patches that seem to correct the issue. The numbered patches are best applied in order.

The c source patches were created by kubo (Kubo Takehiro); I created the spec patch, along with some testing.

All related tests (run parallel) pass on my system for MinGW builds and x64-mswin64_140 builds. I did some checks with Appveyor, and there may be issues with the logger tests, due to the fact that the tests are very time sensitive (I ran -j3 on a quad system, Appveyor runs -j2 on a dual core).

I believe file time values are stored with 100 ns resolution, but times have 100 µs resolution. This was the reason for some of the changes to the tests, as time appears to be truncated (floor), as opposed to round.

I would appreciate if consideration could be given to adding this (or similar) code. Thank you.


Files

02-file-utime.patch (5.53 KB) 02-file-utime.patch MSP-Greg (Greg L), 07/07/2017 02:25 AM
spec-rubyspec-core-file-time.patch (2.99 KB) spec-rubyspec-core-file-time.patch MSP-Greg (Greg L), 07/07/2017 02:25 AM
01-time-subsec.patch (15.2 KB) 01-time-subsec.patch MSP-Greg (Greg L), 07/07/2017 02:25 AM
spec-rubyspec-core-file-time.patch (3.28 KB) spec-rubyspec-core-file-time.patch Updated utime MSP-Greg (Greg L), 07/11/2017 02:42 PM

Related issues 2 (0 open2 closed)

Related to Ruby master - Misc #13702: TZ checking in rb_w32_fstati64()Closedusa (Usaku NAKAMURA)Actions
Related to Ruby master - Feature #13731: inode for Windows on ReFSClosedusa (Usaku NAKAMURA)Actions
Actions #1

Updated by MSP-Greg (Greg L) over 6 years ago

  • Subject changed from Windows builds - fractional second file times to PATCH: Windows builds - fractional second file times

Updated by MSP-Greg (Greg L) over 6 years ago

I was mistaken about Time.now resolution in windows.

What I believe does occur is that when windows natively writes file times, it seems to write them with 100 µs resolution, and truncated.

But, Ruby can write with higher resolution using utime (after the patch).

Updated by kubo (Takehiro Kubo) over 6 years ago

01-time-subsec.patch changes declarations in include/ruby/win32.h.
IMO, it could be in ruby 2.5.x but could not be backported to ruby 2.4.x because of ABI incompatibility.
It also deletes meaningless TZ checking I pointed at #13702.

Updated by MSP-Greg (Greg L) over 6 years ago

While running tests today on ruby 2.5.0dev (2017-07-11 trunk 59311) [x64-mingw32], I had an error in spec/rubyspec/core/file/utime_spec due to the truncation issue. So, I've modified the test code.

When looking at this issue, please replace the patch in the first message with the attached one.

Updated by usa (Usaku NAKAMURA) over 6 years ago

  • Status changed from Open to Assigned
  • Assignee set to usa (Usaku NAKAMURA)
Actions #6

Updated by usa (Usaku NAKAMURA) over 6 years ago

  • Related to Misc #13702: TZ checking in rb_w32_fstati64() added
Actions #7

Updated by usa (Usaku NAKAMURA) over 6 years ago

Actions #8

Updated by usa (Usaku NAKAMURA) over 6 years ago

  • Tracker changed from Bug to Feature
  • ruby -v deleted (trunk)
  • Backport deleted (2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN)

Updated by usa (Usaku NAKAMURA) over 6 years ago

I've tested the patches, and I found that my Windows7 box often set the ctime and mtime as a little future -- some hundreds nano seconds.
I can't find the reason, but it makes some troubles with test-all and test-spec...

Updated by MSP-Greg (Greg L) over 6 years ago

usa (Usaku NAKAMURA) wrote:

I've tested the patches, and I found that my Windows7 box often set the ctime and mtime as a little future -- some hundreds nano seconds.
I can't find the reason, but it makes some troubles with test-all and test-spec...

Thanks for looking at it. I'll look over the patches I'm using in ruby-loco, as all the time tests/specs have been stable for quite a while, both locally and on Appveyor. I believe all the time patches are in the patches/gte20500 folder. Disk IO on AV gets a little funny when running parallel tests/specs, so I did need to increase (or add) some deltas in any numeric comparison tests...

Thanks, Greg

EDIT: Sorry, I should have been clearer, I may have increased some of the deltas in the current test/spec patches in ruby-loco to allow for reliable parallel testing on Appveyor.

Updated by larskanis (Lars Kanis) over 6 years ago

@usa (Usaku NAKAMURA) I think the reason is, that precise file times are working as expected on Windows 7, but Time.now has a precision of 1/64 seconds only. This is because Windows 7 doesn't support GetSystemTimePreciseAsFileTime(), which is required to retrieve precise Time.now: https://github.com/ruby/ruby/blob/b7de978e450779103dfe62084ce87757c5acae7d/win32/win32.c#L4604

So to remove these inconsistencies we could truncate file times to 1/64 precision, when GetSystemTimePreciseAsFileTime() is not available?

Updated by usa (Usaku NAKAMURA) over 6 years ago

@larskanis (Lars Kanis), thank you for information.

larskanis (Lars Kanis) wrote:

So to remove these inconsistencies we could truncate file times to 1/64 precision, when GetSystemTimePreciseAsFileTime() is not available?

OK, I'll test it.

Actions #13

Updated by usa (Usaku NAKAMURA) over 6 years ago

  • Status changed from Assigned to Closed

Applied in changeset trunk|r61013.


support nanosec file timestamp on newer Windows

Support nanosec file timestamp on Windows 8 or later.
Original patches are written by kubo (Kubo Takehiro).
Windows 7 and earlier also supports nanosec file timestamp, but it's too
accurate than system time. so, this feature is disabled on such versions.
[Feature #13726]

this change also includes [Misc #13702]

Updated by kubo (Takehiro Kubo) over 6 years ago

@usa (Usaku NAKAMURA) Thanks for merging the patches.

I'm afraid that File.mtime(filename).usec returns always zero just after ruby.exe starts up on Windows 8 or later.

have_precisetime is -1 at process startup.
https://github.com/ruby/ruby/blob/5a8a270/win32/win32.c#L4574

It is changed to zero or one in get_systemtime().
https://github.com/ruby/ruby/blob/5a8a270/win32/win32.c#L4586-L4589

If File.mtime is called before get_systemime(), the fractional second is always zero even on Windows 8 or later.
https://github.com/ruby/ruby/blob/5a8a270/win32/win32.c#L5493

I cannot check it by myself because I failed to compile ruby from SVN source.
Could somebody check it?

Updated by larskanis (Lars Kanis) over 6 years ago

@kubo (Takehiro Kubo) I had the same concern, but tried the latest RubyInstaller snapshot. It proves that get_systemtime() is called somewhere as part of the initialization.

$ ruby --disable-gems -e "p File.mtime('README').usec"
657156
$ ruby -v
ruby 2.5.0dev (2017-12-05 trunk 61034) [x64-mingw32]

Updated by kubo (Takehiro Kubo) over 6 years ago

@larskanis (Lars Kanis) Thank you! I worried needlessly.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0