Project

General

Profile

Actions

Misc #20488

closed

Document source file size restrictions

Added by kddnewton (Kevin Newton) 6 months ago. Updated 5 months ago.

Status:
Closed
Assignee:
-
[ruby-core:117866]

Description

I was hoping we might be able to decide on an official maximum size for a source file in terms of bytes/lines/columns/etc.

Ruby uses fixed integer sizes to represent line numbers, column numbers, offsets, etc. These can overflow with files that are too big. Sometimes they will fail with cryptic messages like:

ruby: negative string size (or size too big) (ArgumentError)

(That's for 2^31 "a"s in a row.) Note that for python for the same file you will get:

OverflowError: Parser column offset overflow - source line is too big

and for perl you will get:

Identifier too long at test.pl line 1.

For files with 2^32 newlines, Ruby just crashes on my machine with the current parser (prism finishes, but its newline counter overflows so it gets all the offsets wrong).

Would it be okay to say:

  • maximum line: 31 bits
  • maximum column: 32 bits
  • maximum file byte size: 32 bits

This would also help with memory savings — I would like to only use 32 bits for offsets in the file, as opposed to the current 64 bits I'm using (which seems unnecessarily large).

Actions #1

Updated by matz (Yukihiro Matsumoto) 5 months ago

The limit will be declared (for CRuby):

  • maximum line: signed 32 bits (eval uses negative line numbers)
  • maximum column: unsigned 32 bits
  • maximum file byte size: unsigned 32 bits

Matz.

Updated by kddnewton (Kevin Newton) 5 months ago

  • Status changed from Open to Closed

Thank you!

Actions

Also available in: Atom PDF

Like0
Like0Like0