Bug #10902
require("enumerator") scans LOAD_PATH 2x on every invocation
Added by tmm1 (Aman Gupta) about 6 years ago.
Updated over 5 years ago.
Description
On every invocation of require "enumerator"
(for example during boot when many gems require it), the VM will scan the load path twice: once for enumerator.rb and again for enumerator.so. Of course, no file is found because enumerator is now shipped within the VM by default.
$ ruby -e' p $LOADED_FEATURES[0] '
"enumerator.so"
$ ruby -e' p $LOAD_PATH.size '
8
$ strace -e trace=open ruby -e' 1.times{ require "enumerator" } ' 2>&1 | grep enumerator.rb | wc -l
8
$ strace -e trace=open ruby -e' 1.times{ require "enumerator" } ' 2>&1 | grep enumerator.so | wc -l
8
$ strace -e trace=open ruby -e' 10.times{ require "enumerator" } ' 2>&1 | grep enumerator.so | wc -l
80
$ strace -e trace=open ruby -e' 100.times{ require "enumerator" } ' 2>&1 | grep enumerator.so | wc -l
800
In enumerator.c, we call rb_provide("enumerator.so")
which adds it to $LOADED_FEATURES. This means require "enumerator.so"
can be optimized, but most libraries do not include the .so extension when requiring enumerator.
This gets much worse as $LOAD_PATH grows. For example in our app $LOAD_PATH.size is 351, causing 702 failed open() syscalls per require.
Having a hard time coming up with a clean patch here. The following works but it's pretty hacky.
diff --git a/load.c b/load.c
index fa225fa..68d15e7 100644
--- a/load.c
+++ b/load.c
@@ -952,6 +952,9 @@ rb_require_safe(VALUE fname, int safe)
} volatile saved;
char *volatile ftptr = 0;
+ if (strcmp(RSTRING_PTR(fname), "enumerator") == 0)
+ return Qfalse;
+
if (RUBY_DTRACE_REQUIRE_ENTRY_ENABLED()) {
RUBY_DTRACE_REQUIRE_ENTRY(StringValuePtr(fname),
rb_sourcefile(),
I tried to fix it int load.c, but failed rubyspec:
http://80x24.org/spew/m/1436044472-18990-1-git-send-email-e@80x24.org.txt
However, this one-liner seems to work:
--- a/enumerator.c
+++ b/enumerator.c
@@ -2060,6 +2060,7 @@ InitVM_Enumerator(void)
rb_define_method(rb_cYielder, "yield", yielder_yield, -2);
rb_define_method(rb_cYielder, "<<", yielder_yield_push, -2);
+ rb_provide("enumerator.rb");
rb_provide("enumerator.so"); /* for backward compatibility */
}
I'm not sure if we need something similar for complex and rational,
not sure if they were really used via require in the past...
http://80x24.org/spew/m/1436044803-31588-1-git-send-email-e@80x24.org.txt
It will disallow a library named "enumerator.rb".
Also available in: Atom
PDF