Feature #12733
closedBundle bundler to ruby core
Description
rubygems team has plan to migrate bundler into rubygems at rubygems 3.0.
ref. https://github.com/rubygems/rubygems/issues/1681
In current status, rubygems only use Bundler
module on test suites.
But we should track rubygems upstream step by step. I propose to bundle bundler
for stdlib as default gem on ruby core repository.
but we have some consideration. such as
- bundler uses rspec for test suites. How to use rspec on ruby core repository?
- Who are maintain bundler on ruby core repository?
This issue is epic topic. Please comment your concerns or idea.
Updated by Eregon (Benoit Daloze) almost 8 years ago
Won't this make it harder to update bundler?
Updated by vo.x (Vit Ondruch) almost 8 years ago
Hiroshi SHIBATA wrote:
- bundler uses rspec for test suites. How to use rspec on ruby core repository?
Not only this, but the test suite also needs connection to internet to succeed. No other test needs this so far and I hope it will stay like that.
Updated by duerst (Martin Dürst) almost 8 years ago
Vit Ondruch wrote:
Not only this, but the test suite also needs connection to internet to succeed. No other test needs this so far and I hope it will stay like that.
There are many parts in Ruby that (would) need connection to the Internet for decent testing. Webrick, open_uri, net::http, and so on immediately come to my mind. Also rubygems of course. Also several i18n-related tests (for String#unicode_normalize/upcase/downcase/...), where data needs to be downloaded from the unicode.org Web site if not yet available.
In the case of the i18n tests, they get skipped if the Internet connection fails. My guess is that all the other components that I mention do something similar. So this should also be possible for bundler, even if it's not yet the case.
In general, it would be a bad idea to not have tests that try to connect to the Internet and exercise the relevant functionality, in particular for a language such as Ruby where one of the very big areas of use in Internet/Web related. But of course, it has also to be possible to test (to the extent possible) in a standalone environment.
Updated by hsbt (Hiroshi SHIBATA) over 7 years ago
- Status changed from Open to Assigned
Updated by hsbt (Hiroshi SHIBATA) over 7 years ago
Created patch of PoC
Updated by indirect (André Arko) about 7 years ago
the test suite also needs connection to internet to succeed
The Bundler test suite no longer requires an internet connection to succeed.
I propose to bundle bundler for stdlib as default gem on ruby core repository
I am not sure what is needed for this, but I would like to help figure out what would be needed.
Updated by hsbt (Hiroshi SHIBATA) about 7 years ago
- Target version set to 2.5
I am not sure what is needed for this, but I would like to help figure out what would be needed.
I hope to promote bundler to default gems on standard libraries.
Requirements of default gems is following things:
- Works with cross-compilation(It's not related bundler)
- Guarantee to work with ruby trunk. So we need to test bundler with test suite of ruby core.
test suite is big problem. so bundler uses rspec. test suite of ruby core is customized test-unit/minitest.
We can choose some strategy.
- Rewrite bundler's rspec to minitest/spec
- Ruby core bundle rspec and integrate rspec and test suite of ruby core.
- Prepare isolation environment for bundler and rspec without test suite of ruby core. We add new test task named "make test-bundler` and iinvoke it on rubyci.
- etc...
These are only technical side problem.
I'm going to solve this problem used isolation environment
as first.
Updated by Eregon (Benoit Daloze) about 7 years ago
hsbt (Hiroshi SHIBATA) wrote:
We can choose some strategy.
- Rewrite bundler's rspec to minitest/spec
- Ruby core bundle rspec and integrate rspec and test suite of ruby core.
- Prepare isolation environment for bundler and rspec without test suite of ruby core. We add new test task named "make test-bundler` and iinvoke it on rubyci.
- etc...
These are only technical side problem.
I'm going to solve this problem used
isolation environment
as first.
I think isolation environment is the best strategy (or the only reasonable).
Rewriting tests is not an option for upstream development and is very time-consuming/error-prone (minitest/spec is a tiny subset of RSpec).
I am not sure what you mean by bundling rspec, but rspec should not be available to normal Ruby users like default gems.
It's a private dependency needed to run tests, and it probably makes most sense to install it as a gem when running the tests.
I suspect it might also be difficult to run the tests in-tree.
In that case I think it makes sense to require an installed Ruby to run more integration-like tests.
Updated by vo.x (Vit Ondruch) about 7 years ago
The upstream version of minitest 4.7.5 supported spec syntax. It does not probably support all the RSpec features, but might be worth of considering ...
Updated by hsbt (Hiroshi SHIBATA) about 7 years ago
I added test-bundler
task to my experimental branch.
https://github.com/ruby/ruby/pull/1536/commits/5aa6d9a4340f8e6a54c9411e56c835c7f026abe3
It install rspec under the $(srcdir)/.bundle directory and use it by bundler test.
But many of examples still failed. I'm going to investigate and fix it.
Updated by indirect (André Arko) about 7 years ago
I added test-bundler task to my experimental branch.
Thank you for your work on this. Please contact me in the Bundler slack if you have any questions or would like any help with this. I am happy to help fix the tests.
Updated by hsbt (Hiroshi SHIBATA) about 7 years ago
I've added make test-bundler
task to the ruby core.
see https://github.com/ruby/ruby/pull/1536
Maybe, I tested this patch with macOS only. Probably the following task is not compatible all of the supported platform.
https://github.com/ruby/ruby/pull/1536/files#diff-7e056ddc530791272d363e50c1f25647R488
I found problems with bundler test suite before make install
environment.
- Tests of
bundle install
with native extensions relied on installed ruby interpreter. - Tests of “default gems” are failed. Because they are not installed before
make install
- Tests of “bundler exec” with gem installed Gemfile relied on installed ruby interpreter.(ex. shebang of rack command use it)
- bundler tests require “groff” command for man verification. ruby core could not prepare it.
- Some tests depend on the directory structure of bundler/bundler.
I omitted or added workaround to theirs.
https://github.com/hsbt/ruby/blob/e16f39690e2e7843690f691e9f68e3371aba96f3/spec/bundler.patch
I have a happy news. make test-bundler
results is here
2269 examples, 0 failures, 3 pending
I ignored 40 examples. but I got green results of most all of examples.
Updated by shevegen (Robert A. Heiler) about 7 years ago
I think that this is a pretty big undertaking, irrelevant of pros or cons
to it (which surely exist).
Bundler is popular, there is no doubt about this even though I myself do not
use it - at https://rubygems.org/stats bundler is ranked #1 by a fair margin
(rank #2 is multi_json and rank #3 is rake).
Since I myself am not using bundler, I feel it would be unjust to comment
on it pro or con, others can do so. But may I suggest something else,
actually - gemify the remaining stdlib parts first before prioritizing
on bundler integration into rubygems 3.x, which then will also be
distributed with the next ruby releases?
I am not sure what is missing aside from fileutils and strscan, if this
is up to date (I am not sure, sorry; I thought there may be more but
I can not find it right now - see the roadmap for 2.5 ... have there
been any ruby core meetings in 2017? Probably have not been added to
2.5 or perhaps the meetings discussed past 2.5 situation).
Updated by matz (Yukihiro Matsumoto) almost 7 years ago
It's OK as long as bundling bundler makes users happy (and I believe it would).
Matz.
Updated by hsbt (Hiroshi SHIBATA) almost 7 years ago
I update my working branch.
https://github.com/ruby/ruby/pull/1536
Changes in this branch:
Directory Structure¶
I organized bundler integration like this:
root -+- lib -+- bundler/*
| |
| +- bundler.gemspec
| |
| +- rubygems/*
| |
|
|
+- spec -+- bundler/*
|
+- rubyspec
|
Differences from bundler repo are gemspec and rspec examples locations.
I'm going to send a pull request adopting Ruby core directory structure to bundler upstream.
Exclude some rspec examples for Ruby core.¶
Some of the bundler examples need to installed Ruby interpreter. But Ruby core test suite doesn't
use installed ruby interpreter. Therefore, It is necessary to exclude examples that require the
installed ruby interpreter.
I also send a pull request these exclusions used rspec filter condition to bundler repo.
Finally, We need to show results of "test-bundler" on Ruby CI (https://rubyci.org).
I will make patches for bundler integration to ruby/rubyci and ruby/chkbuild.
Updated by vo.x (Vit Ondruch) almost 7 years ago
I don't understand it. Why is it not simple bundled gem similartly to minitest, test-unit, rake, etc? This gems broken to Ruby directories were always the worst idea. It could make sense when Ruby have not shipped with Rubygems, but nowadays?
Updated by shyouhei (Shyouhei Urabe) almost 7 years ago
vo.x (Vit Ondruch) wrote:
I don't understand it. Why is it not simple bundled gem similartly to minitest, test-unit, rake, etc? This gems broken to Ruby directories were always the worst idea. It could make sense when Ruby have not shipped with Rubygems, but nowadays?
Rubygems itself is going to depend on bundler. It must exist beforehand. Bundled gem won't work.
Updated by hsbt (Hiroshi SHIBATA) almost 7 years ago
- Status changed from Assigned to Closed
Applied in changeset trunk|r59779.
Merge bundler to standard libraries.
rubygems 2.7.x depends bundler-1.15.x. This is preparation for
rubygems and bundler migration.
- lib/bundler.rb, lib/bundler/*: files of bundler-1.15.4
- spec/bundler/*: rspec examples of bundler-1.15.4. I applied patches.
- https://github.com/bundler/bundler/pull/6007
- Exclude not working examples on ruby repository.
- Fake ruby interpriter instead of installed ruby.
- Makefile.in: Added test task named
test-bundler
. This task is only
working macOS/linux yet. I'm going to support Windows environment later. - tool/sync_default_gems.rb: Added sync task for bundler.
[Feature #12733][ruby-core:77172]
Updated by hsbt (Hiroshi SHIBATA) over 6 years ago
- Status changed from Closed to Assigned
I postpone merging bundler to stdlib.
I reverted this feature ar r61416
Updated by naruse (Yui NARUSE) over 6 years ago
- Target version changed from 2.5 to 2.6
Updated by hsbt (Hiroshi SHIBATA) over 6 years ago
- Related to Bug #14219: package size of Ruby 2.5 is larger than 2.4 added
Updated by hsbt (Hiroshi SHIBATA) almost 6 years ago
- Status changed from Assigned to Closed
Applied in changeset trunk|r65509.
Added bundler as default gems. Revisit [Feature #12733]
- bin/, lib/bundler/, lib/bundler.rb, spec/bundler, man/*:
Merge from latest stable branch of bundler/bundler repository and
added workaround patches. I will backport them into upstream. - common.mk, defs/gmake.mk: Added
test-bundler
task for test suite
of bundler. - tool/sync_default_gems.rb: Added sync task for bundler.