Project

General

Profile

Bug #2681

some behavior changes of lib/csv.rb between 1.8 and 1.9

Added by mame (Yusuke Endoh) over 10 years ago. Updated about 9 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 1.9.2dev (2010-01-28 trunk 26458) [i686-linux]
Backport:
[ruby-core:27930]

Description

=begin
Hi jeg2, or anyone who knows the implementation of FasterCSV,

I'm now checking for ruby trunk to pass rubyspec. Are these behavior
changes of lib/csv.rb intended or bug?

1) CSV.generate

$ ruby18 -rcsv -e 'w = CSV.generate("foo18.csv"); w << [1, 2, 3]; w.close'
$ cat foo18.csv
1,2,3

$ ruby19 -rcsv -e 'w = CSV.generate("foo19.csv"); w << [1, 2, 3]; w.close'
/home/mame/work/ruby19/local/lib/ruby/1.9.1/csv.rb:1231:in
generate': no block given (yield) (LocalJumpError)
from -e:1:in
'

There is the comment of csv.rb:

The old CSV's Reader and Writer classes have been dropped.
CSV::generate() is different from the old method.
They mean this change, don't they?

2) CSV.generate_line

$ ruby18 -rcsv -e 'p CSV.generate_line([])'
""

$ ruby19 -rcsv -e 'p CSV.generate_line([])'
"\n"

3) CSV.generate_line (2)

$ ruby18 -rcsv -e 'p CSV::generate_line(["foo", "bar"], ?;)'
"foo;bar"

$ ruby19 -rcsv -e 'p CSV::generate_line(["foo", "bar"], ?;)'
/home/mame/work/ruby19/local/lib/ruby/1.9.1/csv.rb:1249:in merge':
can't convert String into Hash (TypeError)
from /home/mame/work/ruby19/local/lib/ruby/1.9.1/csv.rb:1249:in
generate_line'
from -e:1:in `'

3) CSV.parse

$ ruby18 -rcsv -ve 'p CSV.parse "\nfoo"'
nil], ["foo"

$ ruby19 -rcsv -ve 'p CSV.parse "\nfoo"'
], ["foo"

4) CSV.parse_line

$ ruby18 -rcsv -ve 'p CSV.parse_line ""'
[nil]

$ ruby19 -rcsv -ve 'p CSV.parse_line ""'
nil

5) CSV.parse_line (2)

$ ruby18 -rcsv -ve 'p CSV.parse_line "\nfoo"'
[nil]

$ ruby19 -rcsv -ve 'p CSV.parse_line "\nfoo"'
[]

Thanks,

--
Yusuke ENDOH mame@tsg.ne.jp
=end

#1

Updated by mame (Yusuke Endoh) over 10 years ago

  • Status changed from Open to Rejected
  • ruby -v set to ruby 1.9.2dev (2010-01-28 trunk 26458) [i686-linux]

=begin
Thank you for so quick response!
I'll work tomorrow.

--
Yusuke Endoh mame@tsg.ne.jp
=end

#2

Updated by mame (Yusuke Endoh) over 10 years ago

=begin
Hi,

2010/1/31 James Edward Gray II james@graysoftinc.com:

On Jan 28, 2010, at 11:13 AM, James Edward Gray II wrote:

On Jan 28, 2010, at 10:51 AM, Yusuke ENDOH wrote:

4) CSV.parse_line

$ ruby18 -rcsv -ve 'p CSV.parse_line ""'
[nil]

$ ruby19 -rcsv -ve 'p CSV.parse_line ""'
nil

We could probably debate this one. It may be that FasterCSV should return a [] instead of nil. I'll make a not to look into that and maybe fix it. We won't be going back to the CSV response of [nil] though, since I don't understand it. (See my answer to #3.)

I take back what I said here. FasterCSV has the convention of returning nil to signal the end of data, as an IO object would. Thus, this change is intended and should probably not be changed (because FasterCSV would then be inconsistent with its own rules).

Yessir.

I have changed csv of rubyspec, though there is still no spec of
CSV.parse_line.
It would be helphul if you could take a look at csv of rubyspec in
your spare time:

http://github.com/rubyspec/rubyspec/tree/master/library/csv/

--
Yusuke ENDOH mame@tsg.ne.jp

=end

#3

Updated by nahi (Hiroshi Nakamura) over 10 years ago

=begin
Hi jeg2,

On Fri, Jan 29, 2010 at 02:13, James Edward Gray II
james@graysoftinc.com wrote:

3) CSV.parse

$ ruby18 -rcsv -ve 'p CSV.parse "\nfoo"'
nil], ["foo"

$ ruby19 -rcsv -ve 'p CSV.parse "\nfoo"'
], ["foo"

Yeah, the old CSV included some seemingly random nils in places.  FasterCSV "corrects" this odd behavior.

Please let me know what you think 'random'.

0% ruby18 -rcsv -e 'p ["", ",", ",,"].map { |c| CSV.parse_line(c) }'
nil], [nil, nil], [nil, nil, nil
0% ruby19 -rcsv -e 'p ["", ",", ",,"].map { |c| CSV.parse_line(c) }'
[nil, [nil, nil], [nil, nil, nil]]

I don't think 1.9's csv follow this behavior for consistency though.

I don't have any changes fastercsv made. Rubyspec should follow fastercsv.

Regards,
// NaHi

=end

#4

Updated by nahi (Hiroshi Nakamura) over 10 years ago

=begin
Hi again,

On Mon, Feb 1, 2010 at 21:35, NAKAMURA, Hiroshi nakahiro@gmail.com wrote:

I don't have any changes fastercsv made. Rubyspec should follow fastercsv.

I mean, I don't have any comment to the changes fastercsv made. No objection.

Regards,
// NaHi

=end

#5

Updated by mame (Yusuke Endoh) over 10 years ago

=begin
Hi,

2010/2/1 James Edward Gray II james@graysoftinc.com:

I have changed csv of rubyspec, though there is still no spec of
CSV.parse_line.
It would be helphul if you could take a look at csv of rubyspec in
your spare time:

http://github.com/rubyspec/rubyspec/tree/master/library/csv/

I browsed through it a bit. I didn't feel like there is much there. :)

I say that meaning that CSV has a lot of tests in Ruby itself. Does RubySpec not make an effort to port existing test suites?

I was asked similar question by Masatoshi Seki (the author of ERB)
in [ruby-dev:40239]. I also want to ask RubySpec expert.

Here is just my personal opinion: RubySpec is (at least, aim to be)
"spec", not test suite.

The library "spec" can be referred to know the standard, strict and
guaranteed behavior of library. In the sense, it is similar to
document, rather than test suites.
"The code is the spec" philosophy is too cumbersome for the purpose.
It is difficult to know what behavior is guaranteed. And, some
optimization and refactoring may lead to spec change easily.

In addition, test suites may include "white-box test," which is
involved with the specific implementation.
For example, when yet another CSV library appears (like FasterCSV
for old CSV) and aims to be strictly compatibile to lib/csv.rb,
the test suites may be too implementation-specific to be used as
comformance test.

Well, in [ruby-core:27930], to be exact, I had to ask you "are these
new behaviors guaranteed (at least in 1.9 series)?"

--
Yusuke ENDOH mame@tsg.ne.jp

=end

#6

Updated by vvs (Vladimir Sizikov) over 10 years ago

=begin
Hi,

On Tue, Feb 2, 2010 at 1:17 PM, Yusuke ENDOH mame@tsg.ne.jp wrote:

2010/2/1 James Edward Gray II james@graysoftinc.com:

I say that meaning that CSV has a lot of tests in Ruby itself.  Does RubySpec not make an effort to port existing test suites?

I'd say that a lot of tests in Ruby are very good candidates for
porting to RubySpec. But the devil is always in details: which tests
are good RubySpec ones and which are not.

Here is just my personal opinion:  RubySpec is (at least, aim to be)
"spec", not test suite.

The library "spec" can be referred to know the standard, strict and
guaranteed behavior of library.  In the sense, it is similar to
document, rather than test suites.
"The code is the spec" philosophy is too cumbersome for the purpose.
It is difficult to know what behavior is guaranteed.  And, some
optimization and refactoring may lead to spec change easily.

I have a bit different view here.

Personally, I'd consider anything (reasonable) that any
external/public Ruby library or any other 3rd party Ruby code expects
from the Ruby implementation to be a RubySpec worthy material.
Otherwise, those libs/application could be working differently on
different implementations, which is always not good. :)

At a minimum, Rubyspec should have tests for public API behavior, including:

  • Regular use cases and boundaries (ideally, with Equivalence Class Partitioning http://www.testing-world.com/58828/Equivalence-Class-Partitioning)
  • Invalid/exceptional use cases (how API/impl react to those invalid/exceptional situations)
  • All the assertions explicitly stated in the public/official documentation (all testable assertions, I mean).

What are the not-so-good candidates for RubySpec:

  • white-box tests
  • tests for private API, and for internal implementation-details
  • most of regression tests (they are implementation specific, typically)
  • heavily platfrom-dependent or platfrom-specific behaviors (very hard to maintain such tests and keep the sanity)
  • performance/stress tests

For tests that are in Ruby repository, while there are lots and lots
of good tests that are very suitable for RubySpecs, they often mixed
among other ones which are not ideal candidates, so it could take
quite an effort to detect which ones belong to which category and to
port only them (and to check that RubySpec doesn't have similar tests
already). Big task.

In JRuby, we tend to encourage to write RubySpec tests first, and only
if there is a good reason why such test is not good for RubySpec, only
then we end up with jruby-specific tests. We also use various
3rd-party test suites, which we also trying to move to RubySpec, where
appropriate, to reduce the number of such 3rd-party suites and to
simplify test maintenance.

Thanks,
--Vladimir

In addition, test suites may include "white-box test," which is
involved with the specific implementation.
For example, when yet another CSV library appears (like FasterCSV
for old CSV) and aims to be strictly compatibile to lib/csv.rb,
the test suites may be too implementation-specific to be used as
comformance test.

Well, in [ruby-core:27930], to be exact, I had to ask you "are these
new behaviors guaranteed (at least in 1.9 series)?"

--
Yusuke ENDOH mame@tsg.ne.jp

=end

#7

Updated by brixen (Brian Shirai) over 10 years ago

=begin
Hi,

On Tue, Feb 2, 2010 at 5:18 AM, Vladimir Sizikov vsizikov@gmail.com wrote:

Hi,

On Tue, Feb 2, 2010 at 1:17 PM, Yusuke ENDOH mame@tsg.ne.jp wrote:

2010/2/1 James Edward Gray II james@graysoftinc.com:

I say that meaning that CSV has a lot of tests in Ruby itself.  Does RubySpec not make an effort to port existing test suites?

I'd say that a lot of tests in Ruby are very good candidates for
porting to RubySpec. But the devil is always in details: which tests
are good RubySpec ones and which are not.

Yes, we make an attempt to port good tests from other test suites
including MRI's tests. However, someone has to volunteer to do the
work.

Here is just my personal opinion:  RubySpec is (at least, aim to be)
"spec", not test suite.

The library "spec" can be referred to know the standard, strict and
guaranteed behavior of library.  In the sense, it is similar to
document, rather than test suites.
"The code is the spec" philosophy is too cumbersome for the purpose.
It is difficult to know what behavior is guaranteed.  And, some
optimization and refactoring may lead to spec change easily.

I have a bit different view here.

Personally, I'd consider anything (reasonable) that any
external/public Ruby library or any other 3rd party Ruby code expects
from the Ruby implementation to be a RubySpec worthy material.
Otherwise, those libs/application could be working differently on
different implementations, which is always not good. :)

At a minimum, Rubyspec should have tests for public API behavior, including:

  • Regular use cases and boundaries  (ideally, with Equivalence Class Partitioning http://www.testing-world.com/58828/Equivalence-Class-Partitioning)
  • Invalid/exceptional use cases (how API/impl react to those invalid/exceptional situations)
  • All the assertions explicitly stated in the public/official documentation (all testable assertions, I mean).

What are the not-so-good candidates for RubySpec:

  • white-box tests
  • tests for private API, and for internal implementation-details
  • most of regression tests (they are implementation specific, typically)
  • heavily platfrom-dependent or platfrom-specific behaviors (very hard to maintain such tests and keep the sanity)
  • performance/stress tests

Vladimir has explained this very well here. (Thanks Vladimir.)

There is an explanation of rubyspec on the wiki
http://rubyspec.org/wiki/rubyspec, especially under the Style Guide.
Please also see this thread on the rubyspec ML
http://groups.google.com/group/rubyspec/browse_frm/thread/36a082f4db5f155b#.
We can improve these explanations if you have specific questions.

Cheers,
Brian

For tests that are in Ruby repository, while there are lots and lots
of good tests that are very suitable for RubySpecs, they often mixed
among other ones which are not ideal candidates, so it could take
quite an effort to detect which ones belong to which category and to
port only them (and to check that RubySpec doesn't have similar tests
already). Big task.

In JRuby, we tend to encourage to write RubySpec tests first, and only
if there is a good reason why such test is not good for RubySpec, only
then we end up with jruby-specific tests. We also use various
3rd-party test suites, which we also trying to move to RubySpec, where
appropriate, to reduce the number of such 3rd-party suites and to
simplify test maintenance.

Thanks,
 --Vladimir

In addition, test suites may include "white-box test," which is
involved with the specific implementation.
For example, when yet another CSV library appears (like FasterCSV
for old CSV) and aims to be strictly compatibile to lib/csv.rb,
the test suites may be too implementation-specific to be used as
comformance test.

Well, in [ruby-core:27930], to be exact, I had to ask you "are these
new behaviors guaranteed (at least in 1.9 series)?"

--
Yusuke ENDOH mame@tsg.ne.jp

=end

Also available in: Atom PDF