Project

General

Profile

Misc #15275

[DOCs] Documentation for ruby's jit model - from the ruby "end user" perspective

Added by shevegen (Robert A. Heiler) 11 months ago. Updated 9 months ago.

Status:
Closed
Priority:
Normal
[ruby-core:89675]

Description

This is a request about documentation, with the specific focus on the new ruby
jit.

The "end user" perspective here means how an average ruby user may look at the
jit; say, someone who may have been using ruby for some time, but may not be a
"full" expert in ruby, just somewhat semi-advanced. A competent (in ruby),
average person.

The aim of this feature request here is to, at the least, document how such a user
may benefit from jit in his or her projects. I will explain more about this lateron,
but since this request is a bit long, I have split it up into several subcomponents.

Up to this point the aim was mostly to provide a short explanation to this feature
request, as introduction - the "TL;DR" variant.


There are quite some documents, blog entries and presentations available in regards
to ruby's jit. In particular matz spoke about this several times before; but also
Takashi and Vlad spoke about the jit.

For example, in one video here, for those who may be curious, matz mentioned the
jit (2018):

https://www.youtube.com/watch?v=jqnVaYUcjCo

(If you are only interested in the jit part, you can skip forward towards e. g.
~10:30, or something like that.)

There are also documents and interviews about the jit-effort in ruby, such as
here:

https://blog.heroku.com/ruby-mjit

But also elsewhere. Anyway; my focus here is on the "end user" perspective,
of regular ruby users. How/if they can benefit from the jit. I assume that
this also depends on the projects at hand.

Matz said that ruby 2.6.x will have jit, but this may not yet be the "fastest"
jit; there will be more improvements in the future. I guess until ruby 3.x we
can expect further changes here.

Matz also mentioned that not every program may benefit from the jit equally;
for example, I/O heavy-tasks may not benefit much at all from the jit. Some
projects may not run faster - and perhaps even run slower. Here I think it may
be important that ruby users understand the trade-offs, advantages and
disadvantages. That way they can make informed decisions in the future, in
regards to their projects and the jit.

For me, as an end user, the main question is - which of my projects may
benefit from the jit? How can I verify this or measure it? Should I use the
jit for this project, but not that project? (Matz also gave another talk
about memory usage and resource-constrained systems; I think there will be
a large difference between classical desktop systems with a lot of RAM,
and smaller smartphone-systems and so forth).

That is my primary reason for this request here, to provide a document
and/or tutorial of some kind.

I should also mention that, obviously, the code itself for the jit has
priority compared to the documentation, but documentation is also very
important.

Back in the ruby 1.8.x to ruby 2.x days, there was not a lot of information
available in regards to how to handle different Encoding situations. For
example, it was new that String objects had an encoding attached. That also
meant that String objects could have different encodings, and merging two
String objects with different encoding led to problems (which could not
happen in the 1.8.x days; or at the least did not happen as frequently).

It took me a while to learn and understand how to handle the new Encoding
situation, which is also a reason why I am creating the issue request here,
so that we can work towards documenting the jit from the end user perspective
as well, in order to make it easier to work and use the jit. But, as said
before, the code for the jit is more important in the sense that ... without
that code, the documentation would not be useful. :)

This suggestion is also not about draining too much time from Takashi Kokubun
and others in this regard. It is ok if documentation comes at a later time,
say, somewhere in the year 2019 perhaps? At some point, though, it would be
very useful for explaining to regular end users how to use jit, when not to
use the jit, what can be expected in regards to speed improvements - things
like that. I am sure there will be many blog entries about this in the future,
but to me personally, I think it would be better if we learn about such changes
from the official documentation rather than "random" blog entries (even though
many such blog entries are very good, don't get me wrong).

With proper documentation, ruby users can understand the jit, as in how to use
it.

I will next list some items that may be useful to have, but feel free to
ignore them if you feel they are not that important to have. I will number
them and list them, roughly, in order of importance, e. g. (1) is the
most important suggestion, whereas (5) is not so important.

Hopefully I am not the only one who would appreciate useful documentation
in this regard.

The documentation could be kept in a .md file distributed with the ruby
tarball itself; and/or it could reside at https://ruby-doc.org/; and/or
at the ruby wiki.

But I think where the documentation for jit resides is not so important,
as long as there is any place where it may be documented. When I write
"documentation", I also specifically mean a "tutorial" or rather a document,
with specific, different usage examples for the jit, too. There is quite
some information available, spread out over github issue trackers (including
some benchmarks), and blog entries (some in japanese, some in english) - I
would like to see a single resource where useful information could be
collected, and updated. This does not have to be long; but it should be
useful. It should have an english version, too, but it could be in japanese
first, and then translated into english (the examples will be in english
anyway, and may be the most important part altogether).


Suggestions:

(1) As a ruby end user, I would like to see specific usage examples in
how to use jit, both commandline-use but perhaps also web-use for jit (e. g.
be this rails and/or sinatra and/or padrino, just to see different examples;
perhaps even via shebang-flags within the .rb files, similar to frozen-string
literal, but this may require a separate suggestion; my issue here is really
primarily about documentation for jit). A few examples here may be useful
to provide ruby users with a start.

(2) As a ruby end user, I would appreciate some information AND benchmarks
in regard to projects when it comes to the jit. This should also show some
information in regards to time spent, as a benchmark; a bit similar to optcarrot.

For example:

      | default time without JIT | time with JIT | gain

project A | 100% | 98% | +2% faster
project B | 100% | 88% | +12% faster
etc..

This is just a suggestion. The aim here is to give people a short overview
what they may expect. For project A, project B etc... I may suggest
commonly used programs in ruby or example .rb files. These could be
distributed within ruby itself (perhaps test-files, or optcarrot-components),
or they could point to .rb files stored somewhere else, be it on github
or anywhere else. The key idea is to just give some projects as examples;
and ideally, make a fair comparison, so people can understand which
projects may benefit more, and which may benefit less.

I don't mean to suggest too many benchmarks here, but perhaps at the least
3 different examples, of a "small" project, a "medium" project, and a
"large" project in ruby. And if it is not too much work, perhaps 5 projects,
or even up to 8 or 10 - but I think 3-5 may suffice if it is somewhat
representative.

The projects do not necessarily have to be that large, but perhaps picking
some projects that require a few other projects would be realistic.

Some popular ruby projects could be picked, where comparison may make sense.
"rack" may be a useful candidate here or perhaps a project that makes use
of rack; perhaps a few more, such as "prawn", and similar projects.

The aim here is to provide ruby users with an overview that is somewhat
accurate. (Since the jit optimizations may become better and faster in the
future, I would also like to suggest to keep an entry in that documentat/file
that states when the document was last updated.)

(3) As matz mentioned before that not all applications may benefit from the
jit equally much, I would like to suggest to keep a short entry/paragraph
that shows which applications may not benefit as much from jit too. Again,
this does not have to be a long list; just a few examples, ideally with a
benchmark. (These benchmarks could ideally also be automatically run.)

(4) After this basic information is made available, it may also be useful
to explain a bit how the jit works (no need for any large technical details
there; just a rough picture or overview how jit works), where it can optimize
more code than in other ruby code; and perhaps also how to write "jit-friendly"
code.

With "jit-friendly code", I do NOT mean that people write ruby code in order
to maximize what the jit can do; but I assume that some code can be optimized
more easily than other code, and it may be useful if people know these
differences.

To explain this, I can provide an an example here:

I remember from the old pickaxe book, the difference between += and << for ruby
Strings, with the latter being faster than += as it avoids creating/allocating
new Strings. So I think this is a good recommendation for ruby Strings, to use
<< if you want to append something, rather than +=.

Something similar in regards to jit may also be useful to have. Again, this does
not have to be long; perhaps a paragraph or two.

(5) Links elsewhere should then be updated to refer to this jit-document too;
the wiki here on bugs.ruby-lang.org should also have a link to this jit document
perhaps; so should the official ruby homepage do, too - perhaps under "Documentation".

Perhaps also a file that is distributed with ruby itself can point to this jit
document (or such a jit-file can also be distributed with ruby, but these are mostly
details - in my opinion, the most important part is that there is one location where
information about the jit is collected in an official way).


I hope I could explain what I meant to say about the jit.

Please feel free to close this issue request here at any moment in time, for any
reason; since the path to JIT may change up to ruby 3.x., I suggest to close this
issue here at the very latest when ruby 3.x is released, or perhaps even earlier.

The 5 points are not all equally important; the two points (1) and (2) are the most
important parts. The jit-code is more important than the documentation and should
receive priority, but it would be nice if a bit of documentation could be added in
the future about the jit from the "end user" perspective. This does not have to
happn soon; think about this in the next ~18 months or so. I hope that this may
be a realistic time frame for this.

Thank you for reading this suggestion!

History

Updated by k0kubun (Takashi Kokubun) 11 months ago

  • Assignee set to k0kubun (Takashi Kokubun)

OK. Preview3's release notes is not gonna be changed so much, but I'll revisit that from user's perspective at preview4, rc1 or the actual release.

Previous releases didn't include that just because it had been a super early stage and we expected a lot of changes. Anyway thanks for the suggestion.

Updated by k0kubun (Takashi Kokubun) 9 months ago

  • Status changed from Open to Feedback

I'm struggling to remember the contents of this ticket in my small memory, so I'll summarize what I interpreted here:

terminology

  • end user: semi-advanced ruby user

missing documentation

  • how such a "end user" may benefit from jit
  • trace-offs to know when not to use the jit
  • how to measure JIT's performance in user's project

suggestions

1 and 2 are important, but less important than improving JIT implementation itself.

(1) command-line and web-use examples
(2) benchmark results for at least 3 projects for comparison
(3) explain which applications may not benefit from JIT
(4) explain what kind of code could be JIT-friendly
(5) link to the documentation from elsewhere

Updated by k0kubun (Takashi Kokubun) 9 months ago

  • Status changed from Feedback to Closed

In https://bugs.ruby-lang.org/projects/ruby/wiki/MJIT, I finished the following parts:

(1) Done.
(5) Linked from https://bugs.ruby-lang.org/projects/ruby/wiki. Also I'll add a link to it in the final release note.

Closing this ticket while (2), (3), (4) are still missing, but we can improve it later from this first step as you said. Thanks.

Updated by k0kubun (Takashi Kokubun) 9 months ago

For the initial version of (3) and (4), I added a few Advantages/Disadvantages explanations on that page.

Updated by k0kubun (Takashi Kokubun) 9 months ago

I know these explanations are obviously not end-user friendly yet. To be improved.

Also available in: Atom PDF