Project

General

Profile

Actions

Misc #17050

closed

profiler gem

Added by ioquatix (Samuel Williams) over 3 years ago. Updated over 3 years ago.


Description

I would like to reuse profiler gem namespace:

https://rubygems.org/gems/profiler/versions

It currently conflicts with profile/lib/profiler.rb however this is no longer part of stdlib.

cc @ko1 (Koichi Sasada)

Updated by ioquatix (Samuel Williams) over 3 years ago

Also I'd be happy to receive/resuse the profile namespace too but either one is fine and the profile gem actually has an interesting implementation.

Updated by duerst (Martin Dürst) over 3 years ago

Can you tell us shortly what you plan to use that gem/namespace for? If it's something that can (in some reasonable way) replace the (former?) profile/profiler functionality, then please go for it. At one point, I was planning to work on this, but I didn't find the time.

Updated by ioquatix (Samuel Williams) over 3 years ago

My initial goal is to expose some general data structures for efficiently capturing large memory profiles on top of ObjectSpace and GC modules, similar to how memory_profiler gem works. I don't mind making separate gem, but it seems like the profile or profiler gem would be ideal and have not been updated for a long time. Initially I was thinking of making separate gems and trying to provide a unified interface, e.g.

  • memory_profiler -> profiler-memory
  • stackprof -> profiler-stack
  • ruby-prof -> profiler-sample

etc.

I think that's too ambitious initially, but yeah, that's a longer term idea, with an initial focus on profiler-memory of which I've already got working code and just looking how to structure it best for longer term integration.

A sideways step would be to provide rspec-memory hooks for using these profilers either for general summary or assertions around blocks of code (of which I already maintain the rspec-memory gem which does some parts of this).

@ko1 (Koichi Sasada) said he wants to use profile gem, but I'm not sure if he also intended to use profiler gem. So, @ko1 (Koichi Sasada), can I use one of them, and you can use the other?

Updated by Eregon (Benoit Daloze) over 3 years ago

I'd recommend starting with something specific like profiler-memory.

profiler is very generic but therefore also unclear and will make people think it's a stdlib-like profiler, but I guess it will not be.

I also think it's too early to tell if your gem will be "the profiler everyone uses".

BTW, stackprof is a CPU sampling profiler, and ruby-prof seems a tracing CPU profiler.

As a side note it'd be nice if profiler-like gems work on more than 1 implementation.
I think most them only work on CRuby, because they use C APIs that are efficient on CRuby but basically inefficient on other implementations (see https://github.com/oracle/truffleruby/issues/2044#issuecomment-654848324 for details regarding stackprof & TruffleRuby).

Updated by ioquatix (Samuel Williams) over 3 years ago

I agree all your points. I already have working profiler-memory to a certain extent.

Updated by Eregon (Benoit Daloze) over 3 years ago

In the profile stdlib, both files are used: -rprofile starts the profiler immediately, while profiler.rb is like a library:
https://github.com/ruby/ruby/blob/ruby_2_6/lib/profile.rb

So it seems to me not advisable to reuse those "require names" for something not strictly compatible since profile was only removed in 2.7.

Updated by byroot (Jean Boussier) over 3 years ago

@ioquatrix I’m curious. What’s the difference between your memory-profiler and the existing memory_profiler?

For context I’ve been working on https://github.com/shopify/heap-profiler for the last week, and I’m also very interested in improving ObjectSpace and GC APIs

Updated by ioquatix (Samuel Williams) over 3 years ago

I was able to profile 200+ million* allocations with my fork which was impossible with the current implementations. Right now my current plan is to upstream these changes to memory_profiler. (*sorry, incorrectly said billions previously :)

Actions #9

Updated by jeremyevans0 (Jeremy Evans) over 3 years ago

  • Tracker changed from Bug to Misc
  • Backport deleted (2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN)

Updated by ko1 (Koichi Sasada) over 3 years ago

  • Status changed from Open to Feedback

Sorry for late response.

I have a plan to remake current profile library with recent APIs.

Updated by ioquatix (Samuel Williams) over 3 years ago

  • Status changed from Feedback to Closed

Okay, I look forward to your profiler.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0