Feature #11817
open
map.parallel
Description
I'd like to suggest a new method: #parallel
That method should take care of executing workloads in parallel across multiple cores.
I believe that #map can sometimes be distributed to many simultaneous threads.
So we would spread out the map blocks into parallel processing,
and then collect the results in order.
I believe this would need some new infrastructure like apple did with great_central,
so I'm talking along these lines.
(Aka we only want to open up as many threads as there are cores)
This would not help much with a simple map (because we need to collect and serialize the results in the end)
but it could be cool for more complex mappings.
Maybe this should have been a rails topic first,
because there it might make a lot of sense to render partials in parallel,
this could really speed up rails response rendering ;-)
But I believe this is a topic for ruby in general.
Given that we now go to really many cores,
it should be at the heart of ruby to use that.
For a start we could just declare #parallel as -> return self.
and than improve on it.
A naive implementation would just open a new thread for every parallel map block,
but that would not work well -- it actually would be very slow.
So we actually need a thread management,
along the lines of great central.
I believe this should be built into the very core libs of the ruby language.
I know this is hard stuff, but it needs to be done.
Let's go parallel from the heart of the language.
Please let us discuss if adding the #parallel method would be a good idea.
Let's come up with some implementations to find the problems.
I'm explicitly referring to map#parallel here.
In rails that might be cache.parallel (if you followed)
for some math tasks it might also be
parallel do {}
Updated by