Feature #11517: Queue enhancement - conditional pop

• Status: Assigned
• Assignee: matz (Yukihiro Matsumoto)
• Target version: -

[jonathanscruz (Jonathan Cruz)]

I’m submitting a patch to enhance Queue#pop. This allows the caller to provide a block that accepts data from the queue. Queue#pop will return the first element for which the block returns a truthy value, and remove it from the queue. Without a block, Queue#pop will behave the same way it currently does.

The motivation for this enhancement: On our project, we have a queue of work and several worker threads. Some work can incur a heavy load on the system and should not be processed while another worker is processing 'heavy load' work. We need a way for Queue#pop to skip over heavy load items while another thread is processing heavy load work.

[funny_falcon (Yura Sokolov)]

Why don't you use separate queue with separate worker? It will solve your use case.

Even if it is hard to separate jobs before put into main queue, workers could push such heavy tasks into separate queue after fetching it from main queue.

[jonathanscruz (Jonathan Cruz)]

Yura Sokolov wrote:

Why don't you use separate queue with separate worker? It will solve your use case.

Even if it is hard to separate jobs before put into main queue, workers could push such heavy tasks into separate queue after fetching it from main queue.

That is an excellent suggestion. It might complicate the implementation, but it's worth thinking about. We have a configurable number of worker threads running and it would be tricky to honor that while having a single worker thread dedicated to watching the 'heavy work' queue.

This really is just a suggestion for convenience. We already have a work around by reimplementing the Queue class in ruby and subclassing it with this new version of #pop (as well as #promote! and #promote_all! as discussed in https://bugs.ruby-lang.org/issues/11518). We just thought others might find it helpful. Since it is no longer implemented in ruby, there is no way to access the underlying array (even from a subclass) and there is no easy way to selectively pop data off the queue or otherwise inspect the queue.

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned