Project

General

Profile

Actions

Bug #14471

closed

DRb mixing up function return values between PIDs after fork()

Added by jrafanie (Joe Rafaniello) over 6 years ago. Updated about 5 years ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-darwin16]
[ruby-core:85524]

Description

This is likely a duplicate of https://bugs.ruby-lang.org/issues/2718, although the Backport191 project in redmine appears to be dead.

We have found the exact same problem with modern rubies such as 2.3, 2.4, 2.5.

The summary is that due to fork and sharing of established connections in the DRbConn pool, the DRb server sometimes sends DRb client 1's message to client 2 and client 2's message to client 1.

My colleague Adam Grare and I have narrowed it down to the "global" DRb::DRbConn class instance variable @pool being inherited in child forked processes.

Our workaround[1] was to close all existing connections in the inherited pool in each fork to ensure each child fork process would get brand new connections:

DRb::DRbConn.instance_variable_get(:@mutex).synchronize do
  DRb::DRbConn.instance_variable_get(:@pool).each(&:close)
end

The attached client and server and log demonstrates the problem and fails on all modern ruby versions. The source of these can be found in a gist here: [2]. Note, we have tested this on Mac OSX and debian.

It would be great if connection pools would not be shared in forked processes or give us a public interface to clear the connection pool.

[1] https://github.com/ManageIQ/manageiq/pull/16953
[2] https://gist.github.com/agrare/d9484884bd297b1615814128129cfc5c


Files

echo_server.rb (134 Bytes) echo_server.rb jrafanie (Joe Rafaniello), 02/13/2018 05:06 PM
echo_client.rb (2.22 KB) echo_client.rb jrafanie (Joe Rafaniello), 02/13/2018 05:06 PM
Example_Run.log (615 Bytes) Example_Run.log jrafanie (Joe Rafaniello), 02/13/2018 05:06 PM

Related issues 1 (0 open1 closed)

Is duplicate of Ruby master - Bug #2718: DRb mixing up function return values between PIDs after fork()Closedseki (Masatoshi Seki)Actions
Actions #1

Updated by jeremyevans0 (Jeremy Evans) about 5 years ago

  • Is duplicate of Bug #2718: DRb mixing up function return values between PIDs after fork() added
Actions #2

Updated by jeremyevans (Jeremy Evans) about 5 years ago

  • Status changed from Open to Closed

Applied in changeset git|d0ed935d5bf8c3fce9800742a36e44fb7f63dda4.


Fix some DRb issues (#2552)

  • Handle BasicObject in drb

Also fix a bug in rescue clause of any_to_s because sprintf
does not handle the %l modifier.

Fixes [Bug #7833]

  • Do not send a reply to the client if there is a connection error

This allows for normal TCP shutdown (fin-ack-fin-ack instead of
fin-ack-push-rst).

Patch from (Pierre-Alexandre Meyer).

Fixes [Bug #2339]

  • Detect fork and do not reuse forked connections in drb

This associates each DRbConn with a pid, and if the pid changes,
it closes any DRbConns in the pool with a pid that no longer
matches. This fixes DRb servers from sending messages intended
for one client to another client after forking.

Fixes [Bug #2718]
Fixes [Bug #14471]

Actions

Also available in: Atom PDF

Like0
Like0Like0