Bug #8983: [PATCH] GC.stat[:heap_free_num] returns number of unused slots on heap

• Status: Closed
• Assignee: ko1 (Koichi Sasada)
• Target version: 2.1.0
• ruby -v: ruby 2.1.0dev (2013-10-03 trunk 43129) [x86_64-darwin12.4.1]
• Backport: 1.9.3: UNKNOWN, 2.0.0: UNKNOWN

[tmm1 (Aman Karmani)]

In ruby 1.9 and 2.0, gc_stat() used to call rest_sweep() so heap_free_num always equals the number of free slots (ObjectSpace.count_objects[:FREE]). In 2.1 the rest_sweep() was removed, so free_num is not always accurate. This patch fixes compatibility with the previous behavior.

With this patch, it is also easier to perform "out-of-band" GC in unicorn and passenger. For example: if your application requires 100k objects per request, then you can run the following in-between every request:

GC.start if GC.stat[:heap_free_num] < 100_000

I also added a objspace->heap.limit because HEAP_OBJ_LIMIT*heap_used is not accurate due to memory re-alignment in heap_assign_slot.

https://github.com/tmm1/ruby/compare/gc-stat-free-num
https://github.com/tmm1/ruby/compare/gc-stat-free-num.patch

[naruse (Yui NARUSE)]

• Status changed from Open to Assigned
• Assignee changed from tmm1 (Aman Karmani) to ko1 (Koichi Sasada)

[tmm1 (Aman Karmani)]

ko1-san, if you agree with this patch I can commit it.

[ko1 (Koichi Sasada)]

(2013/10/03 18:29), naruse (Yui NARUSE) wrote:

https://github.com/tmm1/ruby/compare/gc-stat-free-num.patch

One point.

@@ -1787,6 +1791,7 @@ struct os_each_struct {
else {
struct heap_slot *slot = (struct heap_slot *)(VALUE)RDATA(p)->dmark;
slot->limit--;

•    heap_limit--;
  

  }
  p = tmp;
  }

Should not reduce heap_limit here.

@@ -2343,6 +2348,7 @@ struct force_finalize_list {
RDATA(pp)->dmark = (void (*)(void ))(VALUE)sweep_slot;
pp->as.free.flags |= FL_SINGLETON; / freeing page mark */
}

•    heap_limit -= (freed_num + empty_num);
     sweep_slot->limit = final_num;
     unlink_heap_slot(objspace, sweep_slot);
  

  }

Should set "heap_limit -= slot->limit".
This slot is already unavaialble from Ruby level.

--
// SASADA Koichi at atdot dot net

[tmm1 (Aman Karmani)]

I understand the slot is removed, but objspace_live_num includes unfinalized objects so I included them in heap_limit also.

Otherwise, objspace->heap.limit - objspace_live_num(objspace) can become negative.

[ko1 (Koichi Sasada)]

(2013/10/04 10:09), tmm1 (Aman Gupta) wrote:

I understand the slot is removed, but objspace_live_num includes unfinalized objects so I included them in heap_limit also.

Otherwise, objspace->heap.limit - objspace_live_num(objspace) can become negative.

I got it.

objspace_live_num() should return the count including zombie (waiting to
finallization) because they are still living (living dead!!).

However, we can eliminate zombiees from live_num() because they are not
living normal objects.

Two approachs?

(1) eliminate zombies from live_num

(2) count zombie_num

(objspace->heap.limit -
(objspace_live_num(objspace) - zombie_num)

--
// SASADA Koichi at atdot dot net

[tmm1 (Aman Karmani)]

(2) count zombie_num

(objspace->heap.limit -
(objspace_live_num(objspace) - zombie_num)

I like this approach. This will be useful also for debugging finalizer leaks.

Once we replace heap_free_num with this formula, I would like to rename the existing heap.free_num counter. Do you have any suggestion for a better name? Maybe heap.reused_num or heap.swept_num

[tmm1 (Aman Karmani)]

Actually we already have heap.final_num. Is this the same as zombie_num?

[tmm1 (Aman Karmani)]

• Status changed from Assigned to Closed
• % Done changed from 0 to 100

This issue was solved with changeset r43138.
Aman, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

gc.c: add rb_objspace.limit

* gc.c: add rb_objspace.limit to keep accurate count of total heap
  slots [ruby-core:57633] [Bug #8983]