Bug #15358: Segfault in rb_transient_heap_mark when running Sequel's specs

• Status: Closed
• Assignee: ko1 (Koichi Sasada)
• Target version: 2.6
• ruby -v: ruby 2.6.0preview3 (2018-11-06 trunk 65578) [x86_64-openbsd]
• Backport: 2.4: UNKNOWN, 2.5: UNKNOWN

[jeremyevans0 (Jeremy Evans)]

When running Sequel's plugin specs, there is a segfault that is dependent on execution order. I have been seeing it on OpenBSD after upgrading from ruby-2.6.0-preview2 to ruby-2.6.0-preview3, and I was not sure if it was OpenBSD specific. It has now occurred on Travis running Linux (https://travis-ci.org/jeremyevans/sequel/jobs/460952288), so this appears to be a general issue and not specific to OpenBSD.

This error is not caused by a C extensions. The only C extension loaded in Sequel's plugin specs is nokogiri, and I have verified that this error occurs even if nokogiri is not loaded. Sequel's plugin specs pass on ruby 1.9.3 to 2.6.0-preview2, and JRuby 1.7 to 9.2, so this is unlikely to be a problem inside Sequel.

Here's the backtrace when running the specs inside gdb:

(gdb) bt
#0  0x0000088014bbd213 in rb_transient_heap_mark (obj=9347566859680, ptr=0x0) at transient_heap.c:525
#1  0x00000880149d0fec in mark_hash (objspace=0x8803bc38800, hash=9347566859680) at gc.c:4267
#2  0x00000880149c1131 in gc_mark_children (objspace=0x8803bc38800, obj=9347566859680) at gc.c:4758
#3  0x00000880149cb753 in gc_mark_stacked_objects (objspace=0x8803bc38800, incremental=0, count=0) at gc.c:4883
#4  0x00000880149cc165 in gc_mark_stacked_objects_all (objspace=0x8803bc38800) at gc.c:4923
#5  0x00000880149cae27 in gc_marks_rest (objspace=0x8803bc38800) at gc.c:5803
#6  0x00000880149cd0e4 in gc_marks (objspace=0x8803bc38800, full_mark=0) at gc.c:5863
#7  0x00000880149c8bd7 in gc_start (objspace=0x8803bc38800, reason=256) at gc.c:6665
#8  0x00000880149c8500 in heap_prepare (objspace=0x8803bc38800, heap=0x8803bc38820) at gc.c:1740
#9  0x00000880149c83c9 in heap_get_freeobj_from_next_freepage (objspace=0x8803bc38800, heap=0x8803bc38820) at gc.c:1752
#10 0x00000880149c828f in heap_get_freeobj (objspace=0x8803bc38800, heap=0x8803bc38820) at gc.c:1789
#11 0x00000880149c8158 in newobj_slowpath (klass=0, flags=24602, v1=0, v2=258601, v3=9346992938920, objspace=0x8803bc38800, wb_protected=1) at gc.c:1919
#12 0x00000880149c7ec9 in newobj_slowpath_wb_protected (klass=0, flags=24602, v1=0, v2=258601, v3=9346992938920, objspace=0x8803bc38800) at gc.c:1931
#13 0x00000880149b8446 in newobj_of (klass=0, flags=24602, v1=0, v2=258601, v3=9346992938920, wb_protected=1) at gc.c:1967
#14 0x00000880149b8618 in rb_imemo_new (type=imemo_ment, v1=0, v2=258601, v3=9346992938920, v0=0) at gc.c:2010
#15 0x0000088014bdf021 in rb_method_entry_alloc (called_id=258601, owner=9346992938920, defined_class=0, def=0x0) at vm_method.c:369
#16 0x0000088014bdef54 in rb_method_entry_create (called_id=258601, klass=9346992938920, visi=METHOD_VISI_PUBLIC, def=0x0) at vm_method.c:390
#17 0x0000088014bdff6a in rb_method_entry_make (klass=9346992938920, mid=258601, defined_class=9346992938920, visi=METHOD_VISI_PUBLIC, type=VM_METHOD_TYPE_ISEQ, def=0x0, original_id=258601, opts=0x7f7ffffd87d0) at vm_method.c:597
#18 0x0000088014bde656 in rb_add_method (klass=9346992938920, mid=258601, type=VM_METHOD_TYPE_ISEQ, opts=0x7f7ffffd87d0, visi=METHOD_VISI_PUBLIC) at vm_method.c:653
#19 0x0000088014be0460 in rb_add_method_iseq (klass=9346992938920, mid=258601, iseq=0x880443190a8, cref=0x88044319530, visi=METHOD_VISI_PUBLIC) at vm_method.c:672
#20 0x0000088014c0f8d1 in vm_define_method (obj=8, id=258601, iseqval=9346992935080, is_singleton=0) at vm.c:2609
#21 0x0000088014bf0453 in m_core_define_method (self=9347027151000, sym=66201868, iseqval=9346992935080) at vm.c:2629
#22 0x0000088014c0a64f in call_cfunc_2 (func=0x88014bf03d0 <m_core_define_method>, recv=9347027151000, argc=2, argv=0x88030e5c5f0) at vm_insnhelper.c:1758
#23 0x0000088014bf9fe7 in vm_call_cfunc_with_frame (ec=0x8801729e248, reg_cfp=0x88030f5b7b0, calling=0x7f7ffffda688, ci=0x880bfb35600, cc=0x880091ccc00) at vm_insnhelper.c:1905
#24 0x0000088014bf3337 in vm_call_cfunc (ec=0x8801729e248, reg_cfp=0x88030f5b7b0, calling=0x7f7ffffda688, ci=0x880bfb35600, cc=0x880091ccc00) at vm_insnhelper.c:1921
#25 0x0000088014bf272b in vm_call_method_each_type (ec=0x8801729e248, cfp=0x88030f5b7b0, calling=0x7f7ffffda688, ci=0x880bfb35600, cc=0x880091ccc00) at vm_insnhelper.c:2243
#26 0x0000088014bf246c in vm_call_method (ec=0x8801729e248, cfp=0x88030f5b7b0, calling=0x7f7ffffda688, ci=0x880bfb35600, cc=0x880091ccc00) at vm_insnhelper.c:2369
#27 0x0000088014bd31f2 in vm_call_general (ec=0x8801729e248, reg_cfp=0x88030f5b7b0, calling=0x7f7ffffda688, ci=0x880bfb35600, cc=0x880091ccc00) at vm_insnhelper.c:2412
#28 0x0000088014bd7b8d in vm_exec_core (ec=0x8801729e248, initial=0) at insns.def:766
#29 0x0000088014bed0dc in rb_vm_exec (ec=0x8801729e248, mjit_enable_p=1) at vm.c:1812
#30 0x0000088014be735d in eval_string_with_cref (self=9346992938920, src=9346992936280, cref=0x88044319530, file=9346746104800, line=784) at vm_eval.c:1320
#31 0x0000088014c0dcab in eval_under (under=9346992938920, self=9346992938920, src=9346992936280, file=9346746104800, line=784) at vm_eval.c:1599
#32 0x0000088014be8458 in specific_eval (argc=3, argv=0x88030e5c5a0, klass=9346992938920, self=9346992938920) at vm_eval.c:1623
#33 0x0000088014be87a4 in rb_mod_module_eval (argc=3, argv=0x88030e5c5a0, mod=9346992938920) at vm_eval.c:1738
#34 0x0000088014c0a537 in call_cfunc_m1 (func=0x88014be8770 <rb_mod_module_eval>, recv=9346992938920, argc=3, argv=0x88030e5c5a0) at vm_insnhelper.c:1740
#35 0x0000088014bf9fe7 in vm_call_cfunc_with_frame (ec=0x8801729e248, reg_cfp=0x88030f5b820, calling=0x7f7ffffdc5b8, ci=0x8804671cda0, cc=0x8807a738b90) at vm_insnhelper.c:1905
#36 0x0000088014bf3337 in vm_call_cfunc (ec=0x8801729e248, reg_cfp=0x88030f5b820, calling=0x7f7ffffdc5b8, ci=0x8804671cda0, cc=0x8807a738b90) at vm_insnhelper.c:1921
#37 0x0000088014bd7b8d in vm_exec_core (ec=0x8801729e248, initial=0) at insns.def:766
#38 0x0000088014bed0dc in rb_vm_exec (ec=0x8801729e248, mjit_enable_p=1) at vm.c:1812
#39 0x0000088014c0d4a2 in invoke_block (ec=0x8801729e248, iseq=0x8800b6b1ac8, self=9349739236600, captured=0x88030f5b8a8, cref=0x0, type=572653569, opt_pc=0) at vm.c:1007
#40 0x0000088014c0d28b in invoke_iseq_block_from_c (ec=0x8801729e248, captured=0x88030f5b8a8, self=9349739236600, argc=1, argv=0x7f7ffffdc968, passed_block_handler=0, cref=0x0, is_lambda=0) at vm.c:1059
#41 0x0000088014c0cf5b in invoke_block_from_c_bh (ec=0x8801729e248, block_handler=9346670246057, argc=1, argv=0x7f7ffffdc968, passed_block_handler=0, cref=0x0, is_lambda=0, force_blockarg=0) at vm.c:1077
#42 0x0000088014c0ce88 in vm_yield (ec=0x8801729e248, argc=1, argv=0x7f7ffffdc968) at vm.c:1122
#43 0x0000088014be6220 in rb_yield_0 (argc=1, argv=0x7f7ffffdc968) at vm_eval.c:970
#44 0x0000088014be61a4 in rb_yield_1 (val=3248396) at vm_eval.c:976
#45 0x0000088014be6281 in rb_yield (val=3248396) at vm_eval.c:986
#46 0x00000880148ed4fd in rb_ary_each (ary=9349739234000) at array.c:2076
#47 0x0000088014c0a590 in call_cfunc_0 (func=0x880148ed460 <rb_ary_each>, recv=9349739234000, argc=0, argv=0x88030e5c558) at vm_insnhelper.c:1746
#48 0x0000088014bf9fe7 in vm_call_cfunc_with_frame (ec=0x8801729e248, reg_cfp=0x88030f5b890, calling=0x7f7ffffde670, ci=0x880ceb4e5c0, cc=0x8804671c7a0) at vm_insnhelper.c:1905
#49 0x0000088014bf3337 in vm_call_cfunc (ec=0x8801729e248, reg_cfp=0x88030f5b890, calling=0x7f7ffffde670, ci=0x880ceb4e5c0, cc=0x8804671c7a0) at vm_insnhelper.c:1921
#50 0x0000088014bd793e in vm_exec_core (ec=0x8801729e248, initial=0) at insns.def:751
#51 0x0000088014bed157 in rb_vm_exec (ec=0x8801729e248, mjit_enable_p=1) at vm.c:1821
#52 0x0000088014c0d4a2 in invoke_block (ec=0x8801729e248, iseq=0x880b8300e80, self=9346731907800, captured=0x7f7ffffdea58, cref=0x88034a29970, type=572653569, opt_pc=0) at vm.c:1007
#53 0x0000088014c0d28b in invoke_iseq_block_from_c (ec=0x8801729e248, captured=0x7f7ffffdea58, self=9346731907800, argc=1, argv=0x7f7ffffdeae8, passed_block_handler=0, cref=0x88034a29970, is_lambda=0) at vm.c:1059
#54 0x0000088014c0cf5b in invoke_block_from_c_bh (ec=0x8801729e248, block_handler=140187732404825, argc=1, argv=0x7f7ffffdeae8, passed_block_handler=0, cref=0x88034a29970, is_lambda=0, force_blockarg=0) at vm.c:1077
#55 0x0000088014be81cd in vm_yield_with_cref (ec=0x8801729e248, argc=1, argv=0x7f7ffffdeae8, cref=0x88034a29970, is_lambda=0) at vm.c:1114
#56 0x0000088014be8711 in yield_under (under=9346731907480, self=9346731907800, argc=1, argv=0x7f7ffffdeae8) at vm_eval.c:1568
#57 0x0000088014be83a4 in specific_eval (argc=0, argv=0x88030e5c4c0, klass=9346731907480, self=9346731907800) at vm_eval.c:1607
#58 0x0000088014be8231 in rb_obj_instance_eval (argc=0, argv=0x88030e5c4c0, self=9346731907800) at vm_eval.c:1680
#59 0x0000088014c0a537 in call_cfunc_m1 (func=0x88014be81f0 <rb_obj_instance_eval>, recv=9346731907800, argc=0, argv=0x88030e5c4c0) at vm_insnhelper.c:1740
#60 0x0000088014bf9fe7 in vm_call_cfunc_with_frame (ec=0x8801729e248, reg_cfp=0x88030f5b970, calling=0x7f7ffffe07b0, ci=0x8802879f510, cc=0x880aa17aca8) at vm_insnhelper.c:1905
#61 0x0000088014bf3337 in vm_call_cfunc (ec=0x8801729e248, reg_cfp=0x88030f5b970, calling=0x7f7ffffe07b0, ci=0x8802879f510, cc=0x880aa17aca8) at vm_insnhelper.c:1921
#62 0x0000088014bd793e in vm_exec_core (ec=0x8801729e248, initial=0) at insns.def:751
#63 0x0000088014bed0dc in rb_vm_exec (ec=0x8801729e248, mjit_enable_p=1) at vm.c:1812
#64 0x0000088014c0d67b in invoke_bmethod (ec=0x8801729e248, iseq=0x880eb3e4d00, self=9346731907800, captured=0x880e8683180, me=0x8803a24ce18, type=572653825, opt_pc=0) at vm.c:1028
#65 0x0000088014c0d2b7 in invoke_iseq_block_from_c (ec=0x8801729e248, captured=0x880e8683180, self=9346731907800, argc=0, argv=0x7f7ffffe0b70, passed_block_handler=0, cref=0x0, is_lambda=1) at vm.c:1062
#66 0x0000088014bebccd in invoke_block_from_c_proc (ec=0x8801729e248, proc=0x880e8683180, self=9346731907800, argc=0, argv=0x7f7ffffe0b70, passed_block_handler=0, is_lambda=1) at vm.c:1152
#67 0x0000088014bebbf7 in rb_vm_invoke_bmethod (ec=0x8801729e248, proc=0x880e8683180, self=9346731907800, argc=0, argv=0x7f7ffffe0b70, block_handler=0) at vm.c:1177
#68 0x0000088014bfa6cb in vm_call_bmethod_body (ec=0x8801729e248, calling=0x7f7ffffe26d8, ci=0x88038a6eb90, cc=0x880a567f828, argv=0x7f7ffffe0b70) at vm_insnhelper.c:1948
#69 0x0000088014bf3cd3 in vm_call_bmethod (ec=0x8801729e248, cfp=0x88030f5b9a8, calling=0x7f7ffffe26d8, ci=0x88038a6eb90, cc=0x880a567f828) at vm_insnhelper.c:1965
#70 0x0000088014bd7b8d in vm_exec_core (ec=0x8801729e248, initial=0) at insns.def:766
#71 0x0000088014bed0dc in rb_vm_exec (ec=0x8801729e248, mjit_enable_p=1) at vm.c:1812
#72 0x0000088014c0d4a2 in invoke_block (ec=0x8801729e248, iseq=0x880861e7da0, self=9348791308800, captured=0x88030f5bd40, cref=0x0, type=572653569, opt_pc=0) at vm.c:1007
#73 0x0000088014c0d28b in invoke_iseq_block_from_c (ec=0x8801729e248, captured=0x88030f5bd40, self=9348791308800, argc=1, argv=0x7f7ffffe2a88, passed_block_handler=0, cref=0x0, is_lambda=0) at vm.c:1059
#74 0x0000088014c0cf5b in invoke_block_from_c_bh (ec=0x8801729e248, block_handler=9346670247233, argc=1, argv=0x7f7ffffe2a88, passed_block_handler=0, cref=0x0, is_lambda=0, force_blockarg=0) at vm.c:1077
#75 0x0000088014c0ce88 in vm_yield (ec=0x8801729e248, argc=1, argv=0x7f7ffffe2a88) at vm.c:1122
#76 0x0000088014be6220 in rb_yield_0 (argc=1, argv=0x7f7ffffe2a88) at vm_eval.c:970
#77 0x0000088014be61a4 in rb_yield_1 (val=9347060672320) at vm_eval.c:976
#78 0x0000088014be6281 in rb_yield (val=9347060672320) at vm_eval.c:986
#79 0x00000880148ed4fd in rb_ary_each (ary=9347060671520) at array.c:2076
#80 0x0000088014c0a590 in call_cfunc_0 (func=0x880148ed460 <rb_ary_each>, recv=9347060671520, argc=0, argv=0x88030e5c250) at vm_insnhelper.c:1746
#81 0x0000088014bf9fe7 in vm_call_cfunc_with_frame (ec=0x8801729e248, reg_cfp=0x88030f5bd28, calling=0x7f7ffffe4790, ci=0x880ec650190, cc=0x880d8392f80) at vm_insnhelper.c:1905
#82 0x0000088014bf3337 in vm_call_cfunc (ec=0x8801729e248, reg_cfp=0x88030f5bd28, calling=0x7f7ffffe4790, ci=0x880ec650190, cc=0x880d8392f80) at vm_insnhelper.c:1921
#83 0x0000088014bd793e in vm_exec_core (ec=0x8801729e248, initial=0) at insns.def:751
#84 0x0000088014bed0dc in rb_vm_exec (ec=0x8801729e248, mjit_enable_p=1) at vm.c:1812
#85 0x0000088014c0d4a2 in invoke_block (ec=0x8801729e248, iseq=0x880aef12390, self=9349181362960, captured=0x88030f5bf38, cref=0x0, type=572653569, opt_pc=0) at vm.c:1007
#86 0x0000088014c0d28b in invoke_iseq_block_from_c (ec=0x8801729e248, captured=0x88030f5bf38, self=9349181362960, argc=1, argv=0x7f7ffffe4ac8, passed_block_handler=0, cref=0x0, is_lambda=0) at vm.c:1059
#87 0x0000088014c0cf5b in invoke_block_from_c_bh (ec=0x8801729e248, block_handler=9346670247737, argc=1, argv=0x7f7ffffe4ac8, passed_block_handler=0, cref=0x0, is_lambda=0, force_blockarg=1) at vm.c:1077
#88 0x0000088014be676b in vm_yield_force_blockarg (ec=0x8801729e248, args=9348791308800) at vm.c:1138
#89 0x0000088014be66c7 in rb_yield_force_blockarg (values=9348791308800) at vm_eval.c:1035
#90 0x00000880148f502d in rb_ary_collect (ary=9347718360040) at array.c:3013
#91 0x0000088014c0a590 in call_cfunc_0 (func=0x880148f4f70 <rb_ary_collect>, recv=9347718360040, argc=0, argv=0x88030e5c0c0) at vm_insnhelper.c:1746
#92 0x0000088014bf9fe7 in vm_call_cfunc_with_frame (ec=0x8801729e248, reg_cfp=0x88030f5bf20, calling=0x7f7ffffe6970, ci=0x880989fa640, cc=0x880f6687aa0) at vm_insnhelper.c:1905
#93 0x0000088014bf3337 in vm_call_cfunc (ec=0x8801729e248, reg_cfp=0x88030f5bf20, calling=0x7f7ffffe6970, ci=0x880989fa640, cc=0x880f6687aa0) at vm_insnhelper.c:1921
#94 0x0000088014bf272b in vm_call_method_each_type (ec=0x8801729e248, cfp=0x88030f5bf20, calling=0x7f7ffffe6970, ci=0x880989fa640, cc=0x880f6687aa0) at vm_insnhelper.c:2243
#95 0x0000088014bf246c in vm_call_method (ec=0x8801729e248, cfp=0x88030f5bf20, calling=0x7f7ffffe6970, ci=0x880989fa640, cc=0x880f6687aa0) at vm_insnhelper.c:2369
#96 0x0000088014bd31f2 in vm_call_general (ec=0x8801729e248, reg_cfp=0x88030f5bf20, calling=0x7f7ffffe6970, ci=0x880989fa640, cc=0x880f6687aa0) at vm_insnhelper.c:2412
#97 0x0000088014bd793e in vm_exec_core (ec=0x8801729e248, initial=0) at insns.def:751
#98 0x0000088014bed0dc in rb_vm_exec (ec=0x8801729e248, mjit_enable_p=1) at vm.c:1812
#99 0x0000088014c0d4a2 in invoke_block (ec=0x8801729e248, iseq=0x880ec290070, self=9349181362960, captured=0x880ce774600, cref=0x0, type=572653569, opt_pc=0) at vm.c:1007
#100 0x0000088014c0d28b in invoke_iseq_block_from_c (ec=0x8801729e248, captured=0x880ce774600, self=9349181362960, argc=0, argv=0x880f76ad108, passed_block_handler=0, cref=0x0, is_lambda=0) at vm.c:1059
#101 0x0000088014bebccd in invoke_block_from_c_proc (ec=0x8801729e248, proc=0x880ce774600, self=9349181362960, argc=0, argv=0x880f76ad108, passed_block_handler=0, is_lambda=0) at vm.c:1152
#102 0x0000088014bebf90 in vm_invoke_proc (ec=0x8801729e248, proc=0x880ce774600, self=9349181362960, argc=0, argv=0x880f76ad108, passed_block_handler=0) at vm.c:1170
#103 0x0000088014bebe08 in rb_vm_invoke_proc (ec=0x8801729e248, proc=0x880ce774600, argc=0, argv=0x880f76ad108, passed_block_handler=0) at vm.c:1191
#104 0x0000088014aad98a in rb_proc_call (self=9347342533080, args=9349999808760) at proc.c:881
#105 0x000008801499f722 in rb_call_end_proc (data=9347342533080) at eval_jump.c:13
#106 0x000008801499fb7d in exec_end_procs_chain (procs=0x88014f63a28, errp=0x8801729e2c0) at eval_jump.c:105
#107 0x000008801499f9eb in rb_exec_end_proc () at eval_jump.c:121
#108 0x00000880149a0415 in ruby_finalize_0 () at eval.c:137
#109 0x00000880149a0649 in ruby_cleanup (ex=0) at eval.c:194
#110 0x00000880149a0daf in ruby_run_node (n=0x880f0e77948) at eval.c:317
#111 0x0000087dfc6006e9 in main (argc=4, argv=0x7f7ffffe7168) at main.c:42
(gdb) info locals
header = (struct transient_alloc_header *) 0xfffffffffffffff0

Based on the backtrace, the issue appears to be in the new transient heap code added in preview3, specifically that rb_transient_heap_mark is called with a NULL pointer as the second argument. If any additional information is needed from gdb, please let me know and I can update with more information.

Currently, to reproduce:

gem install --development sequel # install dependencies
git clone [e-mail]:jeremyevans/sequel.git
cd sequel
git checkout a0203701957de236d4da06a13ced6a89c21e0f5d
ruby spec/plugin_spec.rb --seed 59498

I will try to work on a smaller reproducer if that would be helpful.

[ko1 (Koichi Sasada)]

Thank you for your report.
I found a bug and will commit soon.

Thanks,
Koichi

[ko1 (Koichi Sasada)]

• Status changed from Open to Closed

Applied in changeset trunk|r66091.

---

clear dst Hash on Hash#replace. [Bug #15358]

• hash.c (linear_copy): solve two issues on Hash#replace.
  (1) fix memory leak
  (1-1) don't allocate memory if destination already
  has a memory area.
  (1-2) free destination memory if src is NULL.
  (2) clear transient heap flag if src is NULL. [Bug #15358]