Bug #20890
closed
MacOS 15.1, Macbook pro 2024 m4, YJIT: Kernel Panic on network access, works w/o YJIT
Description
I'm getting the following panic which seems like it happens on network access, but only when YJIT is enabled:
panic(cpu 15 caller 0xfffffe00509d800c): [SPTM] VIOLATION_ILLEGAL_SPRR_INDEX: sptm_map_page(sptm.c:406) - page_fte(0xfffffde005159750), page_fte->type(0xe), new_sprr_index(0x7), new_pte(0x6001145892ce43), vaddr(0x11c2c8000)
Debugger message: panic
Memory ID: 0x6
OS release type: User
OS version: 24B2083
Kernel version: Darwin Kernel Version 24.1.0: Thu Oct 10 21:06:57 PDT 2024; root:xnu-11215.41.3~3/RELEASE_ARM64_T6041
Fileset Kernelcache UUID: 918FFAAEF5863665571822DC0F4C5BC0
Kernel UUID: 58CD7EDF-BF0B-3375-B9F8-F2484179E8A1
Boot session UUID: CE59B2C1-E45D-47C9-84B7-B6659DA80D25
iBoot version: iBoot-11881.41.5
secure boot?: YES
roots installed: 0
Paniclog version: 14
Debug Header address: 0xfffffe002bef5000
Debug Header entry count: 3
TXM load address: 0xfffffe003be74000
TXM UUID: 772CA8A2-ABBC-34D1-8CC3-C295F9A174FE
Debug Header kernelcache load address: 0xfffffe004be74000
Debug Header kernelcache UUID: 918FFAAE-F586-3665-5718-22DC0F4C5BC0
SPTM load address: 0xfffffe002be74000
SPTM UUID: 9EF80C6B-0E45-300A-8EC1-8C49E7CA3D3B
KernelCache slide: 0x0000000044e70000
KernelCache base: 0xfffffe004be74000
Kernel slide: 0x0000000044e78000
Kernel text base: 0xfffffe004be7c000
Kernel text exec slide: 0x0000000046534000
Kernel text exec base: 0xfffffe004d538000
mach_absolute_time: 0x30d81dc4f
Epoch Time: sec usec
Boot : 0x67334fd9 0x0001740d
Sleep : 0x00000000 0x00000000
Wake : 0x00000000 0x00000000
Calendar: 0x673351f0 0x000386f7
Zone info:
Zone map: 0xfffffe161110c000 - 0xfffffe361110c000
. VM : 0xfffffe161110c000 - 0xfffffe1adddd8000
. RO : 0xfffffe1ade000000 - 0xfffffe1c78000000
. GEN0 : 0xfffffe1c78000000 - 0xfffffe2144b30000
. GEN1 : 0xfffffe2144b30000 - 0xfffffe2611660000
. GEN2 : 0xfffffe2611660000 - 0xfffffe2ade190000
. GEN3 : 0xfffffe2ade190000 - 0xfffffe2faacc4000
. DATA : 0xfffffe2faacc4000 - 0xfffffe361110c000
Metadata: 0xfffffe8fe7cec000 - 0xfffffe8fefcec000
Bitmaps : 0xfffffe8fefcec000 - 0xfffffe8fffcec000
Extra : 0 - 0
CORE 0 recently retired instr at 0x0000000000000000
CORE 1 recently retired instr at 0x0000000000000000
CORE 2 recently retired instr at 0x0000000000000000
CORE 3 recently retired instr at 0x0000000000000000
CORE 4 is offline, skipping
CORE 5 is offline, skipping
CORE 6 is offline, skipping
CORE 7 is offline, skipping
CORE 8 is offline, skipping
CORE 9 is offline, skipping
CORE 10 recently retired instr at 0x0000000000000000
CORE 11 recently retired instr at 0x0000000000000000
CORE 12 recently retired instr at 0x0000000000000000
CORE 13 recently retired instr at 0x0000000000000000
CORE 14 recently retired instr at 0x0000000000000000
CORE 15 recently retired instr at 0x0000000000000000
TPIDRx_ELy = {1: 0xfffffe1c7e8626f8 0: 0x000000000000200f 0ro: 0x00000001e8fcbba0 }
CORE 0: PC=0xfffffe004d6ebe30, LR=0xfffffe004d6ebe2c, FP=0xfffffe89f9197e40
CORE 1: PC=0xfffffe004d6ebe30, LR=0xfffffe004d6ebe2c, FP=0xfffffe89f82e3e40
CORE 2: PC=0xfffffe004d6ebe30, LR=0xfffffe004d6ebe2c, FP=0xfffffe89f895be40
CORE 3: PC=0xfffffe004d6ebe30, LR=0xfffffe004d6ebe2c, FP=0xfffffe89f9437e40
CORE 4: PC=0xfffffe004d6e1158, LR=0xfffffe004d6ebe24, FP=0xfffffe89f810fe40
CORE 5: PC=0xfffffe004d6e1138, LR=0xfffffe004d6ebe24, FP=0xfffffe89f6d0be40
CORE 6: PC=0xfffffe004d6e1138, LR=0xfffffe004d6ebe24, FP=0xfffffe89f9167e40
CORE 7: PC=0xfffffe004d5c54d0, LR=0xfffffe004d5c54cc, FP=0xfffffe89f895bed0
CORE 8: PC=0xfffffe004d6e1138, LR=0xfffffe004d6ebe24, FP=0xfffffe89f93efe40
CORE 9: PC=0xfffffe004d6e1138, LR=0xfffffe004d6ebe24, FP=0xfffffe89f930be40
CORE 10: PC=0xfffffe004d5c54d0, LR=0xfffffe004d5c54cc, FP=0xfffffe89f91d3ed0
CORE 11: PC=0xfffffe004d5c54d0, LR=0xfffffe004d5c54cc, FP=0xfffffe89f8fc7ed0
CORE 12: PC=0xfffffe004d5c54d0, LR=0xfffffe004d5c54cc, FP=0xfffffe89f6b2bed0
CORE 13: PC=0xfffffe004d5c54cc, LR=0xfffffe004d5c54cc, FP=0xfffffe89f9347ed0
CORE 14: PC=0xfffffe004d5c54d0, LR=0xfffffe004d5c54cc, FP=0xfffffe89f9233ed0
CORE 15 is the one that panicked. Check the full backtrace for details.
Compressor Info: 0% of compressed pages limit (OK) and 0% of segments limit (OK) with 0 swapfiles and OK swap space
Panicked task 0xfffffe2612a55698: 1621 pages, 2 threads: pid 5169: ruby
Panicked thread: 0xfffffe1c7e8626f8, backtrace: 0xfffffe89f91df1e0, tid: 29558
lr: 0xfffffe004d588c60 fp: 0xfffffe89f91df270
lr: 0xfffffe004d6e7f34 fp: 0xfffffe89f91df2e0
lr: 0xfffffe004d6e6330 fp: 0xfffffe89f91df3a0
lr: 0xfffffe004d53dd18 fp: 0xfffffe89f91df3b0
lr: 0xfffffe004d588564 fp: 0xfffffe89f91df7a0
lr: 0xfffffe004ddf7370 fp: 0xfffffe89f91df7c0
lr: 0xfffffe00509d800c fp: 0xfffffe89f91df7d0
lr: 0xfffffe002bf35cd8 fp: 0xfffffde00beebed0
Kernel Extensions in backtrace:
com.apple.sptm(24.1)[9EF80C6B-0E45-300A-8EC1-8C49E7CA3D3B]@0xfffffe002beec000->0xfffffe002bf37fff
last started kext at 717123504: com.apple.filesystems.autofs 3.0 (addr 0xfffffe004c9950b0, size 5847)
loaded kexts:
com.apple.filesystems.autofs 3.0
com.apple.driver.AppleTopCaseHIDEventDriver 8410.3
com.apple.driver.CoreKDL 1
com.apple.driver.AppleBiometricServices 1
com.apple.driver.DiskImages.ReadWriteDiskImage 493.0.0
com.apple.driver.DiskImages.UDIFDiskImage 493.0.0
com.apple.driver.DiskImages.RAMBackingStore 493.0.0
com.apple.driver.DiskImages.FileBackingStore 493.0.0
com.apple.driver.AppleSmartBatteryManager 161.0.0
com.apple.driver.AppleThunderboltIP 4.0.3
com.apple.driver.AppleUSBDeviceNCM 5.0.0
com.apple.driver.AppleFileSystemDriver 3.0.1
com.apple.nke.l2tp 1.9
com.apple.filesystems.tmpfs 1
com.apple.filesystems.nfs 1
com.apple.filesystems.lifs 1
com.apple.filesystems.apfs 2313.41.1
com.apple.IOTextEncryptionFamily 1.0.0
com.apple.filesystems.hfs.kext 674
com.apple.security.BootPolicy 1
com.apple.BootCache 40
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.driver.AppleALSColorSensor 1.0.0d1
com.apple.driver.ApplePMP 1
com.apple.driver.AppleT6041CLPC 1
com.apple.driver.AppleTypeCRetimer 1.0.0
com.apple.driver.SEPHibernation 1
com.apple.driver.AppleCS42L84Audio 810.2
com.apple.driver.AppleSmartIO2 1
com.apple.AppleEmbeddedSimpleSPINORFlasher 1
com.apple.driver.AppleSN012776Amp 810.2
com.apple.driver.AppleT6041SOCTuner 1
com.apple.driver.AppleT6041MCC 1
com.apple.driver.AppleSamsungSerial 1.0.0d1
com.apple.driver.AppleSerialShim 1
com.apple.driver.AppleQSPIMC 1
com.apple.driver.AppleT6041PMGR 1
com.apple.driver.AppleS8000DWI 1.0.0d1
com.apple.driver.AppleEventLogHandler 1
com.apple.driver.AppleS8000AES 1
com.apple.driver.ApplePMPFirmware 1
com.apple.driver.ApplePMUFirmwareDriver 1.0.0
com.apple.driver.AppleDPDisplayTCON 1
com.apple.driver.AppleInterruptControllerV3 1.0.0d1
com.apple.driver.AppleBCMWLANBusInterfacePCIe 1
com.apple.driver.AppleBluetoothModule 1
com.apple.AGXG16X 322.10
com.apple.driver.AppleSecureRepair 1.0.0
com.apple.driver.usb.AppleSynopsysUSB40XHCI 1
com.apple.driver.AppleAVD 807
com.apple.driver.AppleJPEGDriver 7.1.5
com.apple.driver.AppleMobileDispT604X-DCP 140.0
com.apple.driver.AppleAVE2 803.36.1
com.apple.driver.AppleProResHW 426.1
com.apple.driver.AppleT8110DART 1
com.apple.driver.AudioDMAController-T604x 400.97
com.apple.driver.AppleS5L8920XPWM 1.0.0d1
com.apple.driver.AppleSDXC 3.5.3
com.apple.driver.AppleSPIMC 1
com.apple.driver.AppleS5L8940XI2C 1.0.0d2
com.apple.driver.AppleT6041 1
com.apple.driver.AppleM68Buttons 1.0.0d1
com.apple.iokit.IOUserEthernet 1.0.1
com.apple.driver.usb.AppleUSBUserHCI 1
com.apple.iokit.IOKitRegistryCompatibility 1
com.apple.iokit.EndpointSecurity 1
com.apple.driver.AppleDiskImages2 379.0.11
com.apple.AppleSystemPolicy 2.0.0
com.apple.kec.InvalidateHmac 1
com.apple.kec.AppleEncryptedArchive 1
com.apple.driver.driverkit.serial 6.0.0
com.apple.driver.AppleHSBluetoothDriver 8410.3
com.apple.driver.IOBluetoothHIDDriver 9.0.0
com.apple.driver.AppleActuatorDriver 8410.3
com.apple.driver.AppleMultitouchDriver 8410.3
com.apple.driver.AppleHIDKeyboard 8010.1
com.apple.driver.AppleMesaSEPDriver 100.99
com.apple.iokit.IOBiometricFamily 1
com.apple.driver.DiskImages.KernelBacked 493.0.0
com.apple.iokit.IOAVBFamily 1300.26
com.apple.driver.AppleXsanScheme 3
com.apple.driver.AppleConvergedIPCOLYBTControl 1
com.apple.driver.AppleConvergedPCI 1
com.apple.driver.AppleBluetoothDebug 1
com.apple.driver.AppleBTM 1.0.1
com.apple.driver.usb.networking 5.0.0
com.apple.driver.AppleThunderboltPCIDownAdapter 4.1.1
com.apple.driver.AppleThunderboltUSBDownAdapter 1.0.4
com.apple.driver.AppleThunderboltDPInAdapter 8.5.1
com.apple.driver.AppleThunderboltDPAdapterFamily 8.5.1
com.apple.nke.ppp 1.9
com.apple.driver.AppleBSDKextStarter 3
com.apple.kext.triggers 1.0
com.apple.driver.IOHIDPowerSource 1
com.apple.driver.AppleCallbackPowerSource 1
com.apple.filesystems.hfs.encodings.kext 1
com.apple.driver.AppleSyntheticGameController 12.1.3
com.apple.plugin.IOgPTPPlugin 1300.51
com.apple.driver.IOPAudioIOBufferDevice 200.39
com.apple.driver.IOPAudioLPMicDevice 200.39
com.apple.driver.IOPAudioClientManagerDevice 200.39
com.apple.driver.AppleIOPADMAStream 200.17
com.apple.driver.IOPAudioVoiceTriggerDevice 400.40
com.apple.driver.AppleSEPHDCPManager 1.0.1
com.apple.driver.AppleDCPDPTXProxy 1.0.0
com.apple.driver.DCPDPFamilyProxy 1
com.apple.driver.AppleTrustedAccessory 1
com.apple.iokit.AppleSEPGenericTransfer 1
com.apple.driver.AppleCSEmbeddedAudio 810.2
com.apple.driver.AppleDiagnosticDataAccessReadOnly 1.0.0
com.apple.driver.AOPAudio2 200.11
com.apple.driver.AppleEmbeddedAudio 810.2
com.apple.iokit.AppleARMIISAudio 400.22
com.apple.driver.IISAudioIsolatedStreamECProxy 400.22
com.apple.driver.ApplePassthroughPPM 3.0
com.apple.iokit.IONVMeFamily 2.1.0
com.apple.driver.AppleNANDConfigAccess 1.0.0
com.apple.driver.ApplePMGR 1
com.apple.driver.AppleARMWatchdogTimer 1
com.apple.driver.AppleStockholmControl 1.0.0
com.apple.driver.AppleHPM 3.4.4
com.apple.AGXFirmwareKextG16XRTBuddy 1
com.apple.AGXFirmwareKextRTBuddy64 322.10
com.apple.driver.AppleSPMIPMU 1.0.1
com.apple.driver.AppleDialogPMU 1.0.1
com.apple.driver.AppleHIDTransportFIFO 8000.26
com.apple.driver.AppleHIDTransport 8000.26
com.apple.driver.AppleSPU 1
com.apple.driver.AppleInputDeviceSupport 8000.26
com.apple.driver.AppleSART 1
com.apple.driver.ApplePTD 1.0.0
com.apple.driver.AppleBluetoothDebugService 1
com.apple.driver.AppleBCMWLANCore 1.0.0
com.apple.iokit.IO80211Family 1200.13.1
com.apple.driver.IOImageLoader 1.0.0
com.apple.driver.AppleOLYHAL 1
com.apple.iokit.IOGPUFamily 104.1.2
com.apple.driver.AppleUSBXDCIARM 1.0
com.apple.driver.AppleUSBXDCI 1.0
com.apple.iokit.IOUSBDeviceFamily 2.0.0
com.apple.driver.usb.AppleSynopsysUSBXHCI 1
com.apple.driver.usb.AppleUSBXHCI 1.2
com.apple.driver.AppleEmbeddedUSBHost 1
com.apple.driver.usb.AppleUSBHub 1.2
com.apple.driver.usb.AppleUSBHostCompositeDevice 1.2
com.apple.driver.AppleT6040TypeCPhy 1
com.apple.driver.AppleSPMI 1.0.1
com.apple.driver.AppleH16ANEInterface 8.107.2
com.apple.iokit.IOMobileGraphicsFamily-DCP 343.0.0
com.apple.iokit.IOMobileGraphicsFamily 343.0.0
com.apple.driver.AppleM2ScalerCSCDriver 265.0.0
com.apple.driver.AppleDCP 1
com.apple.driver.DCPAVFamilyProxy 1
com.apple.driver.AppleFirmwareKit 1
com.apple.driver.AppleH16CameraInterface 3.111.0
com.apple.driver.AppleT6040PCIe 1
com.apple.driver.AppleT6040PCIeC 1
com.apple.driver.ApplePIODMA 1
com.apple.driver.AppleThunderboltNHI 7.2.81
com.apple.driver.DMAChannelProxy 400.97
com.apple.driver.ExclavesAudioKext 200.54
com.apple.driver.AudioSharedDARTMapperProxy 200.54
com.apple.driver.AudioDMAFamily 400.97
com.apple.driver.AudioDMACLLTEscalationDetector-Stub 400.97
com.apple.driver.AppleDockChannel 1
com.apple.driver.AppleA7IOP-MXWrap-v1 1.0.2
com.apple.driver.AppleA7IOP-ASCWrap-v6 1.0.2
com.apple.driver.AppleDisplayCrossbar 1.0.0
com.apple.iokit.IOThunderboltFamily 9.3.3
com.apple.iokit.IODisplayPortFamily 1.0.0
com.apple.driver.AppleTypeCPhy 1
com.apple.iokit.IOPortFamily 1.0
com.apple.iokit.IOPAudioDriverFamily 200.11
com.apple.driver.AppleEmbeddedAudioLibs 400.9
com.apple.driver.AppleFirmwareUpdateKext 1
com.apple.driver.AppleGPIOICController 1.0.2
com.apple.driver.AppleEverestErrorHandler 1
com.apple.driver.AppleMultiFunctionManager 1
com.apple.driver.corecapture 1.0.4
com.apple.driver.AppleEmbeddedPCIE 1
com.apple.driver.AppleMobileApNonce 1
com.apple.driver.usb.AppleUSBHostPacketFilter 1.0
com.apple.iokit.IOTimeSyncFamily 1300.51
com.apple.driver.DiskImages 493.0.0
com.apple.iokit.IOGraphicsFamily 598
com.apple.iokit.IOBluetoothFamily 9.0.0
com.apple.driver.AppleUIO 1
com.apple.driver.AppleT6041ANEHAL 8.107.3
com.apple.driver.AppleSSE 1.0
com.apple.driver.AppleSEPKeyStore 2
com.apple.driver.AppleUSBTDM 556
com.apple.iokit.IOUSBMassStorageDriver 250
com.apple.iokit.IOPCIFamily 2.9
com.apple.iokit.IOUSBHostFamily 1.2
com.apple.driver.AppleUSBHostMergeProperties 1.2
com.apple.driver.usb.AppleUSBCommon 1.0
com.apple.driver.AppleSMC 3.1.9
com.apple.driver.RTBuddy 1.0.0
com.apple.driver.AppleEmbeddedTempSensor 1.0.0
com.apple.driver.AppleARMPMU 1.0
com.apple.iokit.IOAccessoryManager 1.0.0
com.apple.driver.AppleOnboardSerial 1.0
com.apple.iokit.IOSerialFamily 11
com.apple.iokit.IOSCSIBlockCommandsDevice 498
com.apple.iokit.IOSCSIArchitectureModelFamily 498
com.apple.driver.AppleRSMChannel 1
com.apple.iokit.IORSMFamily 1
com.apple.driver.AppleLockdownMode 1
com.apple.driver.AppleIPAppender 1.0
com.apple.iokit.IOSkywalkFamily 1.0
com.apple.driver.mDNSOffloadUserClient 1.0.1b8
com.apple.iokit.IONetworkingFamily 3.4
com.apple.driver.AppleFDEKeyStore 28.30
com.apple.driver.AppleEffaceableStorage 1.0
com.apple.driver.AppleCredentialManager 1.0
com.apple.driver.AppleSEPManager 1.0.1
com.apple.driver.IODARTFamily 1
com.apple.driver.AppleA7IOP 1.0.2
com.apple.driver.IOSlaveProcessor 1
com.apple.driver.AppleBiometricSensor 2
com.apple.iokit.IOHIDFamily 2.0.0
com.apple.AUC 1.0
com.apple.iokit.IOSurface 372.2.3
com.apple.iokit.IOAVFamily 1.0.0
com.apple.iokit.IOHDCPFamily 1.0.0
com.apple.iokit.IOCECFamily 1
com.apple.iokit.IOAudio2Family 1.0
com.apple.driver.AppleIISController 400.1
com.apple.driver.AppleAudioClockLibs 400.9
com.apple.driver.FairPlayIOKit 72.10.0
com.apple.driver.AppleARMPlatform 1.0.2
com.apple.iokit.IOSlowAdaptiveClockingFamily 1.0.0
com.apple.iokit.IOReportFamily 47
com.apple.security.quarantine 4
com.apple.security.sandbox 300.0
com.apple.iokit.IOStorageFamily 2.1
com.apple.kext.AppleMatch 1.0.0d1
com.apple.driver.AppleMobileFileIntegrity 1.0.5
com.apple.iokit.CoreAnalyticsFamily 1
com.apple.security.AppleImage4 7.0.0
com.apple.kext.CoreTrust 1
com.apple.iokit.IOCryptoAcceleratorFamily 1.0.1
com.apple.kec.pthread 1
com.apple.kec.Libm 1
com.apple.kec.Compression 1.0
com.apple.kec.corecrypto 14.0
I have no problems with a Macbook pro 2023 m2 with the same configuration, event with YJIT.
I can reproduce the panic with 100 % success rate by just starting my Rails app and trying to access it. Or by editing a file in neovim which uses some ruby plugins.
I compiled ruby w/o YJIT and the panics seem to be gone.
ruby -v is without YJIT (same revision with YJIT panics).
Updated by markus_d (Markus Doits) 6 days ago
For reference here is another backtrace from 3 days ago which I found by googling around, so somebody else seems to have the same problem (no further info though): https://pastecode.io/s/t1a3a8yp. Maybe the new MacbookPros are having this problem.
I updated to 15.2 beta 3 to see if it fixed it but nope, same problem.
Ruby is installed by rbenv.
Updated by k0kubun (Takashi Kokubun) 6 days ago
- Status changed from Open to Feedback
- Assignee set to yjit
How can we reproduce your issue? We don't know how to run "my Rails app". If you can show commands for reproducing it including rails new or git clone of a public repository, that would help us debug the issue.
Updated by markus_d (Markus Doits) 6 days ago
Thanks for your fast reply. You're right the reproduction steps are missing, though unfortunately I tried to recreate a minimal example (downloading some files with net-http, suspecting it has something to do with network access), but couldn't reproduce it.
Maybe it has to be of a certain size for the YJIT. For example my Rails app loads correctly but the panic happens when triggering a request to a controller. And my editor creates the panic when opening multiple files with ruby-lsp attached.
I will try to find a minimal example for you to reproduce the bug, feel free to wait for it until looking further. In the meantime maybe others having the same problem find this issue.
Updated by markus_d (Markus Doits) 6 days ago
OK, I got no small example (tried a lot) but a reproducible case with then open source rails app mastodon at least. Simply starting it triggers the panic.
Here's how to reproduce it on MacOS:
# I compiled 3.3.6 without YJIT, so I install it with YJIT under a different prefix to have both
RUBY_CONFIGURE_OPTS=--with-openssl-dir=/opt/homebrew/opt/openssl@3 ruby-build 3.3.6 ~/.rbenv/versions/3.3.6-yjit
# clone mastodon
git clone https://github.com/mastodon/mastodon.git
cd mastodon
# run setup for mastodon, postgresql must be installed, will need/install yarn, too ...
RAILS_ENV=development RBENV_VERSION=3.3.6-yjit bin/setup
# start it
RAILS_ENV=development RBENV_VERSION=3.3.6-yjit bin/dev
After starting it, it prints some debug output to the console for 2-5 seconds (starting up etc) until the screen goes black and it reboots.
I have installed a version without YJIT under the prefix 3.3.6 which does not trigger the panic:
# no YJIT no panic
RAILS_ENV=development RBENV_VERSION=3.3.6 bin/dev
I hope this helps to reproduce it.
Updated by k0kubun (Takashi Kokubun) 6 days ago
- Status changed from Feedback to Open
- Backport changed from 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN to 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: REQUIRED
Updated by alanwu (Alan Wu) 6 days ago
- Status changed from Open to Third Party's Issue
Fascinating. Could you please also report this to Apple? It's clearly a kernel bug when an userspace applications can reliably cause the kernel to panic. We'll try to look for a workaround, but Apple is in a much better position to make a fix.
Updated by markus_d (Markus Doits) 6 days ago
Thanks for looking into it. I already reported it before posting here, feedback ID at Apple is FB15774033. No answer yet.
Btw.: Can you reproduce it or am I still the only one?
Updated by k0kubun (Takashi Kokubun) 6 days ago
- Backport changed from 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: REQUIRED to 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
Updated by alanwu (Alan Wu) 6 days ago
Btw.: Can you reproduce it or am I still the only one?
I don't have access to the new M4 Macs to try, unfortunately.
Updated by aidenfoxivey (Aiden Fox Ivey) 6 days ago
Not sure if it's of any use, but this doesn't show up on my M1 Mac with Ruby 3.3.6 compiled with YJIT.
Updated by markus_d (Markus Doits) 5 days ago
I have no problems on a m2 MacbookPro Max either. We got a new m4 MacbookPro Max yesterday where this happens constantly. It was freshly installed.
Updated by aaronjensen (Aaron Jensen) 5 days ago
· Edited
I can also reproduce this on two M4 Max Macbook Pros. For my app, it actually panics on app start (either during or right after preload) when I start with multiple workers. It crashes before it prints the individual workers, so I wondered if it had to do with forking and copy-on-write. I also (coicindentally) chose Mastodon to reproduce it when I reported it to Puma: https://github.com/puma/puma/issues/3551
I went the Apple Support route and it's currently waiting to be reviewed so that they can determine whether or not they will even show it to the developers. Apparently they typically don't report "application issues" to developers and they don't seem to understand that reliable kernel panics are typically not "application issues".
I also have YJIT enabled, so that factor is interesting.
Also of note is that it seems to be dependent on other applications running. If I don't have enough running on my machine, I can no longer reproduce it. It doesn't seem to matter what those things are, however.
If I disable app preload in Puma, it does not reproduce, even with YJIT.
I just reported with Apple Feedback Assistant as well: FB15789489
Updated by aaronjensen (Aaron Jensen) 5 days ago
One more note is that you can still use preload if you disable Rail's enabling of YJIT (config.yjit = false). You can still use YJIT if you enable YJIT in on_worker_boot.
This may point to the problem specifically being enabling YJIT before forking.
Updated by aaronjensen (Aaron Jensen) 5 days ago
Unfortunate update from Apple:
Thank you for being patient while the case was under review. I did receive a response from my escalation approval team about the matter at hand. Since it appears that the Macs are running 3rd party software on the macOS system causing it to crash, this would be an issue we would refer you to the developer of those applications/programs for. IF the Macs are crashing due only to one of our apple applications we can definitely investigate further, but with the 3rd party software/programs we wouldn’t be able to proceed beyond this.
Hopefully the feedback reports go straight to their dev team and someone will address it.
That said, is there any possibility that this is actually a YJIT or Ruby fork bug somehow?
Updated by alanwu (Alan Wu) 5 days ago
That said, is there any possibility that this is actually a YJIT or Ruby fork bug somehow?
No. Modern kernels don't allow unprivileged programs like YJIT to crash them, period. If they did we'd have many broken promises like for example, letting a program running in a VM bring down the entire host system. The line in the sand is clear and has been for decades.
Updated by aaronjensen (Aaron Jensen) 5 days ago
· Edited
No. Modern kernels don't allow unprivileged programs like YJIT to crash them, period.
Makes sense to me. Does anyone have experience to getting Apple to pay attention or is this just a waiting game at this point?
Updated by friek (Johan Mulder) 4 days ago
PHP developer chiming in here.
I ran into the exact same point at which my Mac Mini M4 panicked when running the Psalm code analyser with PHP opcache enabled. With opcache disabled it doesn't panic. As the same is apparently happening to Ruby I believe this is probably a kernel bug.
Also tried it on a Mac Mini M1 running the same Sequoia and it doesn't happen there at all.
Updated by alanwu (Alan Wu) 4 days ago
With respect to getting Apple's attention, maybe they prioritize reports from beta versions. Maybe worth a try?
Updated by maximecb (Maxime Chevalier-Boisvert) 4 days ago
If we can clearly show that it's a kernel bug I am sure that we can get Apple to pay attention and fix it.
Two important questions:
- Can we get a minimal reproduction going? Ideally we would like to know which specific conditions trigger the bug. Ideally we'd have a single-file C program that triggers the bug.
- Which specific versions of the kernel is the bug happening on? @aaronjensen (Aaron Jensen) you said you can repro this on two M4 machines, can you report your Darwin kernel version and the make and model of the two machines you reproduce this on?
We (the YJIT team) don't yet have M4 macs so we'll need some help from people in this thread to come up with a reproduction.
Updated by mattjankowski (Matt Jankowski) 4 days ago
I have an M4 Max MacBook Pro and also see this issue. Like the initial report, I also saw it by just launching a rails app locally. Was able to replicate in multiple different local apps.
OS Version is Sequoia 15.1 (current latest non-beta) which is Darwin Kernel Version 24.1.0
I also filed an apple feedback but havent heard back yet. Will update if I do.
Can attempt to work on a minimal reproduction case when I have time to watch my system crash repeatedly. Also happy to run something from others if anyone thinks they have a reliable reproduction but don't have the right hardware.
Updated by maximecb (Maxime Chevalier-Boisvert) 4 days ago
Thank you @mattjankowski (Matt Jankowski), your help is appreciated.
Updated by jmanrubia (Jorge Manrubia) 1 day ago
· Edited
I am getting this error, too, when running the suite of one of our Rails apps. It only happens for one app, not for others. It seems to be related to forking during parallel test executions. With PARALLEL_WORKERS=1, it does not happen.
I managed to create a Ruby script that makes MacOS crash consistently for me:
PROCESS_COUNT = 50
MEMORY_CHUNK_SIZE = 10_000_000
child_pids = []
PROCESS_COUNT.times do |i|
pid = fork do
Array.new(MEMORY_CHUNK_SIZE) { rand }
puts "Child #{i} allocated memory"
sleep rand(0.1..1.0)
exit!(0)
end
child_pids << pid
end
child_pids.each do |pid|
Process.wait(pid)
end
When running the script with --disable-yjit, it does not happen. When running it with --yjit, MacOS crashes.
Looks like a MacOS bug for sure, there are other reports in other systems not related to Ruby (example).