MP 1,1-5,1 Fatal Error

[PMartist]

Any help would be greatly appreciated, I keep getting the following error....

panic(cpu 4 caller 0xffffff801c6d0860): nvme: "Fatal error occurred. CSTS=0xffffffff US[1]=0x0 US[0]=0x2f VID=0x126f DID=0x2260
. FW Revision=C2.3.23A\n"@/System/Volumes/Data/SWE/macOS/BuildRoots/d7e177bcf5/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.120.1/Common/IONVMeController.cpp:5499
Backtrace (CPU 4), Frame : Return Address
0xffffffa134c53a70 : 0xffffff8019e8e04d
0xffffffa134c53ac0 : 0xffffff8019fd4e13
0xffffffa134c53b00 : 0xffffff8019fc540a
0xffffffa134c53b50 : 0xffffff8019e32a2f
0xffffffa134c53b70 : 0xffffff8019e8d86d
0xffffffa134c53c90 : 0xffffff8019e8db63
0xffffffa134c53d00 : 0xffffff801a69dc0a
0xffffffa134c53d70 : 0xffffff801c6d0860
0xffffffa134c53d90 : 0xffffff801c6b545b
0xffffffa134c53ef0 : 0xffffff801c6b563e
0xffffffa134c53f30 : 0xffffff801a6688de
0xffffffa134c53f70 : 0xffffff801a668602
0xffffffa134c53fa0 : 0xffffff8019e3213e
Kernel Extensions in backtrace:
com.apple.iokit.IONVMeFamily(2.1)[8E899C96-D93C-337A-952A-B8C54E90FBC4]@0xffffff801c6ae000->0xffffff801c6d7fff
dependency: com.apple.driver.AppleEFINVRAM(2.1)[DC3B80FD-4D23-3608-8AA2-9C526D44F5D3]@0xffffff801b2fe000->0xffffff801b307fff
dependency: com.apple.driver.AppleMobileFileIntegrity(1.0.5)[152D7047-0D43-3CA8-B1EA-251D3F80AC28]@0xffffff801b4c0000->0xffffff801b4d5fff
dependency: com.apple.iokit.IOPCIFamily(2.9)[F7AA4BAF-35CF-3784-A50C-5FD9CDCBE488]@0xffffff801c96c000->0xffffff801c994fff
dependency: com.apple.iokit.IOReportFamily(47)[AE367C5D-FEFC-390D-ABDF-86E037890F00]@0xffffff801c9a3000->0xffffff801c9a5fff
dependency: com.apple.iokit.IOStorageFamily(2.1)[6CD2A6EC-9FFC-370D-8FEE-C8016E5C6BBA]@0xffffff801ca6f000->0xffffff801ca80fff

Process name corresponding to current thread: kernel_task

Mac OS version:
20G95

Kernel version:
Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:31 PDT 2021; root:xnu-7195.141.2~5/RELEASE_X86_64
Kernel UUID: FECBF22B-FBBE-36DE-9664-F12A7DD41D3D
KernelCache slide: 0x0000000019c00000
KernelCache base: 0xffffff8019e00000
Kernel slide: 0x0000000019c10000
Kernel text base: 0xffffff8019e10000
__HIB text base: 0xffffff8019d00000
System model name: MacPro6,1 (Mac-F60DEB81FF30ACF6)
System shutdown begun: YES
Panic diags file available: NO (0xe00002cd)
Hibernation exit count: 0

System uptime in nanoseconds: 255324073318735
Last Sleep: absolute base_tsc base_nano
Uptime : 0x0000e83744b16571
Sleep : 0x0000000000000000 0x0000000000000000 0x0000000000000000
Wake : 0x0000000000000000 0x000ef44b9e66461d 0x0000000000000000
last started kext at 9910407124986: >usb.cdc.ncm 5.0.0 (addr 0xffffff801b6e5000, size 28672)
loaded kexts:
|SCSITaskUserClient 436.140.1
>!ATopCaseHIDEventDriver 4050.1
>X86PlatformShim 1.0.0
>AGPM 122.1
>!APlatformEnabler 2.7.0d0
@filesystems.autofs 3.0
@fileutil 20.036.15
@fsplaceholder 1.14.0
>AudioAUUC 1.70
>!AUpstreamUserClient 3.6.8
>!AMCCSControl 1.14
@kext.AMDFramebuffer 4.0.6
@kext.AMDRadeonX4000 4.0.6
@kext.AMDRadeonServiceManager 4.0.6
>!AGraphicsDevicePolicy 6.3.5
@AGDCPluginDisplayMetrics 6.3.5
>pmtelemetry 1
>!AHDAHardwareConfigDriver 283.15
|IOUserEthernet 1.0.1
>usb.!UUserHCI 1
>!AHDA 283.15
|IO!BSerialManager 8.0.5d7
>!AFWOHCI 5.7.4
@Dont_Steal_Mac_OS_X 7.0.0
>!AHV 1
>!ADiskImages2 1
>eficheck 1
>!A!ISlowAdaptiveClocking 4.0.0
>ACPI_SMC_PlatformPlugin 1.0.0
>!AThunderboltIP 4.0.3
@kext.AMD7000!C 4.0.6
>!ALPC 3.1
>!A!IMCEReporter 115
>!AFIVRDriver 4.1.0
>!AMGPUPowerControl 6.3.5
>!UDisplays 380
|IO!BUSBDFU 8.0.5d7
>!AFileSystemDriver 3.0.1
@filesystems.tmpfs 1
@filesystems.hfs.kext 556.100.11
@BootCache 40
@!AFSCompression.!AFSCompressionTypeZlib 1.0.0
@!AFSCompression.!AFSCompressionTypeDataless 1.0.0d1
@filesystems.apfs 1677.141.1
>AirPort.BrcmNIC 1400.1.1
>!AAHCIPort 346.100.2
|!ABCM5701Ethernet 11.0.0
@private.KextAudit 1.0
>!AACPIEC 6.1
>!AACPIButtons 6.1
>!ARTC 2.0
>!AHPET 1.8
>!ASMBIOS 2.1
>!AAPIC 1.7
>!A!ICPUPowerManagementClient 222.0.0
@!ASystemPolicy 2.0.0
@nke.applicationfirewall 311
|IOKitRegistryCompatibility 1
|EndpointSecurity 1
>!A!ICPUPowerManagement 222.0.0
>usb.cdc.ncm 5.0.0
>!UMergeNub 900.4.2
>!AMultitouchDriver 4440.3
>!AInputDeviceSupport 4400.35
>!AHS!BDriver 4050.1
>IO!BHIDDriver 8.0.5d7
@kext.triggers 1.0
>!AHIDKeyboard 224
>!ASMBus!C 1.0.18d1
@kext.AMDRadeonX4030HWLibs 1.0
|IOAccelerator!F2 442.9
@kext.AMDRadeonX4000HWServices 4.0.6
>DspFuncLib 283.15
@kext.OSvKernDSPLib 529
|IOAVB!F 940.4
|IOEthernetAVB!C 1.1.0
>usb.cdc.ecm 5.0.0
|IOFireWire!F 4.8.3
>usb.cdc.acm 5.0.0
>usb.serial 6.0.0
|IOSerial!F 11
@!AGPUWrangler 6.3.5
|IOSlowAdaptiveClocking!F 1.0.0
>IOPlatformPluginLegacy 1.0.0
>!AHDA!C 283.15
|IOHDA!F 283.15
>!UAudio 405.39
|IOAudio!F 300.6.1
@vecLib.kext 1.2.0
>X86PlatformPlugin 1.0.0
>!AThunderboltEDMSink 5.0.3
@plugin.IOgPTPPlugin 985.2
@kext.AMDSupport 4.0.6
@!AGraphicsDeviceControl 6.3.5
>!ASMBusPCI 1.0.14d1
>IOPlatformPlugin!F 6.0.0d8
>!AGraphicsControl 6.3.5
>!ABacklightExpert 1.1.0
|IONDRVSupport 585.1
|IOGraphics!F 585.1
>!UComposite 900.4.2
|Broadcom!BHost!CUSBTransport 8.0.5d7
|IO!BHost!CUSBTransport 8.0.5d7
|IO!BHost!CTransport 8.0.5d7
>usb.IOUSBHostHIDDevice 1.2
>usb.cdc 5.0.0
>usb.networking 5.0.0
>usb.!UHostCompositeDevice 1.2
>!AThunderboltDPOutAdapter 8.1.4
>!AThunderboltPCIUpAdapter 4.1.1
>usb.!UHub 1.2
>!AThunderboltDPInAdapter 8.1.4
>!AThunderboltDPAdapter!F 8.1.4
>!AThunderboltPCIDownAdapter 4.1.1
>!ABSDKextStarter 3
|IOSurface 290.8.1
@filesystems.hfs.encodings.kext 1
>!AXsanScheme 3
>!AThunderboltNHI 7.2.8
|IOThunderbolt!F 9.3.2
|IONVMe!F 2.1.0
>usb.!UXHCIPCI 1.2
>usb.!UXHCI 1.2
|IO80211!F 1200.12.2b1
|IOSkywalk!F 1
>corecapture 1.0.4
|IOAHCI!F 294.100.1
>mDNSOffloadUserClient 1.0.1b8
>usb.!UEHCIPCI 1.2
>usb.!UEHCI 1.2
>usb.!UHostPacketFilter 1.0
|IOUSB!F 900.4.2
>!AEFINVRAM 2.1
>!AEFIRuntime 2.1
|IOSMBus!F 1.1
|IOHID!F 2.0.0
$!AImage4 3.0.0
|IOTimeSync!F 985.2
|IONetworking!F 3.4
>DiskImages 493.0.0
|IO!B!F 8.0.5d7
|IOReport!F 47
|IO!BPacketLogger 8.0.5d7
$quarantine 4
$sandbox 300.0
@Kext.!AMatch 1.0.0d1
|CoreAnalytics!F 1
>!ASSE 1.0
>!AKeyStore 2
>!UTDM 511.141.1
|IOUSBMass!SDriver 184.140.2
|IOSCSIBlockCommandsDevice 436.140.1
|IO!S!F 2.1
|IOSCSIArchitectureModel!F 436.140.1
>!AMobileFileIntegrity 1.0.5
@kext.CoreTrust 1
>!AFDEKeyStore 28.30
>!AEffaceable!S 1.0
>!ACredentialManager 1.0
>KernelRelayHost 1
|IOUSBHost!F 1.2
>!UHostMergeProperties 1.2
>usb.!UCommon 1.0
>!ABusPower!C 1.0
>!ASEPManager 1.0.1
>IOSlaveProcessor 1
>!AACPIPlatform 6.1
>!ASMC 3.1.9
|IOPCI!F 2.9
|IOACPI!F 1.4
>watchdog 1
@kec.pthread 1
@kec.corecrypto 11.1
@kec.Libm 1

 

[rpmurray]

I would think it's likely a problem with the internal NVMe drive:

panic(cpu 4 caller 0xffffff801c6d0860): nvme: "Fatal error occurred. CSTS=0xffffffff US[1]=0x0 US[0]=0x2f VID=0x126f DID=0x2260
. FW Revision=C2.3.23A\n"@/System/Volumes/Data/SWE/macOS/BuildRoots/d7e177bcf5/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.120.1/Common/IONVMeController.cpp:5499

Does this happen frequently? If not you may try running a utility that checks the drive (SMART status) if it stays up long enough to run the tests. Something like DriveDx (free trial).

 

[PMartist]

I would think it's likely a problem with the internal NVMe drive:

panic(cpu 4 caller 0xffffff801c6d0860): nvme: "Fatal error occurred. CSTS=0xffffffff US[1]=0x0 US[0]=0x2f VID=0x126f DID=0x2260
. FW Revision=C2.3.23A\n"@/System/Volumes/Data/SWE/macOS/BuildRoots/d7e177bcf5/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.120.1/Common/IONVMeController.cpp:5499

Does this happen frequently? If not you may try running a utility that checks the drive (SMART status) if it stays up long enough to run the tests. Something like DriveDx (free trial).

Yes it happens often. I restart it works for a bit, that’s when I get the error. Haven’t pin pointed anything I am doing that causes the slow down

 

[PMartist]

I would think it's likely a problem with the internal NVMe drive:

panic(cpu 4 caller 0xffffff801c6d0860): nvme: "Fatal error occurred. CSTS=0xffffffff US[1]=0x0 US[0]=0x2f VID=0x126f DID=0x2260
. FW Revision=C2.3.23A\n"@/System/Volumes/Data/SWE/macOS/BuildRoots/d7e177bcf5/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-557.120.1/Common/IONVMeController.cpp:5499

Does this happen frequently? If not you may try running a utility that checks the drive (SMART status) if it stays up long enough to run the tests. Something like DriveDx (free trial).

Thank you for the response and I have installed the DriveDX... most of the indicators are good but the composite temperature is pretty low (38%) do you think this could be causing the fatal error? Our thoughts are the heatsink on the SSD is causing the failure.

 

[rpmurray]

I haven't had this happen to me myself but I understand from what I've read that this may have something to do with sleep/wake. One suggestion was to use the following command in Terminal:

sudo pmset -a standby 0