Linux Kernel up to 6.11.10/6.12.1 nvme-fabrics race condition

Summaryinfo

A vulnerability marked as problematic has been reported in Linux Kernel up to 6.11.10/6.12.1. The affected element is an unknown function of the component nvme-fabrics. This manipulation causes race condition. This vulnerability is registered as CVE-2024-53169. No exploit is available. It is suggested to upgrade the affected component.

Detailsinfo

A vulnerability classified as problematic was found in Linux Kernel up to 6.11.10/6.12.1. Affected by this vulnerability is an unknown code of the component nvme-fabrics. The manipulation with an unknown input leads to a race condition vulnerability. The CWE definition for the vulnerability is CWE-362. The product contains a code sequence that can run concurrently with other code, and the code sequence requires temporary, exclusive access to a shared resource, but a timing window exists in which the shared resource can be modified by another code sequence that is operating concurrently. As an impact it is known to affect availability. The summary by CVE is:

In the Linux kernel, the following vulnerability has been resolved: nvme-fabrics: fix kernel crash while shutting down controller The nvme keep-alive operation, which executes at a periodic interval, could potentially sneak in while shutting down a fabric controller. This may lead to a race between the fabric controller admin queue destroy code path (invoked while shutting down controller) and hw/hctx queue dispatcher called from the nvme keep-alive async request queuing operation. This race could lead to the kernel crash shown below: Call Trace: autoremove_wake_function+0x0/0xbc (unreliable) __blk_mq_sched_dispatch_requests+0x114/0x24c blk_mq_sched_dispatch_requests+0x44/0x84 blk_mq_run_hw_queue+0x140/0x220 nvme_keep_alive_work+0xc8/0x19c [nvme_core] process_one_work+0x200/0x4e0 worker_thread+0x340/0x504 kthread+0x138/0x140 start_kernel_thread+0x14/0x18 While shutting down fabric controller, if nvme keep-alive request sneaks in then it would be flushed off. The nvme_keep_alive_end_io function is then invoked to handle the end of the keep-alive operation which decrements the admin->q_usage_counter and assuming this is the last/only request in the admin queue then the admin->q_usage_counter becomes zero. If that happens then blk-mq destroy queue operation (blk_mq_destroy_ queue()) which could be potentially running simultaneously on another cpu (as this is the controller shutdown code path) would forward progress and deletes the admin queue. So, now from this point onward we are not supposed to access the admin queue resources. However the issue here's that the nvme keep-alive thread running hw/hctx queue dispatch operation hasn't yet finished its work and so it could still potentially access the admin queue resource while the admin queue had been already deleted and that causes the above crash. The above kernel crash is regression caused due to changes implemented in commit a54a93d0e359 ("nvme: move stopping keep-alive into nvme_uninit_ctrl()"). Ideally we should stop keep-alive before destroyin g the admin queue and freeing the admin tagset so that it wouldn't sneak in during the shutdown operation. However we removed the keep alive stop operation from the beginning of the controller shutdown code path in commit a54a93d0e359 ("nvme: move stopping keep-alive into nvme_uninit_ctrl()") and added it under nvme_uninit_ctrl() which executes very late in the shutdown code path after the admin queue is destroyed and its tagset is removed. So this change created the possibility of keep-alive sneaking in and interfering with the shutdown operation and causing observed kernel crash. To fix the observed crash, we decided to move nvme_stop_keep_alive() from nvme_uninit_ctrl() to nvme_remove_admin_tag_set(). This change would ensure that we don't forward progress and delete the admin queue until the keep- alive operation is finished (if it's in-flight) or cancelled and that would help contain the race condition explained above and hence avoid the crash. Moving nvme_stop_keep_alive() to nvme_remove_admin_tag_set() instead of adding nvme_stop_keep_alive() to the beginning of the controller shutdown code path in nvme_stop_ctrl(), as was the case earlier before commit a54a93d0e359 ("nvme: move stopping keep-alive into nvme_uninit_ctrl()"), would help save one callsite of nvme_stop_keep_alive().

The advisory is shared at git.kernel.org. This vulnerability is known as CVE-2024-53169 since 11/19/2024. The exploitation appears to be difficult. Neither technical details nor an exploit are publicly available.

The vulnerability scanner Nessus provides a plugin with the ID 216493 (Ubuntu 24.10 : Linux kernel vulnerabilities (USN-7276-1)), which helps to determine the existence of the flaw in a target environment.

Upgrading to version 6.11.11 or 6.12.2 eliminates this vulnerability. Applying the patch 30794f4952decb2ec8efa42f704cac5304499a41/5416b76a8156c1b8491f78f8a728f422104bb919/e9869c85c81168a1275f909d5972a3fc435304be is able to eliminate this problem. The bugfix is ready for download at git.kernel.org. The best possible mitigation is suggested to be upgrading to the latest version.

The vulnerability is also documented in the vulnerability database at Tenable (216493). If you want to get the best quality for vulnerability data then you always have to consider VulDB.

Productinfo

Type

• Operating System

Vendor

• Linux

Name

• Kernel

Version

• 6.11.0
• 6.11.1
• 6.11.2
• 6.11.3
• 6.11.4
• 6.11.5
• 6.11.6
• 6.11.7
• 6.11.8
• 6.11.9
• 6.11.10
• 6.12.0
• 6.12.1

License

• open-source

Website

• Vendor: https://www.kernel.org/

CPE 2.3info

• 🔍
• 🔍
• 🔍

CPE 2.2info

• 🔍
• 🔍
• 🔍

CVSSv4info

CVSSv3info

CVSSv2info

Exploitinginfo

Threat Intelligenceinfo

Countermeasuresinfo

Timelineinfo

Sourcesinfo

Entryinfo

If you want to get the best quality for vulnerability data then you always have to consider VulDB.

Discussion