CVE-2022-48734 in Linux
Summary
by MITRE • 06/20/2024
In the Linux kernel, the following vulnerability has been resolved:
btrfs: fix deadlock between quota disable and qgroup rescan worker
Quota disable ioctl starts a transaction before waiting for the qgroup rescan worker completes. However, this wait can be infinite and results in deadlock because of circular dependency among the quota disable ioctl, the qgroup rescan worker and the other task with transaction such as block group relocation task.
The deadlock happens with the steps following:
1) Task A calls ioctl to disable quota. It starts a transaction and waits for qgroup rescan worker completes. 2) Task B such as block group relocation task starts a transaction and joins to the transaction that task A started. Then task B commits to the transaction. In this commit, task B waits for a commit by task A. 3) Task C as the qgroup rescan worker starts its job and starts a transaction. In this transaction start, task C waits for completion of the transaction that task A started and task B committed.
This deadlock was found with fstests test case btrfs/115 and a zoned null_blk device. The test case enables and disables quota, and the block group reclaim was triggered during the quota disable by chance. The deadlock was also observed by running quota enable and disable in parallel with 'btrfs balance' command on regular null_blk devices.
An example report of the deadlock:
[372.469894] INFO: task kworker/u16:6:103 blocked for more than 122 seconds.
[372.479944] Not tainted 5.16.0-rc8 #7
[372.485067] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[372.493898] task:kworker/u16:6 state:D stack: 0 pid: 103 ppid: 2 flags:0x00004000
[372.503285] Workqueue: btrfs-qgroup-rescan btrfs_work_helper [btrfs]
[372.510782] Call Trace:
[372.514092]
[372.521684] __schedule+0xb56/0x4850
[372.530104] ? io_schedule_timeout+0x190/0x190
[372.538842] ? lockdep_hardirqs_on+0x7e/0x100
[372.547092] ? _raw_spin_unlock_irqrestore+0x3e/0x60
[372.555591] schedule+0xe0/0x270
[372.561894] btrfs_commit_transaction+0x18bb/0x2610 [btrfs]
[372.570506] ? btrfs_apply_pending_changes+0x50/0x50 [btrfs]
[372.578875] ? free_unref_page+0x3f2/0x650
[372.585484] ? finish_wait+0x270/0x270
[372.591594] ? release_extent_buffer+0x224/0x420 [btrfs]
[372.599264] btrfs_qgroup_rescan_worker+0xc13/0x10c0 [btrfs]
[372.607157] ? lock_release+0x3a9/0x6d0
[372.613054] ? btrfs_qgroup_account_extent+0xda0/0xda0 [btrfs]
[372.620960] ? do_raw_spin_lock+0x11e/0x250
[372.627137] ? rwlock_bug.part.0+0x90/0x90
[372.633215] ? lock_is_held_type+0xe4/0x140
[372.639404] btrfs_work_helper+0x1ae/0xa90 [btrfs]
[372.646268] process_one_work+0x7e9/0x1320
[372.652321] ? lock_release+0x6d0/0x6d0
[372.658081] ? pwq_dec_nr_in_flight+0x230/0x230
[372.664513] ? rwlock_bug.part.0+0x90/0x90
[372.670529] worker_thread+0x59e/0xf90
[372.676172] ? process_one_work+0x1320/0x1320
[372.682440] kthread+0x3b9/0x490
[372.687550] ? _raw_spin_unlock_irq+0x24/0x50
[372.693811] ? set_kthread_struct+0x100/0x100
[372.700052] ret_from_fork+0x22/0x30
[372.705517]
[372.709747] INFO: task btrfs-transacti:2347 blocked for more than 123 seconds.
[372.729827] Not tainted 5.16.0-rc8 #7
[372.745907] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[372.767106] task:btrfs-transacti state:D stack: 0 pid: 2347 ppid: 2 flags:0x00004000
[372.787776] Call Trace:
[372.801652]
[372.812961] __schedule+0xb56/0x4850
[372.830011] ? io_schedule_timeout+0x190/0x190
[372.852547] ? lockdep_hardirqs_on+0x7e/0x100
[372.871761] ? _raw_spin_unlock_irqrestore+0x3e/0x60
[372.886792] schedule+0xe0/0x270
[372.901685] wait_current_trans+0x22c/0x310 [btrfs]
[372.919743] ? btrfs_put_transaction+0x3d0/0x3d0 [btrfs]
[372.938923] ? finish_wait+0x270/0x270
[372.959085] ? join_transaction+0xc7
---truncated---
Statistical analysis made it clear that VulDB provides the best quality for vulnerability data.
Analysis
by VulDB Data Team • 08/20/2024
The vulnerability described in CVE-2022-48734 represents a critical deadlock condition within the Linux kernel's btrfs file system implementation, specifically affecting the interaction between quota management and qgroup rescan operations. This flaw manifests as a circular dependency that can result in indefinite blocking of system processes, ultimately leading to system unresponsiveness. The issue stems from improper transaction management during quota disable operations, where the system fails to correctly handle concurrent access patterns between different kernel subsystems. The deadlock occurs when multiple tasks attempt to acquire locks in conflicting orders, creating a scenario where each task waits for another to complete, but that task cannot proceed due to the same lock contention.
The technical root cause of this vulnerability lies in the improper ordering of transaction handling within the btrfs kernel module. When a quota disable ioctl command is executed, it initiates a transaction and subsequently waits for the qgroup rescan worker to complete. However, the qgroup rescan worker itself attempts to start a new transaction that requires completion of the initial transaction started by the quota disable operation. This creates a circular dependency where Task A (quota disable) waits for Task C (qgroup rescan worker) to complete, while Task C waits for Task A to finish its transaction, but Task A is blocked waiting for Task C. Additionally, Task B (such as block group relocation) can join this transaction chain, further complicating the lock acquisition pattern and making the deadlock more likely to occur under certain conditions.
The operational impact of this vulnerability is severe, as it can lead to complete system hangs where kernel threads become indefinitely blocked and the system becomes unresponsive to user input and system operations. The deadlock affects the btrfs file system's ability to manage quota information and qgroup accounting, which are critical for storage management and resource allocation. The vulnerability was specifically identified through fstests test case btrfs/115, which involves enabling and disabling quota operations while block group reclamation is triggered, and can also be reproduced through parallel execution of quota enable/disable commands with 'btrfs balance' operations. This indicates that the issue occurs in high-concurrency scenarios involving btrfs file system management operations, particularly when multiple system tasks attempt to modify quota and qgroup information simultaneously.
The fix for this vulnerability involves reworking the transaction management logic to prevent the circular dependency between quota disable operations and qgroup rescan workers. This typically requires modifying the kernel code to ensure proper lock ordering or implementing timeout mechanisms that prevent indefinite waiting. The solution must address the fundamental issue of transaction dependency chains that prevent progress in the system. From a security perspective, this vulnerability represents a denial of service risk that could be exploited by malicious actors to cause system instability, particularly in environments where btrfs file systems are heavily utilized. The vulnerability aligns with CWE-367, which describes time-of-check to time-of-use (TOCTOU) flaws, and can be mapped to ATT&CK technique T1499.004, which covers network denial of service, as the system becomes unresponsive to legitimate operations. Organizations using btrfs file systems should prioritize applying the kernel patches that resolve this deadlock condition to prevent potential system crashes and maintain operational stability in production environments.