Breaking the 10 MB/s Barrier: A Faster, Sequential Alternative to BTRFS RAID5/6 Scrubbing

by

If you are running a multi-disk BTRFS array using a RAID5 or RAID6 layout—especially over an encryption layer like LUKS—you are likely familiar with the dreaded “Scrub Crawl.”

You kick off a routine btrfs scrub, expecting your modern SATA drives to hum along at their typical 100+ MB/s sequential speeds. Instead, you open iotop or iostat only to find your array agonizingly throttled at a miserable 10 to 20 MB/s. Even worse, your disk heads are thrashing violently, and your monitoring tools show inexplicable write activity on an operation that should be strictly read-only.

The native BTRFS scrub engine has a severe architectural limitation when handling parity-based RAID geometries. In this article, we will break down exactly why the native scrub engine chokes on RAID5/6, and provide a completely automated, 3-phase alternative verification method that restores your true sequential hardware throughput (100+ MB/s) while validating 100% of your data, metadata, and RAID5/6 P/Q parities.

Why Native BTRFS RAID5/6 Scrub Behaves Like a Random I/O Nightmare

To understand why a native scrub is slow, we have to look at how the BTRFS kernel code sequences its read requests.

When you run btrfs scrub start -d 1 /mnt/array, BTRFS targets that specific device ID. However, because it is a RAID5/6 array, data is striped across your disks (e.g., a 4+2 data and parity matrix for a RAID6 setup). To verify any single block on Disk 1, the scrub thread must mathematically validate the entire stripe.

This forces BTRFS to issue concurrent, uncoordinated read requests across all the other disks in the array at the exact same time. Because mechanical SATA disks lack a hardware RAID controller to synchronize their physical platters:

  1. Disk 1 asks the other drives for the corresponding blocks in its current stripe.
  2. By the time those drives respond, Disk 1’s head has shifted, or the other disks have moved on to process a different layer of the BTRFS metadata B-Tree.
  3. The drive heads are forced to constantly seek back and forth across the physical platters to piece together the RAID5/6 stripes out of order.

What should be a clean, linear sequential read turns into a chaotic, high-latency random read workload.

The Solution: A Decoupled, Multi-Phase Sequential Sweep

We can bypass this random I/O trap entirely by splitting the validation process across the storage stack. Instead of forcing BTRFS to check filesystem data hashes and recalculate RAID5/6 parity algorithms simultaneously, we decouple the checks into three highly efficient, linear phases.

Phase 1: File Data Validation via find + cat

BTRFS is a Copy-on-Write (CoW) filesystem that natively stores cryptographic checksums for all file data blocks. BTRFS validates these checksums every single time a file is read normally. If a block doesn’t match its hash, BTRFS catches it instantly and triggers its read-healing mechanism.

By using a raw byte stream like find /mnt/array -type f -exec cat {} + > /dev/null, we force the kernel to read every active data block sequentially. Because this follows the standard, optimized OS file-read pipeline, the drive heads glide smoothly across the disks, maxing out your hardware sequential speed.

Phase 2: Parity (P/Q) & Replica Surface Validation

BTRFS does not store filesystem-level checksums for RAID5/6 P/Q parity blocks or backup metadata replicas (like RAID1c3). They are entirely dependent on physical disk block integrity.

Thankfully, modern SATA disks and SSDs possess an invisible line of defense: Hardware ECC (Error Correction Code) or LDPC (Low-Density Parity-Check). A SATA drive cannot physically hand data over the cable to Linux without validating it against its internal hardware ECC matrix first. If a parity block has suffered from silent bitrot or magnetic decay, the drive’s internal read head will catch it, fail the ECC calculation, and throw a hard UNC (Uncorrectable Media Error) down to the OS kernel.

By reading 100% of the underlying decrypted LUKS mapper devices (/dev/mapper/map_d*) linearly in parallel via dd, we force every single sector—including unallocated space, P/Q parities, and metadata replicas—through the drive’s internal hardware ECC engines at maximum sequential speeds.

Phase 3: Metadata Tree Integrity Scan

Finally, we run a swift, read-only structure traversal using btrfs inspect-internal tree-stats to verify that the logical indexing layout of the BTRFS metadata B-Trees is sound.

Conclusion: Total Protection with Zero Performance Penalties

By splitting our data verification strategy, we achieve total coverage without forcing our hardware into a state of structural gridlock:

  • If active file data rots: Phase 1 catches it immediately. BTRFS detects the hash mismatch and prints a clear checksum mismatch to dmesg.
  • If a RAID5/6 P/Q parity block rots: Phase 2 forces a sequential read over every block. The drive’s internal hardware ECC engine scans the data, catches the uncorrectable media fault, and sends a UNC Error up the stack, which is caught instantly by the script’s background monitor.

Stop letting your array spend days grinding away at 10 MB/s. Deploy the multi-phase sequential sweep, let your storage controller leverage its native hardware pipelines, and complete your full-array validation at maximum wire speed.

The Automation Script: btrfs-fast-verify.sh

This Bash script automates the entire process. It optimizes the LUKS inline crypto workers, maps out the sequential verification streams, and launches a background kernel monitor to trap any data corruption or hardware errors.

Create the script at /usr/local/bin/btrfs-fast-verify.sh, mark it as executable (chmod +x), and adjust the configuration header to match your drive names:

#!/bin/bash

# ==============================================================================
# CONFIGURATION - ADJUST TO MATCH YOUR SYSTEM
# ==============================================================================
MOUNT_POINT="/mnt/your_btrfs_raid5-6"
SUBVOLUME="@" 
PHYS_DISKS=("sda" "sdb" "sdc" "sdd" "sde" "sdf")
LUKS_MAPS=("map_d1" "map_d2" "map_d3" "map_d4" "map_d5" "map_d6")
# ==============================================================================

TARGET_PATH="${MOUNT_POINT}/${SUBVOLUME}"
DM_DIR="/dev/mapper"
ERR_LOG="/tmp/btrfs_scan_errors.log"

if [ "$EUID" -ne 0 ]; then
    echo "[-] Error: This script must be run as root (sudo)."
    exit 1
fi

cleanup() {
    echo -e "\n\n[*] Cleaning up background validation processes..."
    
    # Kill background dmesg monitor
    if [ -n "$DMESG_PID" ] && kill -0 "$DMESG_PID" 2>/dev/null; then
        kill "$DMESG_PID" 2>/dev/null
    fi

    # Kill any runaway parallel block reads if aborted early
    for pid in "${PIDS[@]}"; do
        if kill -0 "$pid" 2>/dev/null; then kill "$pid" 2>/dev/null; fi
    done

    if [ -s "$ERR_LOG" ]; then
        echo -e "\n==========================================================================="
        echo "CRITICAL: STORAGE ERRORS WERE DETECTED DURING ONLINE RUNTIME:"
        echo "==========================================================================="
        cat "$ERR_LOG"
        echo "==========================================================================="
    else
        echo -e "\n[+] Online validation complete. No errors detected in dmesg."
    fi
    rm -f "$ERR_LOG"
}
trap cleanup EXIT

echo "==========================================================================="
echo "          BTRFS LIVE ONLINE DATA & PARITY VALIDATOR (IONICE)                  "
echo "==========================================================================="

# 1. Optimize LUKS crypto pipelines (Ensures inline processing)
echo "[*] Ensuring optimized non-blocking LUKS paths..."
for map in "${LUKS_MAPS[@]}"; do
    cryptsetup refresh --perf-no_read_workqueue --perf-no_write_workqueue "$map" 2>/dev/null
done

# 2. Start background dmesg monitor to trap live filesystem/hardware errors
rm -f "$ERR_LOG"
dmesg -w | grep -iE "btrfs|checksum|corrupt|io error|ata|hardware|critical" > "$ERR_LOG" &
DMESG_PID=$!

echo -e "[+] System remains Read-Write. Verification running in background priority.\n"

# ==============================================================================
# PHASE 1: FILE DATA VALIDATION (LIVE FILESYSTEM)
# ==============================================================================
echo "------------------------------------------------------------------------------"
echo "PHASE 1/2: Verifying Active Files via BTRFS Checksums (Idle I/O Priority)..."
echo "------------------------------------------------------------------------------"
# 'ionice -c 3' drops the entire find+cat stream into the background Idle class
ionice -c 3 find "$TARGET_PATH" -type f -exec cat {} + \ 
  2>/dev/null | ionice -c 3 dd of=/dev/null bs=1M status=progress
echo -e "\n[+] Phase 1 Complete."

# ==============================================================================
# PHASE 2: PARITY (P/Q) & METADATA REPLICA SURFACE VALIDATION (LIVE BLOCK LAYER)
# ==============================================================================
echo "------------------------------------------------------------------------------"
echo "PHASE 2/2: Verifying P/Q Parities & Replicas via Parallel LUKS Reads (Live)..."
echo "------------------------------------------------------------------------------"
echo "[*] Launching low-priority block streams. Active applications take priority..."

PIDS=()
for map in "${LUKS_MAPS[@]}"; do
    # Read 100% of the mapped device linearly under ionice idle constraints
    ionice -c 3 dd if="${DM_DIR}/${map}" of=/dev/null bs=4M status=none &
    PIDS+=($!)
done

# Wait for all background block reads to finish
for pid in "${PIDS[@]}"; do
    wait "$pid"
done
echo "[+] Phase 2 Complete. All parity sectors and replicas verified."

Leave a Comment