Modern Linux machines survive a power cut without the long, frightening filesystem check that older systems needed. The journal is why. But people often assume it means “nothing is lost”, and that is not what it promises — it promises that the filesystem is consistent, which is a much narrower claim than that your data is there.
The gap between those two is where surprising data loss lives.
The problem a journal solves
Almost nothing a filesystem does is a single write. Appending a block to a file means updating the free-space map, writing the data, updating the inode’s block list, and updating its size and timestamps — four separate changes to four separate places on disk.
Lose power halfway through and the filesystem is left in a state that cannot be true: a block marked in use that belongs to no file, or an inode claiming a block that the free map also offers to the next writer. That second one is genuine corruption — two files eventually sharing one block.
The old fix was fsck, which walked the entire filesystem after a crash looking for such contradictions. That is thorough, and on a large disk it takes a very long time.
How the journal works
Before touching the real structures, the filesystem writes a description of what it is about to do into a small reserved area — the journal — and waits for that to be safely on disk. Then it makes the real changes. Then it marks the journal entry complete.
After a crash, recovery reads only the journal, which takes seconds:
- An entry marked complete — nothing to do
- A complete entry not yet marked done — replay it, finishing the operation
- A half-written entry — discard it; the operation never began
Every operation therefore either happened entirely or not at all. The filesystem is never left in an impossible state, and that is the whole guarantee.
Which parts get journalled
Writing everything twice would halve the write speed of the disk, so by default only metadata — the structure — goes through the journal. File contents are written straight to their final location.
| ext4 mode | Journals | Trade-off |
|---|---|---|
data=ordered (default) | Metadata, but data is written first | Good balance; no stale-block exposure |
data=writeback | Metadata only, any order | Fastest; a file can briefly contain old disk contents |
data=journal | Metadata and data | Safest; roughly halves write throughput |
data=ordered exists because of a nasty edge case in writeback: if the metadata says a file is now 4 KB longer but the data write never landed, that file contains whatever was previously in those blocks — potentially another user’s deleted data. Ordering the data write before the metadata commit closes that hole at almost no cost, which is why it is the default.
Btrfs and ZFS take a different approach entirely. They never overwrite live data: changes are written to new locations and a single atomic update switches to the new tree. Same guarantee, no journal.
Why your file was empty anyway
A journal protects the filesystem’s structure, not your last thirty seconds of work. When a program calls write(), the data goes into the page cache in RAM and the call returns immediately — successfully. The kernel writes it out later, typically within thirty seconds. A power cut in that window loses it, and the filesystem is entirely consistent afterwards: it never promised that write was durable, only that its own bookkeeping would not be corrupt. The only thing that makes a write durable is fsync(), and the program has to call it.
This is why a database is slower than copying the same bytes with cp. Postgres calls fsync() at every commit and waits for the disk to confirm; cp does not wait for anything. The difference in speed is the difference in what each is promising you.
The safe way to replace a file, which most well-written software uses:
# write to a temporary file, flush it, then rename over the original
# rename is atomic - after a crash you have either the old file or the new one
write config.new -> fsync(config.new) -> rename(config.new, config)Without the fsync step, a crash can leave the rename recorded while the new file’s contents are not, producing a zero-length config file where a working one used to be — which was a well-known way to lose files on early ext4 and is the reason the pattern is spelled out so carefully now.
From the shell, sync flushes everything pending:
sync # flush all pending writes, then return
sync -f /mnt/usb # just that filesystem
cat /proc/meminfo | grep -i dirty # how much is waiting right nowA large Dirty: figure is the answer to “why did unmounting the USB stick take forty seconds” — the copy finished in the page cache long before it finished on the device.
The layer below: disks that lie
All of this assumes that when the kernel tells the disk to flush, the disk really does. Drives have their own volatile write caches, and a cheap one may report success as soon as the data is in that cache. If power fails a moment later, the filesystem’s careful ordering was undone by hardware that lied about it.
sudo hdparm -W /dev/sda # is the write cache on
cat /sys/block/sda/queue/write_cacheReputable drives honour cache-flush commands, and enterprise SSDs have capacitors that let them finish writing after power is gone. Cheap consumer SD cards and USB sticks are the usual offenders, which is why a Raspberry Pi on an SD card is far more likely to come back corrupted than a server is.
A RAID controller with battery-backed cache is the same idea in hardware: it can safely acknowledge a write before the platters have it, because it can complete it after a power failure.
Checking and repairing
# NEVER on a mounted filesystem - it will corrupt it
sudo fsck -n /dev/sdb1 # report only, change nothing
sudo fsck /dev/sdb1 # actually repair, unmounted
sudo tune2fs -l /dev/sda1 | grep -iE 'state|mount count|check'
sudo dumpe2fs -h /dev/sda1 | grep -i journalJournal replay happens automatically at mount, so a clean boot after a power cut has already done the work. You only need fsck when something is actually wrong — and if it puts files in lost+found, those are inodes with valid data and no directory entry pointing at them, named after their inode number.
The mount option worth knowing is errors=remount-ro, which flips a filesystem to read-only the moment the kernel detects inconsistency, rather than continuing to write into the damage. It is the default on most distributions and is the right choice.
Symptoms and what they mean
| Symptom | What happened | What to do |
|---|---|---|
| File empty or truncated after a power cut | Data was still in the page cache | Nothing to repair; the application needed fsync |
| Filesystem mounted read-only unexpectedly | errors=remount-ro triggered | dmesg, then unmount and fsck |
| Unmounting a USB stick takes ages | Dirty pages flushing now | Normal — wait for it |
Files appear in lost+found | Inodes recovered without a name | Identify by content; check backups |
| Boot pauses for a long check | Mount count or interval exceeded | Normal maintenance; tune2fs -c |
| Repeated corruption on a Pi or SD card | Hardware ignoring flushes | Better media; move writes off the card |
| Database far slower than file copies | fsync on every commit | Correct behaviour — that is the durability |
fsck on a mounted filesystem made it worse | Repairing under a live kernel | Only ever unmounted, or from rescue media |
The first row is the one people misread most. An empty file after a crash is usually not filesystem damage and not something fsck can help with — it is the honest consequence of a write that was never made durable, on a filesystem that kept every promise it made.
Related reading
- How filesystems work — inodes, blocks and directory entries
- Disks and mounting — mount options, including
errors=remount-ro - Memory, swap and the OOM killer — the page cache these writes sit in
- Automated backups — the only real answer to lost data
- Putting a database somewhere sensible — where durability is the whole point