Every page on this subject converges on the same sentence: device names are not stable, so use the UUID. That advice is correct about the problem it addresses, and it is almost always the last thing said on the subject.
A filesystem UUID is not an identity. It is a value written inside the filesystem, in the superblock, which means it is copied by every mechanism that copies a filesystem — cloning a VM, restoring a disk image, dd from one disk to another, attaching a snapshot beside its origin. When two attached disks carry the same UUID, /dev/disk/by-uuid/ is a directory, a directory cannot hold two entries of the same name, and so exactly one of the two disks gets a name. The other gets nothing at all — not a broken name, not a warning, no entry. Your fstab line then reaches whichever one won, and everything about the result looks correct.
Underneath that is a fact about the whole naming system, which this page is really about. Every name a Linux disk has is either a value stored inside the disk — and so copied by anything that copies the disk — or a description of the slot it is plugged into, and so changed by moving it. There is exactly one name that is neither, and it is reset at every boot and may not be used in fstab. No name your disk has is both stable across a move and unique across a copy. “Always use the UUID” answers the instability half of that and hands you the collision half without mentioning it.
This page follows one disk through the eight stages between the kernel noticing it and a person asking which physical device they are actually looking at. The mechanism finishes at stage five. The question does not.
The eight stages, and what each one can hand you.
- The device appears — the kernel assigns the only two names nothing in userspace can influence.
- The probe — udev reads two entirely different places on the disk and derives four families of name from them.
- Arbitration — one name, one slot, one winner. This is where a duplicate stops being visible.
- The lookup — something turns a name back into a device. There is one arbiter and everything on the machine is downstream of it.
- The mount — and the filesystem check that was aimed at the same symlink.
- The dependents — everything that now believes it knows which disk this is.
- The diagnosis — asking the machine which disk you actually got, and the commands that cannot answer.
- The other copy — the disk that also carries this name, which is somewhere else and is where the fault was created.
The reading. Not “what is this disk called” — every tool answers that. Ask instead: of the names udev gave this disk, do they all still point back at it?
dev=/dev/sdb1 # the device you are asking about
udevadm info -q symlink -n "$dev" | tr ' ' '\n' |
grep -v '^disk/by-diskseq/' |
while read -r l; do
printf '%-58s -> %s\n' "/dev/$l" "$(readlink -f "/dev/$l")"
doneOn a healthy disk every line ends in the device you named. Any line that does not is the finding, and it names the device that took the name.
| What comes back | What it means | Go to |
|---|---|---|
Every line resolves back to $dev | Nothing is contesting this disk’s names. Whatever is wrong is downstream of naming | Stage 5. The mount, the unit, the dependents |
A by-uuid or by-label line resolves to a different device | Two disks carry one identity and this one lost. The winner is named in the output | Stage 3, then stage 8. Something copied a filesystem |
Only by-diskseq — no by-uuid, no by-label, no by-partuuid | Nothing probed a filesystem here. There is no signature to derive a name from | Stage 2. blkid -p and wipefs -n on the raw device |
| It holds a name it did not hold yesterday | The previous holder was detached and the name moved rather than breaking | Stage 8. There is another copy and it has just left the building |
Two limits, stated up front. This needs udevadm, so it does not work in an initramfs, a rescue shell or a chroot — which are three of the places where the answer is different, because that is exactly where the symlinks are absent and a fallback scan chooses instead. And it tells you which device holds a name; it cannot tell you which of two identical filesystems is the one you wanted. That is stage 8, and it is off this machine by construction.
Stage 1 — the device appears, and the kernel names it
When the kernel registers a disk it assigns two things, and neither is negotiable from userspace.
loop0 7:0
loop0p1 259:0
loop0p2 259:1
diskseq: 24The major:minor pair is the primary key. Every other name on this page is an alias that resolves to it. diskseq is a counter that increases monotonically, is never reused within a boot, and resets to 1 when the machine reboots.
One correction worth making early, because the received belief is wrong in a way that matters: udev does not create /dev. devtmpfs creates the device node the moment the kernel registers the disk. udev then fixes its ownership and mode and adds symlinks. You can watch the handover in the permissions of a node udev has not processed yet — brw------- root root before, brw-rw---- root disk after. The consequence is that the node exists and is openable before any name for it does, which is why a program racing udev at boot can find a disk it cannot name.
Kernel names are recycled and diskseq is not. Attaching and detaching the same image five times:
attach 1: kernel name=/dev/loop0 diskseq=201 fs UUID=1feed34c-...
attach 2: kernel name=/dev/loop0 diskseq=203 fs UUID=1feed34c-...
attach 3: kernel name=/dev/loop0 diskseq=205 fs UUID=1feed34c-...
attach 4: kernel name=/dev/loop0 diskseq=207 fs UUID=1feed34c-...
attach 5: kernel name=/dev/loop0 diskseq=209 fs UUID=1feed34c-...The kernel name repeats. The filesystem UUID repeats. diskseq never does — it is the only guaranteed-unique name the machine has, and it is the one you are not allowed to use, because it means nothing after a reboot. Hold on to that: it is the first corner of the trilemma in the next stage.
Stage 2 — the probe, and where the names come from
udev’s job here is one rule, and it is the origin of everything that follows:
# 60-persistent-storage.rules
KERNEL!="sr*|mmcblk[0-9]boot[0-9]", IMPORT{builtin}="blkid"That builtin reads two entirely different places on the disk. It reads the filesystem superblock and sets the ID_FS_* properties; separately it reads the partition table and sets the ID_PART_ENTRY_* ones. The split is the whole story, and udevadm info lays it out:
ID_FS_LABEL=rootfs
ID_FS_TYPE=ext4
ID_FS_UUID=12345678-1111-1111-1111-111111111111 <- inside the filesystem
ID_PART_ENTRY_NAME=rootpart
ID_PART_ENTRY_UUID=aaaaaaaa-0001-0001-0001-000000000001 <- in the partition table
ID_PART_TABLE_UUID=11111111-2222-3333-4444-555555555555 <- in the GPT headerFive further rules turn those properties into symlinks, and these five lines are the entire naming system:
ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*",
SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"
ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_LABEL_ENC}=="?*",
SYMLINK+="disk/by-label/$env{ID_FS_LABEL_ENC}"
ENV{ID_PART_ENTRY_UUID}=="?*",
SYMLINK+="disk/by-partuuid/$env{ID_PART_ENTRY_UUID}"
ENV{ID_PART_ENTRY_SCHEME}=="gpt", ENV{ID_PART_ENTRY_NAME}=="?*",
SYMLINK+="disk/by-partlabel/$env{ID_PART_ENTRY_NAME}"
ENV{DISKSEQ}=="?*", ENV{ID_IGNORE_DISKSEQ}!="1",
SYMLINK+="disk/by-diskseq/$env{DISKSEQ}$env{.PART_SUFFIX}"Note crypto in the first two. That is why a LUKS container gets a by-uuid entry before anything has been unlocked, and why crypttab can use UUID= at all. A sixth family comes from a different builtin, explicitly skipped for virtual devices: path_id, which produces by-path.
Now the part that decides everything. For each family: where does the value physically live, and what changes it?
| Family | The value lives in | Survives a dd clone? | Survives a move to another port? | Allowed in fstab? |
|---|---|---|---|---|
by-uuid | The filesystem superblock | Yes — identically | Yes | Yes |
by-label | The filesystem superblock | Yes — identically | Yes | Yes |
by-partuuid | The GPT partition entry | Yes — identically | Yes | Yes |
by-partlabel | The GPT partition entry | Yes — identically | Yes | Yes |
by-id | The device’s own firmware | No — a clone is a different device | Yes | Yes |
by-path | The sysfs bus topology | No | No | Yes |
by-diskseq | The kernel, per boot | No | No | No — resets at boot |
Read the two “survives” columns together and the trilemma is plain. The four families anybody puts in fstab are content-derived, so a clone duplicates all four at once. The two that a clone cannot duplicate are descriptions of hardware, and by-path is a description of a slot:
$ udevadm test-builtin path_id /sys/block/vda
ID_PATH=pci-0000:00:02.0
$ udevadm test-builtin path_id /sys/block/vdb
ID_PATH=pci-0000:00:03.0Nothing in that came off the platter. by-path is unique and it is not stable — which is precisely the failure everybody correctly warns about for /dev/sdb, wearing a more respectable name. And the one name that is both unique and unforgeable, by-diskseq, is meaningless after a reboot. There is no fourth option.
One more table, because it answers most of the practical questions people actually have. Change one thing and see which names move:
| You do this | by-uuid | by-label | by-partuuid | Kernel name | diskseq |
|---|---|---|---|---|---|
tune2fs -L newname | unchanged | moves | unchanged | unchanged | unchanged |
tune2fs -U newuuid | moves | unchanged | unchanged | unchanged | unchanged |
mkfs the partition | new value | new value | unchanged | unchanged | unchanged |
| Detach and re-attach | unchanged | unchanged | unchanged | reused | new |
dd the whole disk | duplicated | duplicated | duplicated | n/a | distinct |
| Move it to another port | unchanged | unchanged | unchanged | may change | new |
The mkfs row is the one honest argument for PARTUUID= that survives everything else on this page: reformatting a partition gives the filesystem a new UUID and leaves the partition’s own GUID untouched, so PARTUUID= in fstab survives a reformat where UUID= does not. It buys you nothing at all against a clone.
Stage 3 — arbitration, and where a duplicate stops being visible
Take one ext4 filesystem, write a marker into it, unmount it, copy the whole disk — which is what dd if=/dev/sda of=/dev/sdb does — write a different marker into the copy, and attach both, settling udev between each so there is no race at all. This is the best case for the tooling.
ORIGINAL is /dev/loop0
/dev/disk/by-uuid/ right now:
aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee -> ../../loop0
CLONE is /dev/loop1
/dev/disk/by-uuid/ right now:
aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee -> ../../loop1The name moved the instant the second disk appeared. Then:
$ blkid -U aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
/dev/loop1
$ mount UUID=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee /mnt/payroll
$ cat /mnt/payroll/marker
I AM THE CLONE - stale copy from three months agoExit 0 throughout. findmnt on that mount point reports the device, the type, the UUID and the label, every column correct and every column useless.
Three things about that are worth more than the headline.
The winner is deterministic, and the rule is not the one people guess. Each symlink name is a directory under /run/udev/links/ holding one entry per claiming device with its link priority. At equal priority udev takes the most recently processed device. Measured over ten rounds, five with the clone attached first and five with the original first: the second device to be processed won every time. So it is not unpredictable given a fixed order — it is a deterministic function of an enumeration order that is itself a race between disks spinning up at speeds nobody controls.
udev knows there are two, and records it. Its own claimant database has both:
$ ls -l '/run/udev/links/disk\x2fby-uuid\x2faaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee/'
b7:0 -> 0:/dev/loop0
b7:1 -> 0:/dev/loop1Two claimants, both at link priority 0. The information exists on the machine. It is simply not in /dev, which is the only place anybody is told to look.
Unplugging the winner does not break the name — it moves it. Detach the disk holding the symlink and udev silently re-points the name at the surviving claimant. The link does not dangle. It is not stale. It resolves cleanly to a different disk, and the next mount succeeds against different data with no error at any point in the transition. That is the mechanism behind an entire class of incident nobody investigates, because nothing failed.
Stage 4 — the lookup, and the single arbiter
The received picture is that mount UUID= searches the block devices for that UUID. It does not, when udev is running. blkid(8) documents the order in its configuration section: EVALUATE=udev,scan, meaning the udev method — /dev/disk/by-* — is tried first, and only if that fails does anything scan /proc/partitions. There is no /etc/blkid.conf on a stock install, so the compiled-in default applies.
The discriminating test is to force the symlink at the disk udev did not choose, then delete it, changing nothing else:
--- STEP 1: point the symlink at the OTHER disk (/dev/loop0) ---
symlink now -> /dev/loop0
blkid -U -> /dev/loop0
findfs -> /dev/loop0
mount UUID= -> /dev/loop0, containing DISK-ONE
--- STEP 2: delete the symlink. Nothing else changes. ---
symlink -> (absent)
blkid -U -> /dev/loop1
findfs -> /dev/loop1
mount UUID= -> /dev/loop1, containing DISK-TWOSame fstab tag, same two disks, different data mounted, with nothing changed but the presence of one symlink. And an absent symlink is not exotic: it is the state of every initramfs before udev has settled, every rescue shell, every chroot, and every container. The same fstab line can reach different disks depending on where you are standing when it runs.
systemd is on the same side of the fence. systemd-fstab-generator never resolves a UUID itself. Run against a four-line fstab using four different spellings, it emits:
mnt-payroll.mount What=/dev/disk/by-uuid/aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
mnt-bylabel.mount What=/dev/disk/by-label/payroll
mnt-bypart.mount What=/dev/disk/by-partuuid/aaaaaaaa-0001-0001-0001-000000000001
mnt-bydev.mount What=/dev/sdb1All four spellings become a path, and three of them a path under /dev/disk/. systemd waits for udev to publish that path and then trusts it completely. So the architecture is one arbiter with one slot per name, and every consumer on the machine downstream of it — mount, blkid -U, findfs, and the generated unit alike. That is good news in one specific way: the question “which disk will this line reach?” has a single answer you can look up right now, rather than a race you have to reason about.
Stage 5 — the mount, and the check that ran first
The generated unit carries more than What=. It also carries an ordering dependency on the device and a filesystem check:
After=blockdev@dev-disk-by\x2duuid-aaaaaaaa\x2d....target
Requires=systemd-fsck@dev-disk-by\x2duuid-aaaaaaaa\x2d....serviceRead the second line for what it means in the duplicate case: the boot-time fsck is aimed at the symlink too. It does not check the filesystem you meant. It checks whichever filesystem currently holds that name, and it does so before the mount, which is to say it will happily repair the clone.
Then mount(2) returns 0, the kernel attaches a superblock at a mount point, and /proc/self/mountinfo gains a line. As far as the operating system is concerned this story is finished. Both disks are mounted, both are healthy, nothing has failed.
Stage 6 — the dependents
Once a filesystem is mounted, a queue of things forms behind it, each of which now believes it knows which disk this is. They are worth listing because each one inherits the naming decision without re-checking it, and because they are where the consequences of stage 3 actually get paid.
- Everything in
fstabunderneath it. A mount orderedAfter=this one — a bind mount, a nested data directory, ax-systemd.requires=chain — attaches to whatever came up, not to what you meant. - Every service with
RequiresMountsFor=on that path. The dependency is satisfied by a mount at that path, not by the right one. A database that starts against a three-month-old copy of its data directory starts cleanly. - Your backup job. It writes into a path, and the path exists, and the write succeeds.
crypttab, which uses the same namespace. A LUKS container gets aby-uuidentry before it is unlocked, and unlocking it creates a second name —/dev/mapper/<name>— in a namespace udev does not arbitrate at all, because that name was chosen by whoever wrotecrypttab.- LVM, which is the one layer here that anticipated this. See below.
What each of those layers does — LVM as address translation, dm-crypt as AES-XTS, RAID and the write hole — belongs to The Life of a Write, which follows the same stack as a data path. What matters here is only what each layer is called, and one measured contrast is worth the whole stage.
Attach two disks carrying the same identity and ask each subsystem what it sees. ext4 says nothing: lsblk -f prints two rows with the same UUID and no marker, blkid prints two lines, both mount, both exit 0. btrfs says nothing at the identity layer — two rows with the same UUID, while btrfs filesystem show lists one filesystem with Total devices 1 and names only the first disk. LVM says this, unprompted, on every pvs, vgs and lvs you will ever run:
WARNING: Not using device /dev/loop1 for PV eeee00-0000-...-000004.
WARNING: PV eeee00-0000-...-000004 prefers device /dev/loop0 because
device name matches previous.
PV VG Fmt Attr PSize PFree
/dev/loop0 shared_vg lvm2 a-- 60.00m 60.00mIt names the duplicate, names the device it rejected, names the device it chose, and states its tie-breaking rule out loud. That is worth carrying because of what it does to the conclusion. “Linux is silent about duplicate identities” sounds like a law of nature and invites fatalism. It is not a law. It is one layer handling a situation its authors anticipated and another layer not handling it, and the difference between those two sentences is the difference between shrugging and knowing which command to run.
Stage 7 — the diagnosis, and what none of it can see
Nine fstab entries, set up simultaneously on one machine — one correct and eight broken in different ways — then every recommended command run once across all nine.
- F1 control, correct in every way · F2 duplicate UUID, two attached disks carry it · F3 the disk is not plugged in · F4 a typo, last character wrong · F5 wrong
fstype, disk is ext4 andfstabsays xfs - F6 duplicate
LABEL, two disks labelledbackup· F7 the mount point directory does not exist · F8 shadowing, the mount point already contains files · F9 the signature was wiped afterfstabwas written
First, three completely different situations produce a byte-identical message.
mount: /mp/f3: can't find UUID=33333333-0000-0000-0000-000000000003.
mount: /mp/f4: can't find UUID=44444444-0000-0000-0000-00000000000X.
mount: /mp/f9: can't find UUID=88888888-0000-0000-0000-000000000008.The disk is not connected, you made a typo, and the filesystem was destroyed differ only in the UUID quoted back at you. mount cannot tell them apart because from its side they are one condition: the tag did not resolve. Nothing in the message distinguishes “go and look for the disk” from “go and look at your fstab” from “go and look for your data”. findmnt --verify collapses all three the same way, to [E] unreachable on boot required source.
Second, and this is the finding: four of the nine are indistinguishable, and three of those four lose or hide data. Here is findmnt --verify --verbose, the standard pre-reboot check, on four of the entries:
/mp/f1 <- CONTROL, correct
[ ] target exists
[ ] UUID=11111111-... translated to /dev/loop0
[ ] source /dev/loop0 exists
[ ] FS type is ext4
/mp/f2 <- DUPLICATE UUID
[ ] target exists
[ ] UUID=22222222-... translated to /dev/loop2
[ ] source /dev/loop2 exists
[ ] FS type is ext4
/mp/f6 <- DUPLICATE LABEL
[ ] target exists
[ ] LABEL=backup translated to /dev/loop6
[ ] source /dev/loop6 exists
[ ] FS type is ext4
/mp/f8 <- SHADOWING
[ ] target exists
[ ] UUID=77777777-... translated to /dev/loop7
[ ] source /dev/loop7 exists
[ ] FS type is ext4Four identical reports, four ticks each. One is healthy. When the machine boots, the other three mount a stale clone, mount the wrong one of two disks labelled backup, and hide a directory of files that was already there. With only the duplicate entry present, the same command ends with Success, no errors or warnings detected and exits 0.
This is worse than silence. A check that says nothing leaves you where you were; a check that certifies moves you further from the answer, because you now have a reason to stop looking. And findmnt --verify cannot fix this without being rebuilt: it asks libblkid to translate the tag, and libblkid’s answer is a single device path. There is nowhere in that conversation for the word “two” to appear.
The whole grid, scored:
| Command | F1 ok | F2 dup UUID | F3 absent | F4 typo | F5 fstype | F6 dup LABEL | F7 no dir | F8 shadow | F9 wiped |
|---|---|---|---|---|---|---|---|---|---|
lsblk -f | clean | two rows, same UUID, unflagged | absent | clean | clean | two rows, same LABEL | clean | clean | blank row |
blkid | listed | two lines, same UUID | absent | listed | listed | two lines | listed | listed | absent |
ls /dev/disk/by-uuid/ | 1 entry | 1 entry | absent | absent | 1 entry | 1 each | 1 entry | 1 entry | absent |
findmnt --verify | 4 ✓ | 4 ✓ | [E] | [E] | [W] | 4 ✓ | [E] | 4 ✓ | [E] |
findmnt --fstab --evaluate | device | device | blank | blank | device | same device twice | device | device | blank |
mount -a --fake | ok | ok | fails | fails | “mounted” | ok | “mounted” | ok | fails |
mount -a | ok | ok | fails | fails | fails | ok | fails | ok | fails |
systemd-analyze verify | silent | silent | silent | silent | silent | silent | silent | silent | silent |
Five things fall out of that grid, and they are most of what you need to know about diagnosing this.
lsblk -f contains the answer and does not present it. Two rows carry the same thirty-six-character UUID. Finding that requires cross-referencing every row against every other row by eye, in a column nobody reads left to right. It is the single most useful command in this area, and being the most useful is not the same as being sufficient. blkid is worse for this purpose than it looks, because its output order is cache order rather than device order — two identical UUIDs four unsorted lines apart is a poor way to notice a collision.
mount -a --fake is more reassuring than the real thing. It reported successfully mounted for the entry whose fstype was wrong and for the entry whose mount point did not exist — both of which the real mount -a refuses. It resolves the tag and stops. A dry run that passes where the real run fails is not a dry run.
systemd-analyze verify said nothing about any of the nine — including the generated unit containing What=/dev/disk/by-uuid/NOT-A-UUID-AT-ALL — and exited 0.
The best tool here is the one that rarely gets mentioned. findmnt --fstab --evaluate prints the device each line will actually reach, and leaves the source blank for the lines that will not resolve — which cleanly separates the three that produce the identical error message. On the duplicate-label entry it printed the same device on two different lines, which is visible. Add it to whatever you run before a reboot:
findmnt --fstab --evaluate # what each line actually reaches
findmnt --verify --verbose # the checks it can do, which are not theseAnd nothing at all reports the shadowing case. F8 is the only failure in the set that destroys nothing and hides everything, and not one command in the grid mentions it. findmnt --verify had the mount point in its hand and did not look inside it.
Stage 8 — the other copy
Stages 1 to 7 all happen on the machine in front of you. This one does not, and that is the whole reason it is the last stage rather than a footnote.
A duplicate identity is never created on the machine that suffers it. It is created by a clone, a restore or a dd performed somewhere else — possibly months earlier, possibly by somebody who has since left. Everything before this stage yields “here is how to find out which disk you got”. This stage is the only one that yields a way to stop it happening again, and there is exactly one instruction: the moment a second copy of a filesystem exists, one of the two needs a new UUID before both are ever attached to the same machine.
The command everybody publishes for that does not work on a current system:
$ sudo tune2fs -U random /dev/sdc1
tune2fs 1.47.0 (5-Feb-2023)
Setting the UUID on this filesystem could take some time.
Proceed anyway (or wait 5 seconds to proceed) ? (y,N) y
This operation requires a freshly checked filesystem.
Please run e2fsck -fD on the filesystem.On e2fsprogs 1.47.0 it refuses on any filesystem last mounted more recently than it was last checked, which is every filesystem you have ever actually used. The working sequence is two commands, and the check is not optional:
sudo e2fsck -f /dev/sdc1 # must be unmounted
sudo tune2fs -U random /dev/sdc1
sudo blkid /dev/sdc1 # write the new value into fstabThe reason for the refusal is worth one sentence, because it tells you when it will not happen: with metadata_csum the UUID is folded into every metadata checksum on the filesystem, so changing it means rewriting all of them, and the tool declines to do that to a filesystem it has not just verified. A filesystem created with -O metadata_csum_seed stores the seed once instead, and changes its UUID instantly. If you build images that get cloned, that flag is worth setting at mkfs time.
Then update every reference. The new UUID has to reach fstab on the machine that will use the disk, crypttab if there is a container involved, and any RequiresMountsFor= or x-systemd.requires= that names the old one. This is the part where a page usually says “and you are done”, and it is worth saying instead that you are not: the old value is still inside the other copy, wherever that is.
This site ends another flagship at the same place for the same reason. Restoring a Linux Server closes on “the other copy, and everyone who cannot tell them apart” — but that page is about machine-id, SSH host keys and network identity, which is a different namespace with a different fix. The division is clean and worth stating: that page owns two machines the network cannot tell apart; this one owns two block devices that one machine cannot tell apart. If you have just restored a server from an image, you have probably created both at once.
A worked diagnosis
A backup disk is mounted at /srv/archive. The mount is clean, the machine has been up for weeks, and the newest file in there is from June. Nothing has failed and nothing has alerted.
$ findmnt /srv/archive
TARGET SOURCE FSTYPE OPTIONS
/srv/archive /dev/sdc1 ext4 rw,relatime
$ sudo findmnt --verify --verbose | tail -6
/srv/archive
[ ] target exists
[ ] UUID=9f2c1a44-... translated to /dev/sdc1
[ ] source /dev/sdc1 exists
[ ] FS type is ext4
Success, no errors or warnings detectedStage 7 answers, and its answer is worth nothing. Four ticks and a success line. Note what has actually been established: that the tag resolves to a device, and that the device exists. Not that it resolves to the device you meant, and not that it resolves uniquely. Those are different questions and this command asks neither.
$ dev=/dev/sdc1
$ udevadm info -q symlink -n "$dev" | tr ' ' '\n' | grep -v '^disk/by-diskseq/' |
while read -r l; do printf '%-58s -> %s\n' "/dev/$l" "$(readlink -f "/dev/$l")"; done
/dev/disk/by-uuid/9f2c1a44-3b7e-4c19-8a05-2d6f1e904bb2 -> /dev/sdd1
/dev/disk/by-label/archive -> /dev/sdd1
/dev/disk/by-partuuid/7c1e0a92-0001-... -> /dev/sdd1
/dev/disk/by-id/wwn-0x5000c500a1b2c3d4-part1 -> /dev/sdc1
/dev/disk/by-path/pci-0000:00:1f.2-ata-3-part1 -> /dev/sdc1The reading, and it is unambiguous. Three of this disk’s five names are held by a different device. The two it still owns are exactly the two that are not derived from its contents — its firmware serial and its bus slot. That pattern is the signature of a clone: everything written inside the disk is contested, everything about the hardware is not. And note which way round it is — /dev/sdd1 holds the names, so /dev/sdd1 is what an fstab line naming that UUID reaches. The mounted disk is the loser, mounted by something that named it directly.
$ ls -l '/run/udev/links/disk\x2fby-uuid\x2f9f2c1a44-3b7e-4c19-8a05-2d6f1e904bb2/'
b8:33 -> 0:/dev/sdc1
b8:49 -> 0:/dev/sdd1
$ sudo mount -o ro /dev/sdd1 /mnt/check && ls -t /mnt/check | head -2
2026-09-05.tar.zst
2026-09-04.tar.zstStage 3 has the whole story. Two claimants at equal priority, and the backups have been running perfectly — onto /dev/sdd1, which is the disk that holds the name. Somebody added a second drive in June and seeded it by cloning the first. Since then the timer has written to whichever disk the fstab line reached and a human has been reading the one that was already mounted, and both have been correct about everything they were asked.
The fix is stage 8, in order. Unmount both. e2fsck -f then tune2fs -U random on the one you are keeping as the second copy. Put the new UUID in fstab. Then, before rebooting, findmnt --fstab --evaluate — because that is the command that would have shown two lines reaching one device, and the one that will now show two lines reaching two.
What everybody says, and what it actually matches
Measured on Ubuntu 24.04, systemd/udev 255.4, util-linux 2.39.3 and e2fsprogs 1.47.0.
| The advice | What is actually true | Stage |
|---|---|---|
| Always use the UUID — it is the stable, authoritative identity of a filesystem | It is stable and it is not unique. It is a value inside the superblock, so dd, a VM clone, an image restore and a snapshot all reproduce it exactly, and /dev/disk/by-uuid/ publishes one entry per name | 2, 3 |
Use LABEL=, it is just as stable and easier to read | Strictly worse. Labels come from a small human vocabulary, so two disks both labelled backup collide exactly as duplicate UUIDs do and are far likelier to | 2 |
PARTUUID= avoids the problem, being in the partition table rather than the filesystem | A whole-disk dd copies the partition table, so PTUUID, PARTUUID and PARTLABEL duplicate along with everything inside. Its one real advantage is surviving a mkfs of the partition | 2 |
mount UUID= searches the block devices for that UUID | It reads /dev/disk/by-uuid/ first and only scans if that fails — EVALUATE=udev,scan in blkid(8). Which is why one fstab line can reach different disks inside and outside an initramfs | 4 |
blkid is the authoritative view of what is on the disks | blkid -U and bare blkid go through the same udev-first evaluation and a cache under /run. blkid -p is the only form that reads the device | 4 |
If you get a duplicate, just run tune2fs -U random | It refuses on any filesystem mounted more recently than it was checked: This operation requires a freshly checked filesystem. The working sequence is e2fsck -f and then tune2fs -U random | 8 |
findmnt --verify checks your fstab before you reboot | It checks target existence, tag resolution, source existence and type. It returns Success, no errors or warnings detected on a duplicate UUID, a duplicate label, and a mount point that already holds data | 7 |
mount -a --fake is the safe dry run | It resolves the tag and stops. It reported successfully mounted for an entry with the wrong fstype and one whose mount point did not exist — both of which the real mount -a refused | 7 |
| If the disk goes away the symlink will dangle and you will notice | It does not dangle. udev re-points it at the surviving claimant and the next mount succeeds against a different disk, with no error at any point | 3 |
udev creates the device nodes in /dev | devtmpfs creates the node when the kernel registers the disk; udev then fixes mode and ownership and adds symlinks. A node udev has not reached yet is already openable | 1 |
The one error, and everything it generates
Every row in that table comes out of one sentence that nobody says aloud because it seems too obvious to state: the UUID is the disk’s identity.
It is not an identity. It is a value stored inside the thing it claims to identify, which means it is reproduced by every mechanism that reproduces the thing. And /dev/disk/by-uuid/ is not a registry of identities; it is a directory of names with one slot each, arbitrated by a daemon on a first-come-last-served basis. Believe it is an identity and the rest writes itself. If it is an identity it is unique, so two disks cannot both have it, so one tidy entry in that directory means there is one disk — which makes a correct-looking listing an all-clear. If it is an identity, mount UUID= finds the disk, so the question “which disk did I get?” never comes up. If it is an identity, a check that resolved it has verified something, so a green tick from findmnt --verify means the right disk. If it is an identity, unplugging the disk must break the name, so the absence of a dangling symlink means nothing moved. And if it is an identity, copying a disk produces a different disk with a different one, so a clone is a new thing rather than a second claimant. Every one of those follows perfectly. Every one is wrong, and every one fails in the same direction — towards a machine that looks fine.
The one-sentence test for anything else written on this subject: when two attached disks carry the same filesystem UUID, which one does /dev/disk/by-uuid/ point at, and what happens to that symlink when you unplug it? A page written from memory answers the first half with “it is undefined” or “whichever udev saw first” and does not answer the second half at all, because the second half only occurs to somebody who has watched it. A page written from a machine says: the last device processed, deterministically; both claimants are recorded under /run/udev/links/, so the machine knows and /dev does not; and unplugging the winner does not dangle the link, it silently re-points it at the loser.
Three things nothing on your machine can tell you, which is the honest shape of this subject:
- Whether the filesystem you mounted is the copy you wanted. Both are real, both are healthy, both answer to the same name, and the difference between them is a fact about your intentions.
- How many other copies of this identity exist. udev records the claimants that are attached; a clone in a drawer, an image in object storage or a snapshot in another datacentre is not on that list and cannot be.
- Which of two disks will win after the next reboot. The answer is the enumeration order, and that is decided by how fast each piece of hardware answers.
And udev has already half-fixed this, which is worth knowing because almost nobody uses the half that exists. Its designers did not overlook collisions — they built for them twice. /run/udev/links/ is a claimant database that records every device wanting a name, with its priority, which is why stage 3 could show you both disks. And udev(7) documents a knob for deciding the contest deliberately rather than by arrival order:
# /etc/udev/rules.d/10-local-disk-priority.rules
ENV{ID_SERIAL_SHORT}=="WD-1A2B3C4D", OPTIONS+="link_priority=100"Highest priority wins, and now the winner is a decision rather than a race. It is half a fix, and the missing half is the honest part: the knob requires you to already suspect there is a contest, and the database that would tell you is in /run rather than in /dev, documented nowhere the man page for the knob points at. The mechanism for deciding is documented; the place that records that there is something to decide is not.
Before you call it done
findmnt --fstab --evaluaterun before the reboot, and every line reaching a different device- The hinge run against any disk you have cloned, restored or snapshotted — all of its names resolving back to it
- Any second copy given a new UUID with
e2fsck -fthentune2fs -U random, before both are ever attached to one machine nofailandx-systemd.device-timeout=on every non-essential entry, so a missing disk is a warning rather than an emergency shell- The mount point checked to be empty before it is mounted over — nothing else on the machine will tell you
- A
mkfsthat will be cloned given-O metadata_csum_seed, so the copy’s UUID can be changed in a second rather than a full check - Something that watches the content — the age of the newest file — rather than the mount, because a mount succeeding is not evidence about which disk it reached
Related reading
- Disks and Mounting — lsblk, mount and fstab — the reference for the commands, the
fstabfields and the options worth knowing - The Life of a Write — the same stack read as a data path: what LVM, dm-crypt and RAID actually do to your bytes, where this page only says what they are called
- How a Path Becomes a File — symlink resolution itself: the link budget,
O_NOFOLLOW, and why the same string can name two different things - From Power-On to a Login Prompt — the loud version of this failure: a UUID in
fstabthat exists nowhere, and the emergency shell it produces - Restoring a Linux Server — the other copy, in the other namespace: two machines the network cannot tell apart
- Filesystems: Inodes, Journalling and Which One to Choose — what is inside the thing being named
- Disk Space — df, du and ncdu — for when the disk that mounted is the right one and is simply full