Somebody types a password and gets a shell. Between those two events sit seven stages, several configuration systems that do not know about each other, and a set of decisions that determine not just whether the login succeeds but what kind of session it is — which is the part nobody explains and the part that generates the confusing bug reports.
This page goes underneath users, groups and root, which covers who the accounts are. This one covers what happens when one of them arrives. It follows an SSH login throughout, deliberately: that is where servers are actually logged into, where readers actually debug, and where the stacks are clean enough to inspect. A display manager runs the same machinery with three differences — it registers a session of its own before yours, it knows a seat and a virtual terminal where SSH knows neither, and it usually adds a keyring module to the session stack — and those are called out where they matter.
Two things are delegated. What the kernel does to turn a process into a running shell — fork, exec, the dynamic linker — is its own page. The credential model underneath it all, capabilities and SELinux included, is another. This page is about the path in.
A great deal of what is written about this subject describes PAM roughly as it was documented in 2012. The specific tells are listed at the end, and there is a heuristic for spotting them in three seconds.
The seven stages, and two questions that cut through them
- Something asks for authentication — and on a modern OpenSSH that is three separate binaries.
- PAM reads its configuration and runs the
authstack — the only stack that can be skipped entirely. - The user is looked up —
passwd,groupandshadowthrough the name service switch. - Credentials are verified — and the algorithm is chosen somewhere you are not looking.
- The
accountstack decides whether this account may log in at all, right now. - The
sessionstack builds the session — and decides what kind it is. - The session ends, and something decides what survives it.
Question one, which splits the spine. The account and session stacks always run; the auth stack does not. So if you got a shell, the account stack returned success — that is a definition, not a heuristic — and any symptom that survived into your shell belongs to stage 6 or later. But you can conclude nothing about the auth stack, because a public-key SSH login never runs it.
That is not a nitpick. It is the whole explanation for the most common confused report in this subject: “I added a lockout module / a nologin check / a second factor to common-auth and nothing happens over SSH.” Of course nothing happens. You are logging in with a key.
Question two, which resolves nearly every “my user services are broken”. Those symptoms — no XDG_RUNTIME_DIR, no keyring, no session bus, systemctl --user refusing to connect — are not several problems. They are one problem with several faces, and pam_systemd(8) states the cause in a single sentence: $XDG_RUNTIME_DIR is not set if the current user is not the original user of the session.
It is bound to the session, not to your uid. So one command decides it:
echo "$XDG_SESSION_ID"
loginctl session-status "$XDG_SESSION_ID"Read the Class: line. user means a full session with a service manager. Any -light class means logind registered you and deliberately did not start your service manager, so systemctl --user fails by design. No session id at all means whatever you are in never registered one.
Stage 1 — Something asks for authentication
PAM does nothing on its own. A program has to call it, and which program called it decides which configuration file gets read — the file is named after the service, not after the user or the method.
Start with the fact that invalidates the first command most people run.
sshd is three binaries now
OpenSSH 9.8, in July 2024, split the server into a listener called sshd and a per-session binary called sshd-session, so that the process listening on the network no longer has to contain the protocol implementation. OpenSSH 10.0 went further and moved the user-authentication phase out again, into sshd-auth. The current release is 10.5, from August 2026.
The release notes said plainly that some log messages would start being tagged with a different process name, and that is exactly what happened:
# What every stale article tells you to run. Returns almost nothing useful.
journalctl _COMM=sshd
# What actually catches the login
journalctl _COMM=sshd _COMM=sshd-session _COMM=sshd-auth
journalctl -u ssh -u sshd # the unit name differs by distribution
journalctl SYSLOG_FACILITY=10 # everything logged as "auth"That first line silently returning nothing is the reason people conclude their PAM configuration is not being read. It is being read. The filter is looking at the wrong process.
What “PAM-aware” actually buys you
Two things hide in sshd_config‘s description of UsePAM, and both matter.
Its upstream default is no. Every distribution sets it to yes, so nobody notices — until somebody builds OpenSSH from source and cannot work out why their PAM configuration is inert.
And it enables PAM authentication only for password and keyboard-interactive logins, while enabling account and session processing for all authentication types. That asymmetry is question one from the box above, and it is written into the manual page in one sentence that almost nobody reads carefully.
One machine, five different service files
| You ran | PAM service file |
|---|---|
su bob | su |
su - bob | su-l — a different file, and almost no article mentions it |
sudo -i | sudo — not su, not login |
| An SSH login | sshd |
| A console login | login |
One more thing in this stage, for anyone running Ubuntu: it replaced sudo with sudo-rs, a Rust reimplementation, in 25.10, and that is the default in 26.04 LTS. The user-visible consequences are documented: the password prompt reads differently, I/O logging and sudoreplay are not supported, and there is no sudoers.ldap — directory-backed sudo rules have to come through PAM instead. Any advice about replaying recorded sudo sessions no longer applies there.
Stage 2 — PAM reads its configuration and runs the auth stack
PAM opens the file named after the service, reads a list of modules, and calls them in order. The current implementation is Linux-PAM 1.7.2. Two things about that file have changed, and the first one breaks the standard instruction for finding it.
“It’s in /etc/pam.d/” is now a half-truth
Since Linux-PAM 1.7.2, the vendor directory is enabled by default. If a service file is absent from /etc/pam.d/, the library falls back to /usr/lib/pam.d/. Files in /etc/pam.d/ override same-named files elsewhere, so the layering is the familiar one — but the file you need to read may simply not be in /etc.
openSUSE has already moved, instructing packagers to install into /usr/lib/pam.d as ordinary files rather than marking them as configuration. Others will follow. Look in both:
ls -l /etc/pam.d/sshd /usr/lib/pam.d/sshd 2>&1
grep -r pam_systemd /etc/pam.d/ /usr/lib/pam.d/ 2>/dev/nullThe four control flags are shorthand, and that is why the file confuses you
Every introduction teaches required, requisite, sufficient and optional. Then the reader opens Debian’s common-auth and finds [success=1 default=ignore], which is in none of those four, and concludes the documentation is wrong.
It is not. The four names are abbreviations for bracketed forms, and the manual page gives the exact equivalences:
required = [success=ok new_authtok_reqd=ok ignore=ignore default=bad]
requisite = [success=ok new_authtok_reqd=ok ignore=ignore default=die]
sufficient = [success=done new_authtok_reqd=done default=ignore]
optional = [success=ok new_authtok_reqd=ok default=ignore]Read that and the bracketed forms stop being mysterious. bad records a failure but carries on; die fails immediately; done succeeds immediately; ignore does not affect the result; and a bare number means skip that many modules, which is how a distribution writes “if this one worked, jump past the fallbacks”.
include and substack are also control values, and the difference between them is the detail most articles omit. Both pull in another file. But in a substack, a done or die action stops only the substack — it does not skip the rest of the enclosing stack. With include, it skips everything. That is precisely the distinction that decides whether your added module runs.
A stale detail worth knowing about, because it is in the manual page itself: it still says modules are found in /lib/security/ or /lib64/security/. On a merged-/usr Debian the real path is /usr/lib/x86_64-linux-gnu/security/, and on Fedora /usr/lib64/security/. The documentation is behind the filesystem here.
Who owns these files
You are usually not supposed to edit them by hand, and the reason differs by family in a way that is worth getting right — because the two big families fail in opposite directions.
Debian and Ubuntu: pam-auth-update does not overwrite your edits. It notices them and stops. Its manual page says that local modifications cause it to treat the files as locally modified and make no further changes unless given --force — and with --force, your version is preserved as .pam-old. So the failure mode is the reverse of what people fear: nothing gets clobbered, but management quietly ceases, and the next package that ships a PAM profile silently fails to be wired in. That is a much harder bug to see than an overwrite.
Red Hat: authselect genuinely does overwrite, and says so in the file header. The supported route is authselect create-profile, optionally symlinking so you still inherit upstream updates, or authselect opt-out to take the whole thing manual. Note the renaming: the minimal profile is now called local, and has been the default for fresh installs since Fedora 40. RHEL 10 ships local, sssd and winbind. Any instruction to run authselect select minimal is giving you a command that fails.
openSUSE is a third model again: pam-config manages a set of included files and warns that manual changes to those are overwritten, but it explicitly does not touch service files such as pam.d/login or pam.d/sshd. The documented escape is to replace the managed symlinks with real files.
Stage 3 — The user is looked up
Before anything can be verified, the account has to be found. That is the name service switch again — the same /etc/nsswitch.conf that decides how host names are resolved, but a completely different set of lines. Name resolution has its own page; this stage is only about passwd, group and shadow.
The configuration systemd now recommends looks like this:
passwd: files systemd
group: files [SUCCESS=merge] systemd
shadow: files systemd
gshadow: files systemdThree things in four lines are worth pulling out.
nss-systemd is a normal part of the passwd line now. It resolves users that exist without being in a file at all: services running under DynamicUser=, users inside systemd-machined containers, foreign uid ranges. It also guarantees that uid 0 and uid 65534 always resolve, which is a quiet safety net for a broken /etc/passwd. It sits after files so that local accounts win. (It also resolves systemd-homed users, which essentially nobody deploys.)
[SUCCESS=merge] on the group line is doing real work. Without it the search stops at the first source that answers, so a user could only ever get supplementary groups from one place. With it, memberships from local files and from a directory are combined. If somebody’s group membership “is there in getent group but does not apply”, that flag — or its absence — is where to look first.
shadow is a real NSS database, and that explains a thing people find odd. getent shadow as an ordinary user returns nothing, because hashes are not readable by you — which is also why pam_unix ships a small setuid helper, unix_chkpwd, whose entire purpose is to check a password without the calling application needing to run as root.
getent passwd bob # whatever the whole switch says
getent -s files passwd bob # only the local file
getent -s systemd passwd bob # only nss-systemd
id bob # the merged group pictureThat pair of -s queries is the fastest way to find out whether a user is local, from a directory, or synthesised — and it is far more informative than reading the configuration file.
On the directory side, SSSD is still the answer for enterprise logins, but two things it used to do are gone: the files identity provider has been removed upstream and explicitly from RHEL 10, so any configuration using id_provider = files now fails to start, and enumeration has been removed for Active Directory and IdM. And do not reach for nscd to cache these lookups: Fedora removed it in Fedora 35, on the reasoning that resolved handles DNS caching and SSSD handles everything else.
Stage 4 — Credentials are verified
The password is hashed and compared. The interesting question is which algorithm, and the answer has moved to a file most people do not associate with it.
The old model was that you set sha512 as an argument on the pam_unix.so line. That is now the override, not the default. pam_unix‘s manual page states that the default hash is taken from ENCRYPT_METHOD in /etc/login.defs, and since shadow 4.14.0 PAM respects that choice rather than overriding it.
And the value there is generally no longer SHA-512. yescrypt is the default on Fedora, Arch, Debian and Ubuntu. You can settle it in one command, because the hash announces itself:
sudo getent shadow "$USER" | cut -d: -f2 | cut -c1-3
# $y$ = yescrypt
# $6$ = SHA-512
# $5$ = SHA-256
grep -E '^ENCRYPT_METHOD|^SHA_CRYPT' /etc/login.defsUpstream shadow’s own default became SHA-512 only in 4.20.0 — before that it was still nominally DES, which no distribution had used for many years. RHEL 10’s value is one I could not confirm, and it should not be assumed to have followed Fedora, because Red Hat has FIPS considerations that Fedora does not. Run the command on your own machine.
Counting failures, or not counting them at all
The module that locks accounts after repeated failures is pam_faillock. Its predecessors, pam_tally and pam_tally2, were deprecated in Linux-PAM 1.4.0 and removed in 1.5.0 — not discouraged, not legacy: the shared objects are not built. Any configuration referencing them fails. (pam_cracklib went in the same release; use pam_pwquality.)
The genuinely useful fact here is about defaults rather than modules. Red Hat systems lock accounts by default through faillock. Debian and Ubuntu lock nothing. The module is present in libpam-modules, but there is no faillock profile in /usr/share/pam-configs/ and nothing wires it into the stack. A stock Debian or Ubuntu server has no failed-login lockout whatsoever.
Three details its manual page states that get lost in retellings: it must appear in both the auth phase (as preauth, authfail and authsucc) and the account phase; options belong in /etc/security/faillock.conf rather than on the module line; and root is not locked unless even_deny_root is set. The counters live under /var/run/faillock/, which means they do not survive a reboot — useful to know before you reboot a machine to “clear” a lockout you were meant to investigate.
faillock --user bob # the counter
sudo faillock --user bob --reset # clear it deliberatelyAnd remember question one: on a key-based SSH login, none of this runs.
Stage 5 — The account stack
Authentication answered “are you who you say you are”. The account stack answers a different question: “may this account log in, right now, here”. It runs for every authentication method, including keys, which is what makes it the reliable half of the hinge.
The nologin file, and which one it actually is
pam_nologin refuses non-root logins when a nologin file exists and prints its contents to the user. People name /etc/nologin. The one that actually fires in daily life is /run/nologin, because systemd-user-sessions.service creates it during boot and again during shutdown. That is the real explanation for not being able to log in during the last half-minute before a reboot, and for the brief window at the start of one.
The module provides both an auth and an account type, and distributions differ about where they place it. Which sets up a consequence worth stating explicitly: an auth-only placement does not block a public-key SSH login. If you are relying on a nologin file to keep people out of a machine during maintenance, check which stack it is in.
Password expiry is being dismantled upstream
This one needs its caveat attached before anything else: the change is upstream, and your distribution has probably not shipped it yet. But the direction of travel is settled, and it is going to make a lot of received wisdom wrong.
shadow-utils 4.19.0 deprecated a set of password-ageing features in December 2025 and 4.20.0 removed them, citing modern security standards — NIST’s guidance among them — that now prohibit periodic password expiration rather than requiring it. Concretely:
expiry(1)is removed, along withgroupmems(8)andlogoutd(8).- The fourth field of
/etc/shadow— minimum days between changes — is now ignored, and removed if present. - Fields 3, 5, 6 and 7 survive but are deprecated for future removal.
chageremains, but several of its options for setting a minimum password age are gone.login.defsno longer ships defaults forPASS_MAX_DAYSandPASS_WARN_AGE.- DES and MD5 were removed from
ENCRYPT_METHODin the same releases.
So “/etc/shadow has seven meaningful fields and expiry is how you enforce policy” is a sentence with a shelf life. If you have a compliance regime that mandates ninety-day rotation, the tooling that implements it is being withdrawn by the people who wrote it.
While you are in login.defs, note that its own manual page says much of its historic functionality now belongs to PAM, and that it is no longer used by passwd and only lightly by login and su. Its cross-reference table is the thing to read: it tells you which command consults which setting, and the answer for ENV_PATH is login and su — not sshd. Which brings us to the next stage.
Stage 6 — The session stack
Everything so far decided whether to let you in. This stage builds what you get, and it is where nearly everything has changed since 2023.
One structural fact first, because it explains why session problems usually degrade rather than deny: session modules are conventionally marked optional. That is a convention, not a law — pam_open_session() returning failure does abort the login, and there are real cases (a module that cannot write /proc/self/loginuid inside a container, a home-directory creation that fails on a full disk, a security-context computation that fails) where you are denied outright. But the modules that commonly break are optional, which is why logind being unavailable does not lock you out of the machine.
logind registers a session — and picks its class
pam_systemd tells systemd-logind that a session exists. What almost nobody knows is that it also assigns the session a class, inferred from the PAM service name and from whether a terminal or display is known — and the class decides whether you get a user service manager at all.
There are eleven classes now, plus a pseudo-class: user, user-early, user-light, user-early-light, user-incomplete, greeter, lock-screen, background, background-light, manager, manager-early, and none. The ones to understand are the suffixes:
| Class | What it means for you |
|---|---|
user | Full session. user@.service starts, so systemd --user runs and XDG_RUNTIME_DIR exists |
anything -light | Registered, but deliberately no service manager. systemctl --user fails by design |
greeter, lock-screen | The display manager’s own sessions, before and around yours |
background | No terminal — a cron job, an FTP transfer |
none | Not registered at all. No session scope, no user manager |
background-light, and sessions for non-root system users became user-light.That reclassification is the thing most likely to have broken something for you between one distribution release and the next. Cron jobs and service-account logins that used to pull in a full user service manager now do not, on purpose, because starting one for every background task was waste.
Session tracking also changed mechanism in 258. logind used to hold a FIFO open in the session leader and notice when it closed; it now uses pidfds, and the exit of the session leader causes the session to stop immediately. Every explanation you will read involving a FIFO is describing removed code — and the practical difference is that logout timing is now sharp where it used to have slack.
Two smaller things while we are here. Your session lands in a cgroup, and since 258 removed the legacy and hybrid hierarchies entirely, that is unconditionally cgroup v2 — you can drop the hedging that older writing includes. And a seat is a set of input and output devices attached to one physical workspace; almost every machine has exactly one, called seat0, which is what XDG_SEAT in your environment refers to.
One sentence explains more login confusion than anything else on this page
From pam_systemd(8): $XDG_RUNTIME_DIR is not set if the current user is not the original user of the session.
The runtime directory belongs to the session, and a session belongs to the user it was created for. su and sudo change your uid inside somebody else’s session. The session does not change; you are simply no longer the user it is for. So the variable is deliberately withheld.
From there the whole symptom cluster follows as a chain:
no XDG_RUNTIME_DIR → no path to a session bus socket → no session bus → systemctl --user reports it cannot connect → the keyring, the user units and everything else that lives there is missing too.
That is one fault with five faces, and it is not a bug. If you need a real session for another user, ask for one — machinectl shell bob@ or an actual login — rather than trying to repair the environment by hand.
Where PATH actually comes from
This gets asked constantly and answered incompletely. Five layers can set it, later ones winning:
- The daemon’s own compiled-in default.
sshdhas one. ENV_PATHandENV_SUPATHinlogin.defs— consulted byloginandsu, and not by sshd. This alone explains a great many “it works on the console and not over SSH” reports.pam_env, reading/etc/security/pam_env.confand/etc/environment.- The shell’s own files —
/etc/profile,/etc/profile.d/*,~/.profile,~/.bash_profile. The only layer most people think about. - For
sudo,secure_pathin sudoers, which overrides everything above it.
And the trap: ~/.pam_environment no longer works. pam_env‘s user_readenv option defaults to off, and the feature has been deprecated since Linux-PAM 1.5.0 pending removal. Advice to put variables there — which was genuinely the right answer once, because it applied to graphical sessions too — now silently does nothing at all.
limits.conf versus systemd, settled
The folklore is that these two fight and systemd wins. They do not fight. They govern disjoint sets of processes, and the boundary is simply who forked what.
pam_limits calls setrlimit on the process running the PAM stack. Your login shell is forked from that process and inherits, and so does everything you launch from it. That is ordinary rlimit inheritance and systemd has no opinion about it.
Meanwhile your shell lives in a session-N.scope, and scopes cannot carry rlimits at all. A scope manages processes systemd did not create; it can carry kill settings and cgroup resource controls, but not LimitNOFILE=, because systemd never execs those processes. So for your interactive shell, pam_limits wins uncontested — because systemd is not playing.
systemd --user and everything beneath user@UID.service is a different branch of the process tree. PID 1 forks it; it is not descended from the PAM process. Its limits come from DefaultLimit*= and from the unit, and limits.conf reaches it only if your distribution put pam_limits into /etc/pam.d/systemd-user. That varies, and it is checkable:
ulimit -n # your shell
systemd-run --user --pty --wait bash -c 'ulimit -n' # a user unit
grep limits /etc/pam.d/systemd-user 2>/dev/null # is it wired in?Two different numbers from the first two commands means you have found the boundary rather than a bug. And system services never see limits.conf at all — that is deliberate and long-standing, not an oversight.
The last-login line moved
That “Last login: …” banner is a session module, and pam_lastlog was deprecated in Linux-PAM 1.5.3 and is no longer built by default. Fedora 43 migrated system-wide to pam_lastlog2, with a SQLite backend that is safe past 2038 and an automatic migration of /var/log/lastlog; Debian and openSUSE got there earlier, and Ubuntu simply removed the module from its login configuration. So the sentence “your last-login line comes from pam_lastlog reading /var/log/lastlog” is now wrong on four of the five distributions most readers use.
Stage 7 — The session ends
You log out. What happens to the processes you left running is decided by a setting whose default is not where you would look for it.
KillUserProcesses= decides whether leftovers are killed. Upstream it defaults to yes. On Debian and Ubuntu it defaults to no — and that difference is compiled in at build time rather than written in a configuration file, which is exactly why grepping /etc/systemd/logind.conf finds nothing and people conclude the upstream default applies. On several distributions that file is not shipped at all any more; local changes belong in a .conf.d/ drop-in.
Rather than trusting any table — including one on this page — ask the machine:
systemd-analyze cat-config systemd/logind.conf | grep -iE 'killuser|removeipc'
loginctl show-user "$USER" | grep -i lingerThe escape hatch is loginctl enable-linger, which starts a user manager at boot and keeps it after logout. That is the supported way to run something as yourself without staying logged in — and on a machine where KillUserProcesses is yes, it is the only way short of a system service.
Three related things that stale writing gets wrong. UserTasksMax= no longer exists in logind.conf; per-user task limits are a TasksMax= drop-in on user-.slice now. RemoveIPC= defaults to yes both upstream and on Debian, so the claim that SysV shared memory is only cleaned up on Red Hat systems is wrong. And the pidfd change from stage 6 shows up here as timing: logout is now immediate when the session leader exits, where the old mechanism left a little slack.
How to watch all this happen
The stacks will narrate themselves if you ask. This is what the farmed articles structurally cannot give you, because it requires a running machine.
# Make the modules talk. Add debug to the module line, temporarily.
# session optional pam_systemd.so debug=1
# auth required pam_unix.so debug
# Watch a login happen, on all three OpenSSH binaries
journalctl -f _COMM=sshd _COMM=sshd-session _COMM=sshd-auth
# What kind of session am I actually in?
loginctl session-status "$XDG_SESSION_ID"
loginctl show-session "$XDG_SESSION_ID" -p Class -p Type -p Remote -p Scope
# Where did I land in the cgroup tree?
systemd-cgls /user.slice
# Force the auth stack to run, to prove whether it normally does
ssh -o PreferredAuthentications=password you@host
# Run a PAM stack without a real login
pamtester sshd "$USER" authenticateThat second-to-last command is worth dwelling on. If your auth stack behaves differently when you force password authentication than it does on an ordinary login, you have just proved your ordinary logins are not running it. That is question one from the opening box, answered in one command.
Advice that has expired
This subject has an unusually stale corpus, and the staleness is not random. It clusters around how PAM was documented roughly between 2010 and 2015, because that is where the written material is densest. Every row below was correct once.
| What you will still read | What is true now |
|---|---|
“Run journalctl _COMM=sshd to see why a login failed” | Wrong since OpenSSH 9.8. Per-connection work is in sshd-session, authentication in sshd-auth since 10.0 |
| “PAM’s control flags are required, requisite, sufficient, optional” | Those four are shorthand for bracketed forms, and include/substack are control values too. Real stacks are written in brackets |
“Your PAM configuration is in /etc/pam.d/“ | Since Linux-PAM 1.7.2 the vendor directory is on by default: /usr/lib/pam.d/ is used when /etc has no such file |
“pam_tally2 is the older lockout module” | Deprecated in 1.4.0, removed in 1.5.0. It does not exist anywhere. Nor does pam_cracklib |
| “Red Hat moved to faillock, so Debian uses something else” | Debian and Ubuntu wire up no lockout at all by default. The module ships; nothing enables it |
“Set sha512 on the pam_unix.so line” | That is the override. The default comes from ENCRYPT_METHOD in login.defs, and is yescrypt on Fedora, Arch, Debian and Ubuntu |
“pam_systemd sets XDG_RUNTIME_DIR for your session” | Only when you are the original user of the session. That is why sudo -i has no runtime dir, no bus and no systemctl --user |
| “logind tracks your session with a FIFO in the session leader” | Replaced by pidfds in systemd 258. Logout is now immediate on leader exit |
“KillUserProcesses defaults to yes” | Upstream yes, Debian and Ubuntu no — set at build time, which is why the config file does not mention it |
“With UsePAM yes, PAM handles SSH authentication” | Only password and keyboard-interactive. A public-key login never runs the auth stack. Upstream’s UsePAM default is no |
“limits.conf and systemd fight, and systemd wins” | They govern disjoint process sets. Scopes carry no rlimits, so for your shell pam_limits wins uncontested |
“Put environment variables in ~/.pam_environment“ | Silently does nothing. user_readenv defaults to off and the feature is deprecated for removal |
“The last-login line comes from pam_lastlog and /var/log/lastlog“ | pam_lastlog is not built by default. Fedora, Debian, openSUSE and Ubuntu have all moved on |
“Hand-edits to /etc/pam.d/common-* get reverted” | Backwards on Debian: edits make pam-auth-update stop managing the file. On Red Hat, authselect does overwrite, and says so in the header |
“authselect select minimal“ | Renamed local, default for fresh installs since Fedora 40. The old command fails |
“su - runs the same PAM stack as su“ | su uses /etc/pam.d/su; su - uses su-l. sudo -i uses neither |
“/etc/nologin blocks non-root logins” | /run/nologin is the one that fires, written by systemd-user-sessions.service. And an auth-only placement will not stop a key-based login |
“ENV_PATH in login.defs sets your PATH“ | For login and su only. sshd does not consult login.defs |
“sudo logs every command; replay with sudoreplay“ | Not on Ubuntu 26.04 LTS, which defaults to sudo-rs: no I/O logging, no sudoreplay, no sudoers.ldap |
“SSSD can serve local users with id_provider = files“ | Removed upstream and from RHEL 10. The configuration now fails to start |
“Use nscd to cache passwd and group lookups” | Removed from Fedora since Fedora 35; SSSD no longer even warns about its configuration |
“Set per-user task limits with UserTasksMax=“ | Gone from logind.conf. Use a TasksMax= drop-in on user-.slice |
| “On systems using the unified cgroup hierarchy…” | Drop the hedge. systemd 258 removed the legacy and hybrid hierarchies entirely |
“/etc/shadow has seven meaningful fields; use expiry“ | Being dismantled upstream: expiry(1) removed, the fourth field ignored, four more fields deprecated. Not in most stable distributions yet |
Which suggests a heuristic worth handing on. The stale corpus is wrong in a consistent way, and there are eight tells: four control flags with no brackets, pam_tally2, /etc/nologin named alone, sha512 as the default, dbus-launch, nscd, /var/log/lastlog, and ~/.pam_environment. If a page contains three of those eight, it was not written from a running system — whoever produced it was working from documentation about a decade old. That test takes about three seconds and it is reliable.
A worked diagnosis
A deployment pipeline restarts an application by running a command over SSH as the service account that owns it. It has worked for two years. The hosts were rebuilt onto a newer release last week, and now every deployment fails:
$ ssh svc-app@host 'systemctl --user restart myapp'
Failed to connect to bus: No medium foundThe reported conclusion is that logind is broken on the new image. Work the stages instead. The command ran, which means there was a shell — so by the hinge, the account stack returned success and the fault is stage 6 or later. And the symptom is on the list that collapses into one question.
$ ssh svc-app@host 'echo "id=$XDG_SESSION_ID rt=$XDG_RUNTIME_DIR"; \
loginctl session-status "$XDG_SESSION_ID" | head -6'
id=47 rt=
svc-app (985)
Since: Tue 2026-09-01 09:14:02 UTC
Leader: 3312 (sshd-session)
Class: user-light
Type: tty
State: activeThere it is, in the Class: line. The session exists and is active — logind is fine. But the class is user-light, and a -light class does not pull in user@.service. There is no user service manager, no runtime directory, and therefore no bus to connect to. Nothing is broken; the behaviour changed on purpose.
Why this account and not a person’s? Look at the uid: 985. It is a system user, created by the package rather than by hand, and systemd 258 reclassified sessions for non-root system users from user to user-light precisely so that cron jobs and service-account logins stop starting a full service manager each time. The old hosts predated that release. Nobody changed anything; the machine did.
$ sudo loginctl enable-linger svc-app
$ ssh svc-app@host 'systemctl --user restart myapp'
$ # and it returns nothing, as it shouldLingering starts the user manager at boot and keeps it running independently of logins, which is what this workload always actually needed — it was getting one by accident before. The alternative, and arguably the better one, is to stop running it as a user unit at all and make it a system service with User=svc-app.
Note what would have happened without the class line. Every plausible next step — restarting logind, checking dbus, comparing package versions, reinstalling systemd — investigates a healthy component. The environment of a shell does not tell you what kind of session it is in. Only logind knows, and you have to ask it.
What to hold on to
Getting in and getting a session are different achievements. The account stack decides the first and always runs; the session stack decides the second and mostly degrades rather than denies. If you have a shell, stop investigating authentication.
The auth stack is the one that might not have run at all. Key-based SSH skips it; so does passwordless sudo. Before spending a day on a module you added there, force a password login and see whether the behaviour changes.
Ask the machine rather than a table. The differences that matter here — whether processes are killed at logout, whether lockout is wired up, which hash is in use, whether pam_limits reaches your user units — are set at build time or in generated files, so no table can be trusted across distributions. systemd-analyze cat-config, loginctl session-status, getent -s and the first three characters of a shadow hash answer all of them in seconds, on the actual machine, correctly.
Related reading
- Users, Groups and Root — the introduction this page sits underneath
- Permissions and Privilege, Properly — what those credentials mean once you have them
- What Happens When You Run a Program — fork, exec and the dynamic linker: the step this page stops short of
- DNS, All the Way Down — the other half of
nsswitch.conf - SSH Keys — the authentication method that skips stage 2 entirely