210 views
"Secure Lock" Research === <style> blockquote { background: #c00 !important; color: white !important; padding-top: 1em !important; padding-bottom: 1em !important; &:before { content: "TODO: "; } } </style> This document details the various requirements for evicting homed keys by locking with GDM, the issues these requirements bring up, and possible solutions to these issues. I'll be marking my favorite ideas w/ :star:. These are my research notes for working on key eviction, probably to be turned into a bunch of issues under https://gitlab.gnome.org/Teams/STF/homed eventually. High-level overview of basic requirements: - We should define some terminology - GDM needs to report to systemd-homed that it supports homed's lock-on-suspend functionality, which will turn it on - Whenever entering a secure lock, we need to get GDM on screen in some way. Otherwise, there won't be any UI that the user can interact with to unlock the system once it comes back up - GNOME must be able to enter secure lock whenever switching users, not just automatically on system suspend. - Various components running inside of the session should be warned when we're about to delete the encryption key, so that they can close sensitive FDs, zero out sensitive memory, etc to help increase the security Future work we must keep in mind when building out a solution: - DE will want to be more intelligent about when it secure locks and when it doesn't: [Smarter Key Eviction](https://gitlab.gnome.org/Teams/STF/homed/-/milestones/3). This extends the requirement above re: locking on user-switch: the DE should be allowed to decide to lock in any arbitrary set of conditions - GNOME on phones will need to run some apps on the lockscreen. More detailed discussion below. Nice-to-haves to consider: - GDM showing special unlock UI, with shield and specialized into one user. We want this for MVP! - GDM having user settings available to it, so that secure sheild is consistent w/ the session's customizations - A nice animation that happens on the transition b/t session, shield, and secure shield would be nice. Not critical for MVP - All this must be at least somewhat robust against parts of GNOME failing/crashing, or even missing altogether ## Defining terms Unfourtunately, there's _lots_ of closely-related functionality across the systemd and GNOME projects that have similar names and are thus all tangled and confusing. So first let's try to describe all the terms, their contexts, and what they mean: - `screen shield`: gnome-shell's name for the lock screen that currently exists in GNOME. Basically just a UI running inside of the session that is rendered over top of all content & prevents the user from interacting with or seeing anything in the session - `unlock` (common usage): Re-running the PAM stack for an already-logged-in user to prove that the user is who they say they are and give them access back to their session. - `reauthentication`: gnome-shell and GDM's name for this - GNOME will try reauthentication whenever the user types their password into the "unlock dialog" part of the shield - `lock` (in the context of `systemd-logind` and throughout GNOME): activate/show the shield - `unlock`: hide the shield - IMPORTANT! logind unlock != common usage of unlock (aka reauthentication) - If logind tells GNOME to unlock the session, GNOME will do so completely bypassing reauthentication - Usually, logind will only ask GNOME to unlock the session when GDM tells it to unlock the session, and GDM usually tells logind to unlock the session only in response to a successful reauthentication - `suspend` (in the context of `dm-crypt`): Wipe the volume's encryption key out of memory, thus making it inaccessible - Any task that tries to access a suspended volume will be frozen by the kernel until the volume is resumed - `resume`: Given user's crednetials, re-instate the encryption key to make the user's data accessible again - LUKS `suspend` / `resume` is an alternate way to talk about this, used on the command-line, even though it's not really a feature of LUKS but of `dm-crypt` - `lock` (in the context of `systemd-homed`): Suspend (in the `dm-crypt` sense) the user's LUKS volume and freeze any session process - `unlock`: Resume (in the `dm-crypt` sense) the user's LUKS volume - Note that this requires access to the user's credentials, and thus _requires_ reauthentication. In fact, homed unlocking is performed as part of reauthentication - Note that these don't imply locking / unlocking in the logind sense! - `suspend` (common usage): power-saving feature where kernel freezes all of userspace and puts hardware into some low-power state - `sleep`: synonym, used in everday speech, similar features in other OSs, and sometimes in systemd - `resume` / `wake`: opposite; bring system back out of a suspended state - Different forms of this exist: `s2idle`, `s3`, etc - `suspend` (in the context of `systemd-homed`): Functionality of systemd-homed to automatically lock (in the homed sense) all users when the system is suspended (in the common usage) Adding support for all the homed functionality will also create new situations that don't have well-defined terms for them yet: - The screen shield running in a separated session as part of GDM - The screen shield running in a separate session but also when the user is homed-locked. As you can see, there's lot of overlap in terms and so it's hard to be very precise about what you're talking about. So I'll define here exactly what I mean for each ambiguous term, and in all other situations I'll try to make sure and specify what I'm talking about: - `lock`: I'm using this term in the logind sense, much like the rest of GNOME - `secure lock`: I'm using this to mean the combination of homed locking with logind locking - `unlock`: I'm using this to mean the combination of the common use of the word "unlock" (i.e. reauthencication) with the logind use of unlock (i.e. hiding the shield) - `suspend` / `resume`: I'm using the common usage of these terms, i.e. relating to the power-saving feature - I'll be calling homed's `suspend` functionality lock-on-suspend or similar - I'll use `session shield` to refer to the existing shield running in the session, `GDM shield` to refer to a shield running in a separate GDM session but without a secure lock active, and `secure shield` to refer to a shield running in a separate GDM session while a secure lock is active. - For the time being, just plain `shield` means session shield - If we decide to always lock via GDM shield (which I'll discuss later in this document), I'd like to redefine `shield` to mean GDM shield. But for the time being `shield` will mean session shield. - `lockdown`: Possible user-facing terminology for triggering a secure lock? - `lock screen`: I'll be using this as a synonym for "shield"; it's a more user-facing name. I'll be talking about `notifications` a lot in this document too. Specifically in the context of apps-on-lockscreen---which we'll be discussing later---they're quite important. I'll be assuming that notifications are powered by [Notification Portal API v2](https://github.com/flatpak/xdg-desktop-portal/issues/983), but I'll try to make clear if I'm talking about notifications as a concept or `notifications v2` in the sense of the new portal API ## Telling homed we support lock-on-suspend Basically, we just need to set `SYSTEMD_HOME_SUSPEND=1` into the session's environment, before we run the PAM stack in that env. `pam_systemd_home` checks the environment variable during authentication, and when opening a session. We should have the variable set in both places. GDM's session startup looks like this: - gnome-shell begins authenticating once user is selected (or we're on the lockscreen) - authentication starts multiple conversations - each conversation forks off a worker process - Once the worker is forked off, we call setup on the worker - Once setup is done, start authentication - Once authentication is done, start authorize - ... We must insert the environment variable between setup and authentication. We should probably do this in `gdm_session_authenticate`: - check if we have `X-GDM-CanHomedSuspend=true` in the session launcher .desktop file ([already implemented](https://gitlab.gnome.org/Teams/STF/homed/-/issues/3#note_1981255)) - call `send_environment_variable("SYSTEMD_HOME_SUSPEND", "true", conversation)` before we call `gdm_dbus_worker_call_authenticate` ## Present GDM's secure shield when secure locking ### Idea 1: Always use GDM to lock Implementation: - Make the session shield just a basic blank screen that consumes all events, maybe with a message in the middle saying "This session is locked" - We still need a session shield because user will be able to `chvt` or <kbd>Ctrl</kbd>+<kbd>Alt</kbd>+<kbd>F#</kbd> back into the session. But we don't need it to actually do anything other than block out the session - We shouldn't make this new minimal session shield interactive at all, because it might be completely frozen if we're homed-locked. - :star: Perhaps GDM should detect that we're looking at the frozen TTY and `chvt` us back to the corresponding secure shield session? <kbd>Ctrl</kbd>+<kbd>Alt</kbd>+<kbd>F#</kbd> to get back to the GDM shield probably won't work from the session because the session's Mutter will be frozen. We want to avoid letting the user get stuck looking at a frozen session - Where we currently handle logind locking, just ask GDM to show the GDM shield - Whenever asked to show the GDM shield, GDM should create a special unlock session and then `chvt` to it. - GDM shield communicates with gnome-shell running inside of the session to proxy across important dynamic/ongoing content - notifications - MPRIS controls - Negotiating apps-on-lockscreen - GDM shield exposes a dbus object that only session's gnome-shell is allowed to talk to, and session's gnome-shell calls methods on this object to send over notifications, MPRIS, etc. Object also exposes a bunch of signals that session's gnome-shell listens for to pass along user interactions - We'd have to proxy sound across out of the session into the shield - Logind sets up ACLs on the audio device in a special way so sound can keep playing - Pipewire module to pass through events from GDM's pipewire instance to session's - If a secure lock is activated, the GDM shield transitions into the secure shield. - Plays a nice little "lockdown mode" animation. Because we're not transitioning b/t VTs here, we can make this really smooth - Cuts off communication with the session, by dropping the communication dbus object - If gnome-shell is frozen, which will be the case pretty much always, it won't be able to communicate via the dbus object anyway - If gnome-shell isn't frozen, which [is possible if someone opts out](https://github.com/systemd/systemd/pull/30612#issuecomment-1868584999), we don't want to try and communicate with it - Hides notifications, MPRIS controls, disables/restricts apps-on-lockscreen, disconnects the audio passthrough, etc - We'd need to have systemd-homed send us a signal when the user gets locked, much like idea 3 below Pros: - Great for security! A lot harder to bypass shield if it's running in a completely separate environment - Case study: iOS's lock screen has historically suceptible to various escapes that let you enter the running session and bypass it - The reason we haven't run into issues like this is because our shield is relatively basic: once we start introducing more advanced notifications on the shield, apps on the lockscreen, etc it becomes really hard to make sure all the holes are patched up. Running the shield outside of the session means that you can't quite so easily bypass it - you need to trick GDM into actulaly tansitioning back into the session, which is a lot harder - We'll have just one implementation of the shield that the user will see - No strange stutter / transition animation as we `chvt` the system in response to a secure lock - No inconsistency. The shield will behave the same way no matter if we're normally locking or secure locking Cons: - Complexity. Need to build the communication layer b/t GDM and gnome-shell, to pass along notifications, MPRIS, etc. Need to build communication layer b/t sound system so media can continue playing in the background - HUGE amount of work. Probably won't fit into MVP. Basically need to get it perfect to ship it to everyone This shouldn't be a loss of functionality, since we don't support locking on non-GDM setups anyways. > Investigate if this is true in remote-desktop situations. Can GDM take over the screen on remote-desktop? Do we even support session locking on remote-desktop? Robustness is neutral here. On one hand, the session needs to activate the GDM shield itself. On the other hand, the session will activate the GDM shield on any logind lock, which generally doesn't fail > If implementing this, we need to figure out in detail where exactly (in what functions) do we activate the transition into GDM? ### Idea 2: Always use GDM lock, but for homed-backed users only Basically this is exactly like Idea 1, but we don't strip out the functionality of the existing session shield. We continue using the existing session shield for users that aren't backed by homed, or situations where the GDM shield might not work like remote logins This will give us some freedom to implement the GDM shield iteratively, across many GNOME versions, without breaking functionality for most normal users. For example, the whole communication between the GDM shield and session's gnome-shell doesn't have to land immediately for the MVP. Ditto on sound passthrough via pipewire. Ditto on complete settings passthrough. For the MVP, these things being broken for homed users is probably OK. Maybe once the GDM shield is perfectly invisible, we can then cut out most of the functionality of the session shield Same pros/cons as Idea 1, except we can drop the con about it being a huge amount of work for the MVP since we'll be able to iterate on it. ### :star: Idea 3: homed emits a signal whenever user is locked GDM listens for this signal and spins up the secure shield to handle it Pros: - Robust: no matter how a secure lock is triggered (even if gnome-shell wasn't expecting a secure lock), we end up with a secure shield on screen - Note that in ideas 1/2 we end up on GDM even if we're not necessarily expecting a secure lock, so this idea isn't necessarily _much_ more robust than those ideas. - Simpler to implement - no need to implement a feature-complete GDM shield; we'd just have the secure shield and basic shield - Thus, no communicating w/ session's gnome-shell, no audio passthrough, no apps-on-lockscreen across sessions, etc etc etc Cons: - Using less secure session shield whenever we're locked but not secure locked - Have to worry about the ability to escape the session shield, or the ability for apps to appear on top of the session shield (possibly exposing user data, settings, etc in the process), which is its own kind of complexity - Foreshadowing: As we'll decide later in the document, our implementation for apps-on-lockscreen will require the session shield anyway so we're not giving up much security - Worse UX: Jank of transitioning between sessions, possibly in the middle of the user interacting with the UI of the session shield - User might be in the middle of typing their password as we transition the session - User might have just opened the a11y menu when we transition to the secure shield that has the menu closed - Actually not a huge deal, because transitions to the secure shield will only happen due to explicit user action OR because the system is actively putting itself into suspend. Both cases make this a non-issue UX wise. It's technically possible to lock a user from something like an SSH session but honestly we shouldn't worry about it ### Idea 4: Session predicts homed lock and asks GDM to take over Pros: - Maybe simplest to implement Cons: - Same cons as idea 3 - Lack of robustness: systemd-homed might decide to homed-lock the session, without the session expecting it, so we'll end up looking at a frozen session with no way to unlock it. ## Letting DE lock itself up There's a couple of situations I can think of where the DE should be able to decide for itself whether or not it gets secure-locked: - Whenever switching users away from some homed-backed user. We should secure-lock that session - Note that this includes switching user from the GDM shield too - Implementing [Smarter Key Eviction UX](https://gitlab.gnome.org/Teams/STF/homed/-/issues/4), i.e. letting the DE prevent secure lock in situations where it leads to bad UX - Implementing a feature where user can promote a normal lock to a secure lock. Like "Lockdown" mode on other OSs Functionality provided to us by systemd to implement these things: - systemd-homed has an auto-lock-on-suspend feature that initiates a homed lock when the system is suspended - systemd-homed has an `InhibitSuspend` call we can make, which returns a file descriptor. While the file descriptor is open, auto-lock-on-suspend is inhibited and will not be performed. When all copies of the FD are closed, then auto-lock-on-suspend is re-enabled - logind provides inhibitors, that can be used to delay suspend - logind emits a `PrepareForSleep` signal immediately before suspend and immediately after resume. This can be used by the DE to ensure shield is on screen before computer goes to sleep: - gnome-shell holds a delay inhibitor for suspend - logind emits `PrepareForSleep(true)` - gnome-shell puts shield on screen - gnome-shell closes delay inhibitor FD - logind detects that all delay inhibitors are gone. Allows sleep to continue. Eventually systemd-sleep suspends the system - On resume, processes are unfrozen - logind emits `PrepareForSleep(false)` - gnome-shell detects this signal an re-acquires the delay inhibitor First thing we should implement, as a form of safety: We want to make sure that the session doesn't get locked out from under us by homed if we're actively looking at the session and would thus have no way of unlocking the session. So, gnome-shell should hold the auto-lock-on-suspend inhibitor as long as its on screen, but if gnome-shell is off screen we can drop the inhibitor. To implement this we can watch `org.freedesktop.login1.Seat.ActiveSession` for changes (or if that doesn't work out we can look into `meta_backend_native_resume/suspend()`; suggested in #gnome-shell Matrix) ### Use-case 1: Switching user The existing switch users call ([`activateSwitchUser`](https://gitlab.gnome.org/GNOME/gnome-shell/-/blob/main/js/misc/systemActions.js#L426)) looks something like this: - show the session shield without animating - switch to GDM login screen We need to add a third step after this, to homed-lock the session. We can probably just have gnome-shell synchronously call org.freedesktop.home1.LockHome via dbus: if successful it won't return until after reauthentication, but that's probably OK. (Note that there's a [known issue](https://gitlab.gnome.org/Teams/STF/homed/-/issues/40) with freezing the session, but isn't affected by whether or not the shell freezes itself) Note that we don't need the GDM/secure shield in this case. Unlock will be performed by GDM's login screen. We also must support switching user from the shield ([`_otherUserClicked`](https://gitlab.gnome.org/GNOME/gnome-shell/-/blob/main/js/ui/unlockDialog.js#L838)) too. Right now it just switches to the GDM login screen. We'll need to add the additional homed lock step too. If we're on a GDM or secure shield, we can also stop the shield from running to conserve resources. ### Use-case 2: Smarter UX transitioning from normal lock → secure lock Seems like either session shield or GDM shield will be what ultimately decides if we're locking-via-homed or not. Basically when activated the session/GDM shield will decide if we should be locking down on suspend or not. Depending on what it picked, it'll immediately either take the auto-lock-on-suspend inhibitor or release it. Time-delay can be implemented by taking the inhibitor, then after 24 hours of inactivity waking the system back up, dropping the inhibitor, and going back to sleep. We can choose to use time-delay mode (which we want on mobile phones but not PCs) based on whether the instance of gnome-shell is in "mobile mode" or not. When actually implementing this, we'll need to figure out when exactly to make the choice and how to delay system suspend to ensure that homed's auto-lock-on-suspend doesn't activate before we make any decisions. Of course, if we're making this decision from the session shield, we need to ensure GDM ends up on screen. This is either done by us manually activating it, or letting homed send a signal that GDM catches. ### Use-case 3: Lockdown mode This is the easiest one. We first need to lock normally, if we're not there already. Possibly transition to GDM manually if we need to do that. Then we activate homed lock ## Warning apps about secure lock Why? So that apps can close sensitive FDs, zero out sensitive memory, etc before the system locks up to increase the security of the lock. :star: The implementation is quite simple. We'll basically just copy logind's `PrepareForSleep` mechanism and delay inhibitors in homed: - `DelayLock() -> (h fd, o path)`: would be callable only by processes running in the session. For convenince, we return the object path of the object you need to listen for `PrepareForLock` on. After the first call the object path will always be the same - `PrepareForLock(b locking)` signal on the user-specific object. If we send out `PrepareForLock(true)`, then do whatever you need to do then drop `DelayLock()` fd. After unlock, we send `PrepareForLock(false)` to tell clients to reacquire the `DelayLock()` fd Eventually, we'll want to be able to encrypt individual flatpak apps (see [my proposal](https://github.com/systemd/systemd/issues/31741)) during normal, insecure, lock too. This encryption stuff will be implemented in Flatpak, and Flatpak should export a similar API into the container. Flatpak will delegate the homed API into the app (and use it itself, to ensure its own per-app keys get dropped whenever homed is locked), but also send its own signals if an individual app is getting locked but homed isn't. Once all this Flatpak stuff is in, it'll probably start becoming useful to expose the API in GLib (via signals in `Gio.Application`) ## Apps on lockscreen If we're going to be using GDM or secure shields, we need to think about the future where GNOME runs on mobile. Mobile OSs generally carve out special functionality for some apps to appear over top of the lock screen. As I redesign the lock screen system to work for homed, I don't want to accidentally get in the way of a great GNOME Mobile experience. ### Use-cases & Prior Art First, let's run through the situations where this is actually necessary. I personally don't think it makes sense to let arbitrary apps put themselves on the lockscreen: we should have a reduced set of situations where it's possible. We should follow the GNOME Way™ and provide targeted APIs for apps to use instead of giving them a broad primitive that we may regret later. So here are the use-cases I can think of, and a description of how they work on other OSs. - Incoming phone call / VOIP call / video call - It's critically important that an incoming call can be answered from the lock screen. Being able to control the call from a locked device is also important, but slightly less so. And all of this must work with third-party calling apps - Prior art: Android - Apps can attach a "full screen intent" to any notifications that they send - OS decides when/where/how it's appropriate to trigger this full-screen intent - Notification is either presented as a pop-up normally, or the OS activates the full-screen intent instead and the app presents its UI. This full-screen intent can be triggered over top of the lock-screen - The UI presented by this intent can activate other intents to present more of the app on top of the lock screen - Example: The device starts recieving a phone call. The dialer app detects this, and sets up a full screen intent to show the full-screen "incoming call" UI. It also sets up a notification for an incoming call also, and links the two together. This is sent as a notification. Android decides whether to show the notification (if the user is busy interacting with the device, for instance) or as a full-screen intent (if the device is idle). The full-screen intent can be rendered over top of the lock screen. When the call is answered via the incoming call UI, the app can launch its ongoing call UI directly over top of the lockscreen - Another example: device recieves a WhatsApp video call. WhatsApp is woken by the push notification, creates a full-screen incoming call intent and a notification, and sends it. If the device is locked Android will show the full-screen UI, and answering the phone will launch the in-call UI directly on the lockscreen. Trying to leave WhatsApp's ongoing-call UI will immediately take you back to the lockscreen - Apparently some apps have been abusing this API - Android 14 makes this a "Special App Permission" that needs to be manually granted to the app by the user. Not even a permission dialog: just takes the user to the settings app to manually flip the switch - App developers can ask for an exception in the Google Play Console, and if they meet the requirements the permission will be automatically granted to the app @ install time - > The exact requirements outlined by the Google Play program are undefined right now, but will be published in April 2024. Right now it _looks like_ apps will be implicitly granted this permission only if they're alarm clocks or voice/video calls. Other apps will need the user to flip the switch in settings ([Support page about this](https://support.google.com/googleplay/android-developer/answer/13392821?hl=en#zippy=%2Cwhen-will-these-changes-for-full-screen-intent-in-android-take-effect%2Cwhat-is-changing-for-full-screen-intent-in-android%2Cif-my-app-is-an-alarm-clock-app-or-an-app-that-receives-phone-or-video-calls-how-can-i-make-sure-my-app-is-automatically-granted-the-permission)) - Google Play policies are updated (effective May 31, 2024) to get your app banned if you abuse the permission to send ads or not-important notifications - Prior art: iOS CallKit - Apps handling incoming calls ask the OS to display an incoming call UI (same UI as the dialer) - Could be full-screen incoming call UI if device is idle - Could be a pop-up incoming call UI on devices w/ a "Dynamic Island" if the device is in-use - If the call is accepted while the device is unlocked, the OS tells the app and the app can put itself on screen to show its own in-call UI - The app could alternatively just wire up the audio backend and let the OS handle the in-call UI too - If the call is accepted while the device is locked, the OS displays its own in-call UI with no video support - A button is added to the OS's in-call UI to launch the app's native in-call UI. Pressing it prompts to unlock the session and then opens the app - If the app declares that this is a video call, iOS will immediately prompt for unlock and open the video call app w/o the user needing to press the extra button in the OS's in-call UI. User can press cancel to return to OS's in-call UI - This API lets the OS handle call-waiting correctly. By telling the OS about the fact that it's in a VOIP call, and telling the OS about incoming calls, it lets the OS handle waiting exactly the same as a normal call: offering to "Hold and Accept", "End and Accept", or "Decline" - Apps can declare that they don't support holding, in which case the only options are "end and accept" and "decline" - This API also lets the OS treat in-app VOIP calls like phone calls, putting them in the recent calls list of the dialer, integrating w/ system-wide blocked contacts system, allowing the call to be initiated by Siri or from the contacts app, etc - FaceTime calls get some special exceptions: - Incoming video calls will show a preview of your camera. I suspect other video-call apps don't do this because iOS doesn't know exactly what stream of video the app will be sending so it cannot show you a preview - Answering an incoming FaceTime call shows the FaceTime in-call video UI immediately on the lock screen - An ongoing normal phone call can be promoted into a FaceTime call (with the normal in-call UI) even if the device is locked - Overall, I'm actually a fan of the iOS API. It's more targeted, and thus allows us to build a better UX. Some changes I'd make, though: - Allow video-calling apps to give the OS a pipewire video stream of what the other end will see if you answer the call. This stream is made the background of the incoming call screen. This will allow the feature that FaceTime has to work w/ other apps (we can pass the FD through dbus into the notifications v2 API) - I'm not super against making the user unlock their device to interact with a video call once they've answered it with the system UI. Alternatively we could allow video-calling apps to hand over a second pipewire video stream (of what the other side is sending us) to be displayed on the OS-managed ongoing call screen - Alternatively, some hybrid solution: iOS-esque incoming call system, but once the call is picked up you're allowed to draw a window on the lockscreen to handle the ongoing call - Incoming alarm / calendar event / reminder - Prior art: Android uses the same notification w/ full-screen-intent mechanism as incoming calls - OnePlus 8 treats calendar events and reminders like normal notifications - Samsung OneUI treats all three the same way, like an alarm clock, with a snooze button and everything. Only difference is that alarm clock has an alarm that rings for extended period of time, others just ding the phone once - Samsung OneUI lets you pick different wallpaper (among a few different gradients) for different alarms/events/reminders so you can tell at a glance what's happening. It can also use TTS to read aloud the time & name of the alarm/event/reminder. - Prior art: iOS - Calendar events / reminders just send normal notifications, not full-screen alarm-clock-esque UI - As far as I can tell, there's no way for third-party alarm clocks to function - Alarm clock snooze/dismiss UI seems to be private API in OS reserved for the stock alarms app. Kinda like we have the gnome-clocks gnome-shell integration DBus API - > Midnight Alarm app seems to by able to present a custom UI here too, somehow... I reached out and asked the dev how they've achieved it - Critical Alerts: given a special entitlement from Apple, apps can send a super-high-priority notification that completely ignores the system's mute switch, DND settings, focus mode, etc to immediately present a notification and make sound. - Apps launched directly from the lock screen - Examples: calculator, camera, create a quick note, etc - Prior art: Android (tested on Samsung OneUI 6.1) - Has customizable shortcuts in the corners of the lock-screen - Any app can be launched from these shortcuts, but only a limited sub-set of apps will actually run on the lock screen, as listed. All other apps prompt for unlock first and then run in normal session - Camera - Photo gallary is blocked out. Trying to open it brings up message "No photos or videos taken since phone was last locked" - Cannot use third-party camera app - Calculator - Trying to open the calculation history prompts to unlock device - Voice recording app - List of recordings replaced w/ a button to unlock the phone - Samsung Pay - Launches to the default "Quick Access" screen, where you can tap the fingerprint scanner or enter a pin to enable tap-to-pay for a couple of seconds - Trying to do anything else in the app prompts to unlock the device - Google Pay - On the Samsung phone there's no way for me to make it appear on the lockscreen other than probably setting it as the default tap-to-pay app and tapping a payment terminal - On my old OnePlus 8, I can make Google Pay one of the apps that launch on the lockscreen and it's similar to Samsung Pay. It has a special lock screen activity that only appears on the lock screen - Shortcut to turn on/off flashlight, not an app - Shortcut to turn on/off DND, not an app - A special version of the "create a new note" screen from the notes app; only appears if I remove the S-Pen while the screen is off and start writing on the screen. However might be part of system UI rather than the notes app - Double-tapping side button (aka power button) will either launch the camera or a custom app, with same behavior: default camera will launch directly on lock screen, everything else prompts for unlock - Not entirely sure how it's implemented but, - Activities have an API to appear over the lockscreen `setShowWhenLocked` - This API continues to work even in lockdown mode. Media also keeps playing. This tells me Android doesn't re-encrypt your session when the device is locked down :grimacing: - Some third-party apps can interject themselves on top of the lockscreen (if you give them permission to) seemingly w/o a notification. So there's nothing really preventing custom apps from running on the lockscreen other than they weren't written to try and do so and the OS pretends it's impossible? - I can't seem to find any third-party app that uses this functionality for anything other than "iOS 17 lockscreen for Android :sparkles:" and malware putting ads over top of your lockscreen. And these two categories of app are very likely the same thing anyway - Prior art: iOS - From memory, UX is very similar but I haven't tested it - Could be a bit different now that Lockscreen widgets exist, some of which let you launch apps directly. I can't imagine iOS will let those apps launch w/o first prompting for unlock - Not sure how it behaves when device is locked down... - Implementation: According to Jonas, similar to Android: lockscreen and apps just run on top of normal session and rendered on top of the lockscreen. Apps check if they're running on top of lock screen and disable parts of themselves when appropriate. - Tap-to-pay - Prior art: Android works like an app launched from the lockscreen - Payment methods can be opened on the lockscreen, authenticate the payment app, and tap to pay - Alternatively, tap to pay will open payment app, authenticate the payment app, then tap again - Tested on OneUI 6.1 w/ Samsung Pay. On OnePlus 8 w/ Google Pay, the experience is slightly different. You're not authenticating just the payment app, but unlocking the whole device. If device is already unlocked, tap-to-pay will immediately pay - Prior art: iOS - I have no idea how tap-to-pay UX works on iPhones. But I'm pretty sure it's baked into the OS and not replaceable w/ a custom app, so they probably have a private API for it in the OS - Android Auto / CarPlay - At least older versions of the phone OSs would block out the phone's UI w/ a window basically saying "you're driving right now look at the road or the Android Auto / CarPlay screen" - Prior art: Android - No longer blocks out phone like it used to - Whenever keyboard is opened on Android auto, for instance when typing destination into Google Maps, a special window opens on the phone (over top of the lock screen) that pops up the phone's keyboard and lets you type on the phone instead of the car - Prior art: iOS - IDK the UX of iOS here - Navigation app appearing on lockscreen - Proposed as something we want by @verdre - Use-case: you're on a bicycle, with phone mounted to handlebars. Need to have map visible to tell you where to go, and can't use hands to authenticate the device - I'm not particularly convinced this is a necessary feature; such an app should probably just inhibit sleep, locking, and screen off while it's showing navigation instructions. So no need for new API IMO. - Prior art: as far as I know, none? I haven't tried lots of map apps specialized for bicycles, though, maybe some of them exist ### :star: Solutions Given all this prior art, here's my proposal how to cover all the use-cases of apps-on-lockscreen: - Features similar to Apple's CallKit in our notifications APIs - Proposed at [Notifications Portal v2 proposal](https://github.com/flatpak/xdg-desktop-portal/issues/983#issuecomment-1989451570) - Ability to communicating incoming/ongoing calls to the system - Difference from CallKit: apps get to handle ongoing calls on their own. Actions in notification that should show ongoing call screen should present the app over the lockscreen somehow (i.e. special xdg-activation token), or prompt for unlock - Used to show special ongoing call indicator in the panel (w/ option to ruturn to ongoing call, again possibly over lockscreen) - Hang up via system UI/hw button/connected BT dev/etc - Properly handle cross-app call waiting - If getting an incoming call notif while we already have an ongoing call notif, present user w/ standard call waiting options and then send actions to both apps: hang up ongoing and accept incoming, hold ongoing (if supported) and accept incoming, or deny incoming - Side-idea: can be used to implement Apple-like "continuity" feature for calls. Just need an agent that talks to bluez and plumbs the various handsfree call controls that bluetooth can do into v2 notifications. Then any phone connected to laptop via BT magically works w/ this continuity stuff. Audio routing _already_ works (try connecting your phone to your laptop via BT and making a call), so we just need to hook up some control UI which can be done via notifications v2. Ongoing call UI for this service can probably be shared w/ the system dialer - At the OS's discretion, incoming call notifications can be presented in a full-screen UI rather than a small notification. - If showing a pop-up notification, tapping on it always opens full-screen UI - Probably should have hints to tell OS that an incoming call is a video call rather than just audio, so it can be reported in the UI like on iOS - Also, for incoming video calls, the fullscreen UI should have a camera preview. App does this by giving pipewire FD to notification v2 server. - Copy the entitlements system from the big OSs - Opened an issue about it: https://github.com/flatpak/flatpak/issues/5721 - We basically already have it via Flatpak static permissions - Just put some portal-esque APIs in a namespace that Flatpak doesn't give automatic access to. Then apps would need to set dbus-talk-name=org.freedesktop.entitled.WhateverProtectedApi - Alternatively, portals w/ some "entitled" features can check the app's `/.flatpak-info` file for permitted entitlements - Flathub will automatically flag any apps that try to talk to these APIs for special review before they're published - APIs we may want to implement and use this for: - Critical Alerts, to send notifications that cut through power saving, do not disturb, muted volume or vibration mode, etc - Apps that would be allowed to use this: extreme weather alert app (hurricanes, tornados, etc), earthquake & tsunami alert apps, other government alerts, home security system ("someone entered your house and failed to enter code within 30 seconds"), etc. - AlarmKit, described below - Remote should be able to revoke certain permissions from apps that abuse them - Something like `flatpak override`s but download from Flathub? - Trust model here: assume good faith by giving developers the entitlements they ask for, but expect bad actors by having a mechanism to quickly revoke these entitlements - "AlarmKit" - Basically an API for alarm clocks to continue functioning, possibly even from secure lock - Use notifications v2 for the system UI side of alarms - Proposed at [Notifications Portal v2 proposal](https://github.com/flatpak/xdg-desktop-portal/issues/983#issuecomment-1989451570) - Would behave similarly to notifications v2 for incoming calls - Posting a ringing alarm notif immediately permits the app to present itself on the lockscreen (i.e. gives special xdg-activation token). This lets fancy alarm clock apps have fancy "solve puzzle to dismiss" functions or whatever - If device is idle / locked normally, present a fullscreen alarm UI instead of a banner. An app can set a noBanner hint if it's expecting to present itself on the lockscreen instead - API to "please wake my app up at this time" - Gated behind an entitelement? - Wakes device up from sleep, maybe even power off if supported by HW, in time to ring alarm - Some generic alarm data can be stored w/ this API, so if device is secure-locked instead of spawning the app we just send an alarm notification on our own, thus presenting the generic alarm UI - Probably need to schedule RTC wake w/ system systemd manager so it works w/ secure lock, but IDK - We can treat calendar events and reminders exactly the same way like alarms, like on Samsung phones. But these probably shouldn't use the "please wake my app" API for privacy reasons. Don't give entitlement unless there's good reason to - Private OS API providing UI for tap-to-pay experience, or travel apps - Android, at one point, let payment apps export tap-to-pay compatible cards into the OS, and they'd show up on the power menu. It's quite a useful feature that we could copy... IDK why they removed it - I liked it and used it regularly, but I guess either many people didn't know about it or it was confusing - The whole chain is special and maybe privileged: authenticating the user, communicating directly w/ NFC hardware, getting notified about when the device is close to a payment terminal, securely storing card data in an enclave, etc. All needs special fancy API. Can throw UI into there too - Very very far off in the future so we probably don't need to worry about it right now. Point is that it's a solveable problem - We can't fully escape putting some apps over top of the shield - Use-cases: - basic system app utilities user launches directly from lock screen (calculator, camera, new note, sound recorder, etc) - parts of system experience: ongoing phone call, tap-to-pay, Android Auto / Car play keyboard, etc - ongoing video calls, so user can participate in call w/o unlocking device - AlarmKit custom alarm clock UIs (when secure lock is inactive; when secure lock is active, alarm coming from AlarmKit will show up w/ OS's built-in alarm clock notification UI) - I have a couple ideas about how to implement this, which I'll detail next. #### Idea 1: Switch back to session shield This idea is quite simple: whenever we want an app from the session to appear on the GDM shield, we just switch back to the session (showing the session shield), launch the app, and show it over top of the session shield. Trying to leave the app (i.e. swipe up on the gesture bar, etc) will switch the device back to the GDM shield. Of course, if we choose not to use GDM shield, then we're already on the session shield and the app can just appear over top of it. If we're on the secure shield, no apps can appear. The OS would give any app that it deems should run on top of the shield a special xdg-activation token that the app can use to surface its window. Or maybe the app can use foreign-toplevel to hand over a handle to a window via a special dbus API. Point is that the app will be somehow allowed to put exactly one window over top of the lockscreen, and any pop-up window that's parented to the window that's allowed to be on the lock screen. And the app is only allowed to do this once the OS gives it an explicit go-ahead to do it once. Pros: - Simple implementation for both OS and third-party app developers - Trivially works w/ Flatpak apps installed in user home directory, since we're running them from the session Cons: - Worse for security: Part of the appeal of using GDM shield is to completely prevent apps-on-lockscreen being abused to escape the shield and re-enter the session. So if we're switching back to the session shield to show apps-on-lockscreen, a big part of the reason we may want a GDM shield goes away - Potentially janky transitions between session and GDM shields - Transition into the session shield while the app is opening, which is a full-screen transition anyways so not a huge deal - Transition back to the GDM shield if user tries to leave the app. This transition isn't particularly smooth on Android either, IMO not a huge deal either - Transition back to shield if we enter secure lock - When secure lock is active, no system apps are available - Not a huge deal, IMO, for most cases: camera/calculator/etc - Is a big deal for phone calls: we should still be able to use the phone as a phone even in a secure lock #### :star: Idea 2: Hybrid, lite This is pretty much idea 1, but we have special support for running the system phone-call UI on the secure shield. Nothing else works during a secure lock: no continuity, no calculator/camera/etc. The phone-call UI is the only system component that is treated differently and allowed to run on the secure shield Same pros/cons as idea 1, just fixes the issue of not being able to use the phone as a phone while in a secure lock. If the user is actively in a call when the system transitions into secure lock, the transition can be janky (as the call controls running in the session become unavailable, we switch to GDM, and open the system copy of call controls). Maybe it's not really a problem: If the device gets locked down during a phone call perhaps we should just hang up the phone call - after all, the user probably locked down the device manually trying to avoid some situation where their privacy is compromised We can possibly transition into idea 4 over time, if we really want things like the Camera to work on the secure lock screen #### Idea 3: Secure containers One sentence outline: we run apps directly on the GDM session System apps (calculator/camera/etc) would be made aware of the fact that they're running on the GDM or secure shield, and will change how they access user data & tweak their UI. These apps run un-isolated in GDM and get to survive a secure-lock. Third-party apps will run in a special locked-down container that prevents them from accessing user data, session services, etc but lets them talk to GDM's wayland server and the network. If a secure-lock is activated, GDM kills all of these containers (potentially ending an ongoing video call, whatever). As an alternative, we let the app fork off a second process just in the session, but we give it GDM's wayland socket so that it can render in the GDM session These special lock-screen instances of apps can recieve a bundle of state from the session's instance (as long as the device isn't secure-locked) when they're started, to be used for things like authentication tokens. Use-case: answering a WhatsApp call runs the special copy of WhatsApp on the lock screen, and the sesion's copy of WhatsApp gives an auth token to the lock-screen copy so that it can authenticate w/ the WhatsApp server and join the call. When the device is unlocked, the lock-screen instance can also hand a bundle of state back over to the session's instance of the app. Use-case: the WhatsApp call that the user started on the lockscreen can continue in the session once the user unlocks their device. There should also be some special space that these apps can use to save data persistently until the session is unlocked. Use-case: the camera app on the lock-screen stores all the pictures it takes in `/var/cache/.../<username>/.../*`; then once the device is unlocked later the camera app is started to pull all the files out of `/var/cache` and move them into `~/Pictures/Camera` where they belong. Pros: - Secure! We stay in GDM, making it much harder to escape back into the session and bypass the lockscreen - System apps remain available during secure lock - No janky UX while we're on the lockscreen (smooth transitions when trying to leave/enter an app from lock screen, etc) Cons: - Very complicated to implement, for everyone: - OS: new strange containers with new levels of privileges, and apps running in GDM. All huge changes - Third-party apps: multiple processes to deal with, serializing data and passing it around, etc - System apps: again multiple processes and juggling serialized data, but also need to be built to survive the transition from normal lock to secure lock and wipe sensitive data from memory / the screen - Janky UX transitioning across an unlock - Unlocking is much more common than entering a secure lock or returning to the lock screen, so IMO this is a bigger deal than the jank introduced by ideas 1 or 2 - If something goes wrong we can lose data, drop calls, etc - I'm sure there's a million edge-cases I didn't think of that will break because we have multiple instances of an app running and trying to pass data around instead of just one. How will sound smoothly transition? Access to the camera? etc etc etc etc - I have no idea how we'd support Flatpak apps running in the user instance - Third-party apps are Flatpaks, probably installed in the user instance. How do we run them? Pretend that the user's actually Flatpak user installation is the GDM user's installation? I'm sure there's lots of issues/edge cases that come up here too... #### Idea 4: Hybrid, full Third-party apps (AlarmKit custom UIs, ongoing video calls) will always use idea 1, but system apps are allowed to use idea 3 so that they can continue working while the device is secure-locked. Most system apps would opt-in to idea 3, but some won't where it makes no sense (notably: tap-to-pay) This means that things like the Camera and calculator can keep working on the secure-lock screen. I'm honestly not sure if we want that, or if it's worth the extra complexity We can slowly implement this over time if we start from idea 2. But this of course will expose more system apps to the negative trade-offs of idea 3. Honestly seems like the worst of all worlds... ## Special UI for secure/GDM shield ### Idea 1: Special "unlock" session We would need to implement a special session (w/ class set to "lock-screen" in logind) that is started to service the unlock screen of the user. It would let us make the secure-lock experience behave a lot more like a locked-down variant of a normal shield (i.e. propagate more settings, have the switch-users button in the same places, etc). Doing it this way is essential if we're going to implement the GDM shield idea. On the other hand, it's more complicated than idea 2 ### :star: Idea 2: pre-select the user on the existing login screen Don't do anything special here: just switch to the normal GDM login screen, and pre-select the user that we just secure-locked. This is nice and easy to implement, and doesn't degrade UX by all that much since secure-locking already lacks so much of the experience of the normal shield, and we're already switching between VTs anyways. The various sharing of user settings would thus be made consistent between the secure shield and the greeter session (since they'll be one and the same!) ## User settings on secure/GDM shield See: https://gitlab.gnome.org/Teams/STF/homed/-/issues/34 List of settings should be propagated: - Background/wallpaper - Could go into homed user record & blob dir - Already have `login-background` homed blob defined - > Dark mode wallpaper? We could have a gnome-settings-daemon service that switches the `login-background` depending on light/dark mode settings of the shell, or put both into the blob dir and ferry along the color scheme preference, or pick one to always use. GDM is always in "dark mode", right? So we should always use the dark-mode variant of a wallpaper if it's available? - > What component should sync this? - AM/PM vs 24hr clock - This could go into the homed user record - accent color, once it exists - This could go into the homed user record - Keyboard layouts - This could go into the homed user record - a11y settings > Talk to people who know about the a11y system - display layouts - This probably shouldn't go into the homed user record - We probably don't want these settings to appear on the login screen when a user is selected, but we do want them to show up in the GDM/secure shield - Idea: why are we still using dconf at all? Maybe we should make the host system's gsettings backed by a file directly like it is with Flatpak apps, and then this file can be opened by the session and passed into GDM? > Talk to the design team about how this should behave > How do lockdown settings play into this? ## Animation - We cannot have a normal shield slides up/down over session content animation because we're actually switching between VTs to show the secure shield - We can't reveal content from under the shield because there is no content under the shield in GDM - Thus, whenever we use the GDM or secure shields, the current animation used by the session shield isn't available - Actually, the unlock animation is already broken even on the session shield. The animation kinda seems to start, but then the whole shield disappears w/o animating the rest of the way - Not criticial for MVP