Normally, this is where I’d put my slides, but my slides weren’t the drawing factor of this talk. Unlike my keynote, I also didn’t write the whole talk out ahead of time, so I don’t have some magnum opus to post here like I did yesterday for the keynote. That’s okay, though, I do have some resources associated with my talk that belong here on this post:

Apple Resources You Should Read:

• Apple Platform Security Guide
• Apple Platform Deployment Guide
• What’s new in Apple Platform Deployment
• What’s new in Apple Platform Security Guide
• Use Apple products on Enterprise Networks

These resources should be on the bookmarks list of every Apple Admin. They represent an incredible collection of knowledge that is ever-growing and ever-changing.

Historical Changes to macOS

Beginning with OS X 10.11 El Capitan, Apple began to protect operating system resources by preventing the user, even if they were privileged, from editing parts of the operating system. This rootless change was Apple beginning to assert positive control over the key parts of the system that the user does not need to edit. At the time, controversial amongst some admins, it began Apple’s journey to protect the operating system as a privileged location only Apple could touch.

Files that were marked with the extended attribute of com.apple.rootless were unable to be deleted or written by any user, even root themselves. This was most of the operating system. Apple released a KB, now updated, that explains the feature thusly:

System Integrity Protection is a security technology designed to help prevent potentially malicious software from modifying protected files and folders on your Mac. System Integrity Protection restricts the root user account and limits the actions that the root user can perform on protected parts of the Mac operating system.

Before System Integrity Protection (introduced in OS X El Capitan), the root user had no permission restrictions, so it could access any system folder or app on your Mac. Software obtained root-level access when you entered your administrator name and password to install the software. That allowed the software to modify or overwrite any system file or app.

Apple, About System Integrity Protection on your Mac

Four years later, macOS 10.15 Catalina introduces volume groups for APFS and moves the operating system into its own read-only volume that’s part of the volume group.

This new read-only volume is the next layer of security for macOS. But if it’s read-only, how do you update it? You mount the volume as read-write, and then write to it, using the same identifying information that allowed Apple to write to the rootless directories previously: special code-signing entitlements which allow only Apple to touch the operating system and change it.

macOS 11 Big Sur announces a sealed system volume, which means that Apple makes a map of the system, creates hashes of all the files contained therein, and then signs that final hash. If your device cannot validate the hash, it boots to recovery mode and requires a new OS be downloaded and installed.

The inimitable Howard Oakley writes, of this new seal:

Once complete, and protected by SIP, the installer then creates the Merkle tree of hashes up to the Seal, the one hash to rule them all, and makes a snapshot. The tree of hashes and its Seal are then stored in the file system metadata which make up that snapshot. The sealed snapshot is then mounted and the System volume itself is unmounted.

Eclectic Light Company: Is Big Sur’s System volume sealed?

Updating macOS means creating a new read-write snapshot from the Sealed System Volume snapshot, updating the operating system, and then converting the snapshot to read-only, and re-sealing the volume.

At the conclusion of the operation, the new snapshot is marked as the boot volume, and things begin from there.

Update Mechanisms in 2023

There are three ways to update macOS in 2023:

1. User-driven updates
2. MDM Commands
3. CLI Commands

Okay, that’s an overstatement. There are really two ways to update macOS in 2023:

1. User-driven updates
2. MDM Commands
3. CLI Commands

Sure, you can truck with softwareupdate but I strongly recommend for your mental health and the security of your systems that not do this unless you have to.

User-Driven Updates

The concept here is simple: get your users to the dirty work for you. Open System Preferences or System Settings, go to Software Update, get them to do the update.

Pros: users control the experience, no passwords or credentials need be sent to the device, updates are largely bulletproof.

Cons: users just, like, don’t do them. No amount of deadline helps.

MDM Updates

Mobile Device Managers are also allowed to send MDM Commands that can govern the update process. The cycle as designed today is as follows:

1. MDM asks the device to check for updates with ScheduleOSUpdateScan command.
2. Device receives command and schedules an update scan.
3. Some time later, the scan is complete, the device has a list of updates available.
4. MDM asks the device to respond with the array of updates available to it.
5. Device sends the array back to the MDM.
6. If desired, the MDM can send a command to ScheduleOSUpdate which requires an Install Action Type, a number of deferrals (macOS 12+), a priority, and a Product Key or Product Version.
7. The device will receive this command act on that InstallAction and download the update, and if requested to, install that update.
8. The device will gather the Bootstrap Token from the MDM for the purpose of unlocking the sealed system volume to conduct that update
9. The device will conduct the update and return the device to the user login screen.

There are challenges here, unfortunately.

MDM Commands offer six install types. There are three that need no explanation: DownloadOnly, NotifyOnly, and Default. Default will download and prepare the update, and ask the user to allow the install. The others do exactly what you’d expect.

InstallLater is the best user experience, but it is escapable even with just a reboot of the device. End users can keep their devices at a lower battery value which prevents the system from updating at all.

InstallForceRestart is a great way to make friends and influence people, but it’s also a great way to have your end users interrupted in unproductive ways, possibly including data loss.

InstallASAP is a compromise experience. It downloads and prepares the update, then tells the user it’s coming and starts a 60-second timer.

Community Help

The Apple Admin Community are a resourceful bunch. Open Source projects like Nudge and Super are absolutely ways around the challenges described.

Nudge is a great way to encourage the user-driven workflows above, as it is designed to launch a dialog window in the user’s field of view if an update is required. Nudge will get more and more aggressive as the deadline approaches. The deadline can be set by MDM Configuration profile or by JSON array, along with customizations related to the upgrade messaging.

Super is a great way to send MDM commands and user alerts from supported MDMs. Currently, Super is supported by Jamf Pro, and uses an MDM API key to send the MDM Update commands to a given device based on user input. It is a solid solution for Jamf users who want end users to deliver InstallASAP commands to their own devices.

Here is the thing: neither of these should be required to stay up to date, but they are required right now, due to the deficiencies in the MDM Software Update Command cycles. I hope that Apple understands that admins don’t want to deploy Nudge or Super, we just want to use the specified commands and make sure that they work.

It is my most fervent hope that Apple is working on better solutions. Were it my job to suggest features for Software Update in late 2023, I would ask for:

• Deadlines for install based on Apple release dates.
• Close deferral escape paths.
• Update alerts triggered by MDM should be customizable in time and persistence.
• Install Later should apply to Major Upgrades.
• Takeover the job of Nudge or Super.
• Spot problems that might result in Recovery Mode during the preflight and act appropriately.

Here’s hoping.