Supraventricular Tachycardia: Or, A Trip to the ER with my Apple Watch

Overall, I’m a pretty healthy person. My blood pressure’s normal, my resting heart rate is in the low 70s, my cholesterol is normal, my blood sugar is normal, and I can go for a good long bike ride or walk without feeling winded. I’m heavy — my BMI is obese — but I’m in good health overall. (General reminder that BMI is BS.)

I bought my Apple Watch Series 4 when Apple announced it this summer, an upgrade from my Series 2. I was attracted by the fall detection (I’m an award-winning accident prone fellow) and also by the new ECG feature. I have a family history of atrial fibrillation, and I’m now 40, so some precautions seemed wise.

This afternoon, I was helping a client move offices, mostly just deconstructing a simple network rack and moving access points into new space. I was doing some physical work, but nothing anyone would mistake for exercise. But, then I felt it. My heart was pounding. I got dizzy. Tunnel vision. I had to sit down.

heart rate city

I took my heart rate on the watch and it was over 200. I spent five years as a competitive swimmer, and to my knowledge I never got above 195. Even riding up Box Hill on Zwift didn’t get me over 170 this winter. 200 is scary territory. I remembered the ECG functionality, and googled how it worked. I took a reading.

img_1630

I didn’t know how to read it, and I knew I was in a bit of trouble, so I had a coworker take me up to MedStar Washington Hospital Center, a mile or two away. Triage saw me rapidly, and I unlocked my phone to show the nurse. She was setting up a more complicated EKG, but because my heart rate had dropped back toward normal, it might not have any clear result they could read beyond just normal operation.

As soon as the tele-doc came on screen, the nurse rotated my phone and put it up to the camera to show the doctor the rapid rhythm from half an hour earlier.

“Oh, that’s an SVT,” he said immediately.

I didn’t see what it had to do with Ford’s Special Vehicle Team, but he clarified that he meant Supraventricular Tachycardia. They wanted to make sure labs were taken, and that nothing abnormal in my blood work showed a more troubling cause. But the diagnosis was there in an instant, thanks to my wrist watch.

Both the attending and her supervisor wanted a look before the day was done, and I was sent home with instructions to go see my doctor (don’t worry, I’m going on Thursday), but now I’ve got something to show my medical team, as well.

Sure, a lot of the time it feels like we live in a dystopian version of the future, and I’m still not sure where the flying cars are, but today I used my wrist computer — list price $399 — to take an ECG before arriving at the emergency room, where a doctor, appearing in my room via video conference, was able to read that medical diagnostic and make a snap judgment that I was probably going to be alright for now.

Apple remains a company that exists five to ten years into the future, building bridges back to the present. Touch ID and Face ID. Secure Enclave. Device Enrollment Program. Apple Watch Series 4 Health Tools. Perfect? No. Better than the rest? By miles and miles.

Thanks, Apple. My heart is in your hands, it seems.

img_1591

2018: Arbitrary Boundary Condition Met

Sunset at Asilomar Beach, December 29th, 2019

The problem with linear time — well, one of them — is that you don’t always know when your personally meaningful boundary conditions have been met. Life is uneven, some chapters are long and interesting, some short but sweet, some arduous and never-ending. 2018 fell into a lot of those categories. So we’ve met our arbitrary boundary condition prescribed by the journey of the Earth around the Sun. Let’s look at what happened?

Crash Migration

2018 began with a crash migration for one of our clients. We had 26 days to handle their office move, and brought them into new digs on time and with complete operating functionality, despite the short timeline. I’m thankful to, of all people, Comcast Enterprise for bringing their might to bear and they brought their A game and got us a gigabit circuit in almost no time at all. Crash Migrations always feel like a bit of a trial-by-fire, and this one was no exception.

41 Episodes

The Mac Admins Podcast had an incredible year, and I couldn’t be prouder of the team. We produced 41 episodes of the podcast, including The One With Apple, live episodes at JNUC and MacDevOps YVR. We talked with Apple Luminary Sal Soghoian, Fraser Speirs about the State of iOS in Education, Thomas Reed about Graykey and iOS, and Tim Perfitt about Secure Boot and the best way to kill a chicken. 2018 brought more than 175,000 downloads of the podcast!

Here’s to more and more episodes in 2019! I’m hopeful our conversation with Apple gets a sequel. If you want to see us do a live show, come on out to MacADUK in March 2019!

Cloud Living

With the, um, active retirement of macOS Server, I sunset the code for Munki in a Box, but not content to just abandon the idea, I also released Munki in a Cloud which works with AWS. If I were a better coder, I’d be combining it with Graham Gilbert’s excellent Munki Terraforming project. I guess I just found my first 2019 goal.

Frontal Boundaries

We also moved our primary software distribution platform from an on-premise Munki server into AWS’ CloudFront. We’ve moved almost a third of our clients to it already, and we’ve got planned migrations for a bunch of the rest. Serving client updates via CloudFront was a really great experience for us from a budget perspective. We centrally manage manifests and applications from a workstation on our network, do QA, then push to production. We’ve got a secure distribution system that I’m pretty proud of. And it cost us much less per client than even our wildest dreams.

What’s The Future Bring?

2019 I’m getting on the stick about two specific things: Python and the SimpleMDM API. I’ve said it before about Python, but I’ve actually accomplished some small tasks this way, so I’m excited to tweak a few more things into Amazon Lambda, Python and the SimpleMDM API as part of our goal to make better touch-less workflows in 2019. I’ve got some ideas for an open library of Python scripts for using the SimpleMDM API, but I need to get some tasks genericized and working, first.

I’m looking forward to 2019, to whatever macOS version ships in beta form come the summer, the demise of kexts and 32-bit applications, and more MDM options.

If the last three years have taught me anything — as a father, as a business person, and as a Mac Admin — it’s that being ready for anything means approaching everything like it’s an angry emo porcupine with lethal quills: carefully, thoroughly, and with as much empathy for the problem as you can muster. Everything’s always changing. We’re always building new things, always undoing old mistakes and making new ones. We’re going to keep that up. The constant is change.

Holding strong opinions loosely is one way to avoid the ossification to orthodoxy that can keep you from seeing where the Future is. Getting stuck doing things one way because it’s how you’ve always done it is a great way to miss what’s coming. Chasing the Future means being willing to abandon work that you’ve done, and that can hurt, because a problem solved elegantly comes with it a certain satisfaction in overcoming entropy through clever application of technical knowledge. But staying with an old solution when a new one avoids the problem entirely is unwise in an ever-changing situation.

Go forth, my friends, and solve new problems in 2019. Solve them together, toward making computing a more seamless and enjoyable task for all the participants. The simultaneous promotion of all interests — usability, security, and repetition — is possible.

Sardines at The Kelp Forest, Monterey Bay Aquarium, December 30th, 2018
Image

Using Prey with SimpleMDM to Recover a Stolen Laptop

Monday morning, I got the call that no one wants to get: “The lock was jimmied. They got some of our computers.”

Immediately, we sprang into action. We’re big fans of SimpleMDM and Watchman Monitoring, and both of those tools came in handy. The first thing we did was check the logs from Watchman Monitoring’s client agent on the machine to see if it had checked in over the weekend.

One of the machines had checked in on Sunday! We set both to alert us if they checked in again, and logged into SimpleMDM to see if the device was checking in there, as well. We could see the one device, which gave us a couple different options: We could lock or wipe the machine and hope that it wouldn’t just end up in a landfill, or we could try to get the machine back by giving some data to the police.

I know from experience that just giving them an IP Address isn’t likely to get a good result, so we started to think what else we could we do to get the machine back? What if we could give them a location, and more information?

Screen Shot 2018 10 17 at 12 28 27 PM

Enter the Prey Project. The Prey tool works as a behind-the-scenes agent on your behalf. When it’s in regular mode, it’s not doing much. But, when you turn on Missing Mode, things get a lot more interesting. Your Mac will now check-in with nearby Wi-Fi networks, perform a full location scan and give the police something to work with. It will also take pictures with the FaceTime Camera on the Computer, and capture screenshots, giving you more material to work with:

Prey Screenshot with Wireless Networks NearbyPrey Screenshot with Map Detail

This post isn’t here to get you to buy Prey, but it’s to tell you how we got Prey installed when we didn’t have the machine in our full control.

By default, Prey requires an API key to register new machines, and their method is just “Hey! Install that at the Command Line by SSH’ing into the machine!” Which, okay, fine, that might work if you can get that far, but how’s about we do something a little bit different?

What we opted to do was to repackage the Prey installer, so that the package installer they built is stored in a common directory, (in our case, /Users/Shared) and then a postinstall script tied to the package handles the install with our API Key:

#!/bin/bash

API_KEY=0xdeadbeef /usr/sbin/installer -pkg /Users/Shared/prey-mac-1.8.1-x64.pkg -target /

To build this package, we used Packages from WhiteBox. I created a new project, gave it a name (Black Widow), Included our Prey installer package to a known directory, and then add a Post Install script to invoke it using our API Key.

Screen Shot 2018 10 17 at 3 09 08 PM

Packages Post Install Script Screen

This gave me a functioning package that installed Prey and keyed it to our instance, which was great! But, how do I get it onto the stolen machine?

Enter SimpleMDM. You can use SimpleMDM to install a package onto a device, but only if you have a properly signed distribution package. The Black Widow package I made in Packages was unsigned, so now I just had to properly sign it using the prodsign command:

 

Screen Shot 2018 10 17 at 3 10 12 PM

This gave us a properly signed package with a valid signature:

Screen Shot 2018 10 17 at 3 17 52 PM

After uploading the package to our SimpleMDM instance, we scoped it to the machine, and waited for its next check-in:

Screen Shot 2018 10 17 at 3 19 16 PM

From there, it was a waiting game until the person who had the laptop now was back in range of the internet. Sure enough, they came back online today:

Screen Shot 2018 10 17 at 3 20 58 PM

The machine’s location and positioning information, as well as some additional detail, gave the police something to use to be a little more active on the case. We’re now waiting to hear if they’ll be able to repatriate the laptop to its owner.

Twenty-Four Hours with an Apple Watch, Series 4

Apple Watch Series 4

I didn’t wear a watch at all from about 2002 until 2015. Watches seemed to be an affectation of the wealthy, who could lay out thousands, or tens of thousands, for a chronograph. If I needed the time, that’s what my cellphone was for, or there’s clocks just about everywhere now. The original Apple Watch (later given the moniker Series 0, or S0) changed my opinion about what watches were for: they’re not limited to the time, they’re capable of being a polite tether to everything else you’re needed for.

When I added the S0 to my daily wear, I ended up naming it Bates, from the Julian Fellows drama Downton Abbey. John Bates was the Earl’s valet, there to remind him who he was meeting with, handle his schedule, and keep him on-time and looking sharp. Not the major domo of the house, or the butler, but rather someone with limited responsibilities. That’s my watch. It could tell me when I was supposed to be somewhere, it could tell me about who’s calling, and really, that was about it.

The Series 2 (S2) that followed on two years later was a substantial improvement, it could run apps, for one, or at least it did on occasion, when it felt like it. The S2 was still largely limited to basic duty: texts, timers, the occasional phone call (but only close by the phone) and telling me if I was getting up and moving enough. I liked the Nike version’s different face, but it wasn’t any more expensive than the standard S2, but their sport bands are the best of the bunch. Most of the time, I ended up wearing the sport loop with hook and loop closure for comfort and breathability.

I was a pretty happy wearer! I wasn’t interested in an upgrade! When the S3 came out, the cellular functionality seemed of limited usefulness, and so I didn’t run for the update.

But then came the Series 4. Fall detection. Better heart rate monitoring. An on-board ECG.

This June, while I was in Vancouver for MacDevOps, I woke up one morning with chest pain. There’s no online medical documentation for chest pain that doesn’t also say “SUMMON THE MEDICAL PERSONNEL AT ONCE. YOU ARE PROBABLY DYING.” So I got a cab to the emergency room nearest my hotel and experienced the Canadian Medical System. Blood tests were indicative that I was fine and that it wasn’t cardiac problems presenting itself. I knew it probably wasn’t when I went. But do you take that kind of risk when you’re thousands of miles from home?

Sadly, that adventure cost me about $1,000. If I’d been here in DC, $75 copay for an urgent care, but traveling far? Alas, I’m out a bunch of money. If I’d had a way I could’ve done some basic diagnosis, I might’ve let it wait til I was home in DC before I sought assistance. Turns out it was a strain of an intercostal muscle between my ribs right at the sternum where my shoulder bag rode. I’ve switched back to backpacks, and all has been well since then.

More than just the health feature, the larger 44mm display supports a new more information-dense display, and the Infograph face as displayed above is the right level of information overload for me. Having to cherry pick one or two complications was difficult, but now I have eight slots to choose from, plus a calendar line item. It’s exactly the right level of ambient and purposed information.

The physical wearing of the watch is also a major upgrade. The thinner but taller case is absolutely welcome. I was afraid of having an aircraft carrier on my wrist, but this is the right level of size. The brighter OLED screen is impossible to miss, and I feel as if the new more sensitive gyroscope makes the raise to wake function much more accurate.

In addition, the new S4 system-in-a-chip is finally, finally fast enough to run applications on the watch itself. Though the number of applications I want to run on my wrist is small, Overcast for podcasts paired with my AirPods is a dream come true. I can leave my phone at home instead of carrying it with me.

I did sign up for the cellular model, and I will say that the setup of the number mirroring was flawlessly easy. The setup of the watch as a whole was spectacularly easy. I had given myself an hour’s time to get the watch picked up and paid for, and then setup. I needed all of 40 minutes. That included the wait to pick up my equipment, unboxing, and restore from my backup, and setup of the cellular line. I was in and out of the Apple Store well before I’d laid eyes on the gold stainless model I just couldn’t justify the extra $200 for, and didn’t want to regret seeing in person!

Post restore, my battery was at 85%. It was 11:45 am

From there, it was a drive from Arlington to Ellicott City, done with Apple Maps giving me queues on my wrist, letting me feel the new Taptic Engine. It’s an improvement over the S2, which either felt like too much, or you couldn’t feel at all. An hour or so later in traffic, I’m to my meeting. Phone in the pocket, and in I go. I received a couple calls – sent to voicemail – and a few texts, which got trivial replies from my wrist.

Meeting concluded, it was back in the car and back into directions mode. I ended up at IKEA to review some cabinetry. As IKEA is best taken at a brisk walk, I figured I’d see if I could trigger the workout detection. Turns out I’d need to have walked faster. I blame the crowding. Anyway, it was adept at capturing it as “exercise” just not “workout.” Off to the house. This time I took a call on my wrist, and wow was the speaker a major improvement. Not something I want to do more often, but something that’ll let me leave my phone wherever.

It’s 5pm, and I’ve told a friend I’ll meet them at Nationals Park at 6, so I pack up and pick a bikeshare bike from our rack at the corner, and setup a workout. 33 minutes of good biking later, I’m docking down at the ballpark. It wasn’t fast riding – the bikeshare bikes are heavy and not built for speed! – but it was a good test of the battery effects of a “normal” workout.

Apple Watch Workout Detail

Apple Watch Workout Map

I swipe up into control center. It’s 5:50pm, and I’m still at 45%. Rock on.

The Nationals do their best to show their 3rd place position, but come up short against the Metropolitans. I use Apple Pay at the concessions to buy dinner, and then a soda later on. After the game, it’s home via the Metro with a friend, and another short ride home on Bikeshare. At 11:30, I put my watch back on the charger at 30%. Not bad.

Shortly before midnight, my friend Kelly texts me that she’s gotten her watch, and can we test Walkie Talkie? Yes, for sure. I do the brief setup of the Walkie Talkie app and add Kelly as a contact. I get a ping on my wrist and hear Kelly’s voice from a continent away. We have a good conversation back and forth via the Walkie Talkie. It’s different than a phone call, somehow less expectations, more tactical. I can see this being great for Amusement Parks, big retail stores, and other places where you might

Overall, the Series 4 Apple Watch represents a huge refinement of Apple’s vision for the future. I am still becoming comfortable with the idea of leaving my phone at home when out on a ride or out for a long walk. I will get there, but I’m not there yet. Count me in love with the Series 4 in a way I never was with the Series 2 or Series 0.

Initial Release: Munki in a Cloud

I wanted this all to come together months ago. It hasn’t! But it is in a state that I can release it.

Welcome, Munki in a Cloud.

This is the initial release of a product that I hope I can get developed more fully. It’s designed to, on the host Mac, prepare a repository of packages for cloud distribution by Amazon Web Services’ S3 file service. It’s not fully complete, in that you will have to take some steps to either add a Cloudfront Distribution to the bucket, or prepare the bucket for public file service. It relies on the awscli command line library to create the S3 bucket based on a set of AWS Credentials, which you’ll need.

As with Munki in a Box, prepare your variables carefully and then fire the script off. Unlike Munki in a Box, you then need to either prepare your S3 bucket for public distribution (not always recommended) or setup a Cloudfront Distribution on top of it and distribute middleware and encryption keys to your clients.

I do want to automate the CF creation in the future, and Clayton Burlison’s munki-terraform seems to be the right way to handle this, I just haven’t been able to make my brain understand enough terraform to roll it in.

If you’ve got questions or concerns, I’m happy to hear them, please file an issue in Github. Pull requests will also be gleefully accepted.

Transitions: Munki in a Cloud

It’s MacDevOps YVR week, one of my favorite of the year. This morning, Clayton Burlison released an awesome package called terraform-munki that does something super helpful: it creates a set of terraform templates to create useful resources within your own AWS account to prepare an S3 bucket, and create a CloudFront Distribution with a TLS certificate.

This is exactly what I’ve been working around in my development of munki-in-a-cloud, which will replace munki-in-a-box due to the deprecation of Server.app’s web services by Apple later this year. I have the script done, except for the creation of the CloudFront Distribution, which I was reading all about when Clayton suddenly said “Oh! I did that! And I’m releasing it this week!”

So I’ll be figuring out which parts of terraform-munki are helpful to this new project and will get used or adapted into munki-in-a-cloud.

The goal is the same as munki-in-a-box: A script to create a functional munki environment and repository, and make it ready for use in the cloud. With a future version of macOS removing the Web service functionality entirely, it seems prudent to look at good cloud options.

If you’ve got opinions on a project like this, I’d love to talk more with you. Find me on the Mac Admins Slack to talk about it more.

Image

Installing Ubuntu 17.10 on ESXi, from a Mac Client, to the ESXi Server

This is a guide as much for me, as much as it is for anyone else. I came to the conclusion I wanted a testbed for Reposado and Margarita, and as much as Clayton Burlison has the install of Reposado and Margarita on lock, I needed a refresher on how to create a new Ubuntu VM on my ESXi-capable Mac Mini.

First up, in VMware Fusion, connect to the server. File > Connect to Server… will give you access to the virtual machines stored on the server. You will then see a list of all the VMs currently on the server, active or not:

ESXi Host VM List

From here, you can click the + at top left to add a new VM:

ESXi Host New VM Screen

Since we are putting the VM directly on the server and not our local machine, select “Create a virtual machine on a remote server.”

ESXi Host Server Picker

Next up, you will be asked to select the server. Choose your local ESXi host.

ESXI Host Choose Host and Datastore

From here, you get to select which Datastore you want to store the new virtual machine on. If you had multiple volumes, you could select it here, whereas I just have my internal storage volume.

ESXi Host Choose HW Vers

VMware will then ask you to select a Hardware Version. There might be reasons to choose earlier versions, depending on what your local situation is like, but I’m up to date, so I’m choosing version 11.

ESXI Host Pick Network

Next, you get to choose which Network you’ll put it on. If you had multiples, you’d want to select the correct VLAN. I just have one, so I’m keeping it right where it is. You can also have VMs that have no network interface, and that’s an option here, too.

ESXI Host Pick OS

Since I’m running Ubuntu Server, 64-bit, for the final project, I’m selecting this version also for my sandbox VM.

ESXI Host Pick Firmware

If you wanted to opt to use UEFI or Secure Boot, here would be your opportunity! Ubuntu doesn’t need that, so I’m just clicking through.

ESXI Host Pick Disk Size

Last but not least, it’s time to pick your disk size. Since I’m using Reposado and Margarita, it’s a 200GB minimum to enter this party.

Now that we have our virtual machine, we need to get our copy of Ubuntu 17.10 Server. I grabbed mine from the Release Notes Page, which includes links to the Ubuntu download system. As long as you have an ISO, you should be fine to get started. Before you turn the VM on, you need to attach that ISO to the VM’s CD-ROM Drive. In the Virtual Machine’s Settings, you can select CD-ROM, and then specify the locally-stored ISO file to use as a connected volume.
VMware Settings CD ROM

Once you have selected the Ubuntu ISO file to attach as a CD, you are free to boot your virtual machine, and you’ll be presented with the next few screens as part of the process.

Ubuntu 1 Language Select

Select your preferred language for the Installer to use.

Ubuntu 2 Install Starter

Select the option to Install Ubuntu Server

Ubuntu 3 Language Select

Select your preferred language for the __operating system__ to use

Ubuntu 4 Region Select

Select the preferred region for the __operating system__ to use.

Ubuntu 5 Keyboard Config

Pick the keyboard you’re using

Ubuntu 6 User Creation

Ubuntu 7 Set Password

Set up your admin username and password. Don’t forget these. Store them in a 1Password item if you can.

Ubuntu 9 Set Timezone

Set your timezone

Ubuntu 10 Set Storage
Ubuntu 11 Volume Config

Also set up how you want the volume to be formatted. Defaults are fine, but you might choose to use Logical Volume Manager to handle your storage.

Ubuntu 12 Actually Doing Stuff

Now, it will install the OS and you’ll get an occasional screen to set up an HTTP proxy, allow security updates automatically, etc.

Ubuntu 13 Proxy

Then you can choose to install just the server, or a bunch of extra tools. Since I’m using Clayton’s guide for installing Reposado and Margarita, and it has the needed download commands, I’m just going to take the OS as it’s given to me.
Ubuntu 14 Select Additionals

More installing will occur here. Get a glass of water, you’re probably not hydrated enough today.

Ubuntu 15 More Installing

After this, you’ll be prompted to setup the GRUB Bootloader. Since this is the only Linux install on the virtual disk (It is, right? It would be really weird if it weren’t.) you can accept this configuration.

Ubuntu 17 Grub

And after that all completes, it’s time to get rolling through to your VM!

Ubuntu 18 Time to Restart

You can eject the CD-ROM on the next startup cycle.

Ubuntu 19 Logged In

Once we’ve got the box up, we want to install sshd to allow for remote access, because while it’s nice to have direct command line via the VM, ssh is so much more convenient!

We’re going to need to do a couple commands here to get it:

sudo apt-get update
sudo apt-get install openssh-server

This will install the standard openssh server and prepare it for use, allowing you to login remotely. There’s a million fiddly bits associated with opens, and you may want to customize it so that two-factor works, or machine tokens like YubiKey tokens act as your key. That’s an exercise best left for the reader. For now, I’m not publicly exposing that interface as part of this process.

Now that you have an Ubuntu 17.10 server ready to go, you can follow all the instructons of Clayton’s guide for Ubuntu 14.04 for installing Reposado and Margarita (it all still works as of 17.10!)