by Kevin Seals

Smart speakers and A.I. will give your physician superpowers

As a hybrid physician/engineer, I spend a lot of time pondering how new platforms can empower doctors.

I am particularly excited about the potential of smart speakers coupled with advances in A.I. and natural language processing (also looking at you, blockchain). I am bullish on conversational agents in general, previously building an iOS chatbot powered by Watson that simulates a human radiologist. Chatbots are cool and useful, but voice — that might be magic.


Sensing potential, I decided to hunker down with my trusty corgi, drink a bunch of coffee, and start building the cool voice tools I want to use in my own clinical practice. This experience made me a lot more excited.

In this article I will synthesize my findings, show a bunch of fun demo videos, and explain why smart speakers represent a transformative technology in healthcare.

Why should people that care about healthcare innovation start thinking about smart speakers? Well…

Smart speakers massively empower surgeons

Imagine for a moment that you are a surgeon. You meticulously scrub your hands and undergo the long and complex process of surgical preparation, methodically putting on sterile gloves and a surgical gown and entering the operating theatre, scalpel in hand, exposed abdomen on the table.

You have entered the world of sterility. You are now incapable of checking your phone, which sucks. More importantly, you can no longer use a calculator, consult a medical reference, check the patient’s record, jot down a note — you can’t even Google things.

Smart speakers thus offer immense value to all surgeons and proceduralist physicians: they give them their modern technology back. Many important applications can be built around sterility needs, from software allowing physicians to dictate paperwork during surgical downtime to this simple (but useful) sizing tool:

The tool I am building uses voice to empower the sterile. The smart speaker allows surgeons to rapidly determine if a particular stent or other device fits within a particular catheter. There are hundreds of devices deployed using catheters, and remembering what fits in what becomes impossible. The current workflow involves constantly asking support staff to check reference materials…an awkward and painful game of telephone.

As a case study of one major sterility need, consider the workflow for determining if a particular device is available (for example, a stent of a particular size).

The surgeon says, “hey…do we have any 5 mm stents?” A human assistant then leaves the room, walks some distance to a storage area, and rummages through piles of boxes looking for stents. Precious minutes later they return and report their findings.

This is insanely inefficient, and a voice application allowing surgeons to rapidly query the inventory will be a game-changer. Please consider giving me a small finder’s fee when this makes you $1 billion.

Smart speakers facilitate eye contact and patient connection

Next, imagine that you are a physician in a busy emergency room. You have to see 20 patients before lunch, and after seeing them you have to complete mountains of paperwork documenting your findings.

This documentation generally occurs from 5-8 pm as you miss a dinner reservation with your wife and struggle to remember the details of your third patient with a cough. You were also supposed to play fetch with your corgi, and you missed it.

He is a good boy and he deserves fetch. Shame on you.

These time pressures create physicians that are buried in a computer screen when they should be focusing on their patient, making eye contact, and creating the connection that is fundamental to the physician-patient relationship. And patients are getting mad:

Smart speakers solve this problem. They allow physicians to chart data in realtime during a clinical encounter while continuing to make eye contact with the patient. This is huge. It both increases data accuracy — you don’t have to remember it later — and creates a better patient experience.

It also creates a better physician experience: we became doctors not to jump through documentation hoops but to take care of people and provide excellent, personable care. Everybody wins.

Smart speakers allow zero-friction access to high quality information

Friction is devastating to busy physicians. When you are responsible for 40 hospital patients and time delays are potentially deadly, making an additional click or opening an additional program becomes maddening.

Healthcare thus places a premium on immediate information accessed seamlessly. Sure, you could take 3 minutes to look up the latest recommendations for lung nodule management, but it is much better if you can simply “ask the room” and get an answer in 5 seconds.

This logic inspired me to build a radiology assistant that helps radiologists (my specialty) rapidly access useful information:

This tool allows radiologists to quickly access important, yet difficult-to-remember, bits of information that are commonly looked up. And it lets them do so in a split second by simply “asking the room.”

Hyper-efficient information retrieval is particularly valuable in the setting of a medical emergency.

Imagine a cardiac arrest with a critically ill patient getting chest compressions and electric shocks to restart their heart — it would be awkward and potentially unsafe to use a smartphone, but you can easily consult Alexa. She might retrieve key information from the medical record, ensure optimal timing of chest compressions, coordinate large teams distributed throughout the hospital, and do many other useful things.

Zero-friction information retrieval also facilitates the use of higher quality information. Try asking your doctor how much radiation you get from the body scanner at the airport, and they will probably respond with confused generalities.

But imagine if they could simply “ask the room” and get a better answer in a split second:

The human brain is imperfect, and voice tools help nudge physicians in the direction of accessing better information and providing better care. And they make it easy.


Hey engineers and healthcare innovators: I encourage you to think about how you can use this platform to build something cool and important that helps people. If you are curious about how I made my voice apps, see the nitty-gritty implementation details here.

If you have an idea for a project or want a physician’s perspective on anything in tech, feel free to reach out on Twitter or LinkedIn. Thanks for reading!

If you enjoyed the article, thought the demo videos were cool, or just appreciate my awesome dog, please hit the “clap” button and/or share…it helps a lot! Thanks. — Kevin