The Heart of AI

The pandemic has opened the eyes of the healthcare world to the promise of artificial intelligence, which is where it all began ….

Robert Marcus
7 min readJun 2, 2021

If you’ve ever witnessed a human heart stopped and removed, repaired, and reinserted into the body, let me tell you it is a fearsome thing to behold. Every second counts, each decision, every surgical movement, the precise balance of blood flow and anesthetic, is orchestrated in something akin to musical symphony. But what really grabbed my attention was when one of the physicians introduced me to a new computer technology called an “expert system.”

This was 1995, when, as co-founder of a healthcare information system’s startup, I had my first exposure to AI watching open-heart surgery at the Dartmouth–Hitchcock Medical Center. The system used historical data from actual surgical procedures, along with information on the patient’s health and real-time vital signs, to infer (to reason) the best techniques for the doctors to use in this complicated, lifesaving surgery. AI did not tell the doctors what to do but guided them to optimal choices that produced measurably better patient outcomes.

Though primitive compared to AI solutions today, the application stunned me. I saw that computers were not just about automating manufacturing or increasing business productivity. I saw a machine not just tabulate data. I saw a ‘thinking machine’ extract insights from data and predict the future. The AI connected the dots forward. The real value of computers would be to form data into knowledge that predicted, even shaped, the future. I had been fascinated as a teenager by Nostradamus, the French astrologer who is best known for his book Les Prophéties, a collection of 942 poetic quatrains allegedly predicting future events. AI was the continuation of Nostradamus’s prophecies in another form.

Connecting the dots forwards …

In 1996 I discovered Open Agent Architecture (OAA), being developed at the SRI International Artificial Intelligence Center as part of an effort to develop intelligent digital personal assistants. OAA was using sophisticated Natural Language Processing (NLP) and Machine Learning (ML) techniques to answer questions, make recommendations, and perform actions by delegating requests to a set of agent-based internet services. In my mind, OAA evolved the idea of expert systems.

I agreed to an R&D partnership with SRI. The mobile wave was building, and I envisaged OAA on every mobile device, activating a sea of mobile applications. I was a decade too early (there’s a lesson in that). OAA went on to become the basis of Siri, releasing as an app for iOS in February 2010, and acquired by Apple two months later.

I joined Microsoft Corporation in 1997 to lead early work on the confluence of AI and mobile. I later headed the knowledge management business, immersing myself in data and ontological science, and authoring my first book, Collective Knowledge, on the topic. After joining the Microsoft M&A team in 2005, I began to explore the potential inherent in data-driven innovation and investment. That is another story …

One way or another, all of these experiences built on my interest in AI.

AI, which is capable of analyzing much more data than humans and of making better, faster decisions, will be at the center of almost any significant solution to the many problems facing our planet, particularly climate change. Long on promise, AI is finally coming to maturity. It has the capacity to search for and discover knowledge and to make well-founded predictions about the complex, multidimensional problems that affect our existence. It is the only approach that can grapple with issues as vast as the climate, healthcare crises, and whatever other threats humanity may create, or Mother Nature may throw at us.

Investors are betting big on healthcare as the leading use-case for AI.

AI in healthcare has come a very long way since my eye-opening experiences at Dartmouth. Also known also as Deep Medicine, it describes the use of algorithms and software to mimic human cognition in the analysis, presentation, and comprehension of complex medical and patient care data. Global fundraising for AI health start-ups has risen steadily since the end of 2019 and hit a new record of $2.5bn in the first quarter, says CB Insights. It was AI that helped Moderna to quickly sort through massive amounts of data to rapidly develop a vaccine for the novel coronavirus. Moderna says the application of AI improved its likelihood of success by a factor of five. AI is also re-engineering drug discovery and delivering life-changing medicines. Genetically based meds are the great hope for the future, and AI can screen genetic compounds for efficacy and side effects a hundred times faster than humans can.

The healthcare applications of AI are limitless. AI does a better job of predicting breast cancer from pathology images than a panel of eleven human experts. ML has been put to work on images, bloodwork, EKGs, and genomics to improve cancer detection. Pattern recognition is pinpointing cancer cells among normal cells. AI microscopes can predict harmful bacteria in the blood with 95 percent accuracy. AI can better classify conditions, refine diagnoses, and target treatments for specific biologies.​

AI has long been a boon for surgery. Ever more precise surgical robots are performing complex operations, reducing complications, lessening pain, and speeding recovery. They have done a better job than human doctors alone in excising some tumors while minimizing damage to healthy tissues. Synthetic vision can aid surgeons in difficult procedures. Machines have not replaced humans, but they have multiplied the capability of the human mind and body.

Healthcare, which encompasses R&D, hospital staff, independent medical practitioners, insurance providers, and the government, has some of the most complex and expensive business processes. It consumes a global average of 9 percent of GDP (in the U.S. it is a staggering 17 percent). AI may significantly reduce these costs. It is now analyzing healthcare operations to speed patient flow and simplify scheduling — even find optimal ambulance routing — and to resolve a vast number of operational issues.

Clinical and scientific research information can be extremely fragmented, dispersed across thousands of public and private databases, across multiple platforms, and the volume that exists grows every day at an overwhelming rate. ​By searching for and discovering knowledge and connections that produce predictions of the future, AI will generate unprecedented operational efficiency and innovation acceleration.

Microsoft surges towards healthcare as a half trillion-dollar addressable market.

Major platform providers see both the need and the opportunity for AI in healthcare. Microsoft, as one example, has begun a cloud-based AI initiative that includes such efforts as the Microsoft Cloud for Healthcare, introduced in 2020, and its recent purchase of Nuance, whose AI system uses NLP to capture and document, correctly and in context, the spoken exchanges between doctors and patients so that doctors can keep their attention on the patient. Microsoft estimates that the Nuance acquisition will double its total addressable market (TAM) in healthcare, bringing it to nearly $500 billion.

Healthcare introduced me to AI, and it continues to evidence the real-world application of the technology. The progress in direct, lifesaving applications — and the current state of the art — illustrates AI’s potential for many other critical issues. I have great faith in the future of AI and its capacity to address humanity’s biggest problems. The speed at which AI will generally accelerate ideation, innovation, and invention has no historical precedent.

In forthcoming articles I’ll explore different elements of AI: the urgency of many worldwide problems; the way AI’s unique attributes will help address climate and other critical issues; the steps needed to spur rapid deployment of AI solutions; the practical social, political, and financial issues involved in its deployment; and many other related matters.

Before I move on next time to the much broader potential of AI, I must cite one more ingenious healthcare solution, which after a quarter of a century brings me full circle. Carnegie Mellon is developing a tiny AI-directed robot, HeartLander, that propels itself like an inchworm across the surface of a beating heart. The tiny bot can attach itself to any location on the organ. Tests show that the AI-driven device can inject drugs, administer other therapies, and even insert electrical leads to which a pacemaker or other instrument could be attached. In my introduction to expert systems years ago, AI helped optimize open-heart surgery. Today, AI might soon replace this most invasive procedure with a minor chest incision.

It is for such lifesaving reasons that AI has inspired me and remained close to my heart.



Robert Marcus

Impact investor, Sand Hill Road VC, PLC CEO. Connecting the people, capital, and ideas, that solve the world’s greatest challenges.