In Computers We Trust: The Thrill of Victory, the Agony of Defeat

After a trip of a hundred million miles, the world cheered the landing August 6th of the rover Curiosity on the surface of Mars. In the 7 minutes after the ship entered our neighboring planet’s atmosphere, it sped up to 13,000 mph and then slowed to 1.7. Aiming for a target area of less than 50 square miles, the landing involved 76 pyrotechnic devices, a huge parachute, rocket thrusters, and finally a tether to drop the rover on land. 

At that distance, it takes 14 minutes for a signal to reach the earth, so at the moment the team got word that the ship had entered Martian atmosphere, 7 minutes had already passed since the rover had either landed or crashed. Half a million lines of code, $2.5 billion dollars, NASA’s reputation and future funding at risk, every step of the landing pre-programmed, zero margin of error, and the result: complete and utter success.

But not all the recent news regarding our reliance on computers has been as reassuring.

The August 3rd Los Angeles Times featured these two headlines: Knight Capital trading debacle shows Wall Street frailties and Patient data outage exposes risks of electronic medical records. That’s a pretty solid one two punch. How we finance society and how we save lives are being increasingly entrusted to computer systems, presenting risks on a massive scale. 

August 8th, a story appeared at wired.com and spread through the net: How Apple and Amazon Security Flaws Led to My Epic Hacking. Gmail was deleted, Twitter and ICloud compromised, hard drive wiped clean, photos and messages gone forever. This wasn’t the story of some naïve webster attacked visiting East European porn sites. No, this happened to Mat Honan, a reporter for Wired, compromised by his accounts with Apple and Amazon, two of the most trusted and successful online entities.

Knight Capital Group, a brokerage that handles nearly 11% of all stock trading in U.S. companies…lost $440 million in less than an hour — nearly four times the company’s profit last year… Knight’s software executed in a matter of minutes a series of trades that were designed to be done over a period of days. This glitch is just one of a series that have plagued Wall Street in recent years.

In the case of the medical info, we’ve been told that we must computerize our healthcare records and operations to be more efficient, effective, and cut costs. The medical field has been ridiculed for it’s continued reliance on paper.  

As part of the 2009 economic stimulus, the Obama administration is spending more than $27 billion to speed the switch to computerized patient records… In 2015, the government will begin imposing financial penalties against hospitals and doctors that don’t adopt electronic health records. 

Cerner Corp., a leading supplier of electronic health records to hospitals and doctors, said “human error” caused the outage July 23 [in which] …Dozens of hospitals across the country lost access to crucial electronic records for about five hours.

Many hospitals choose to outsource [electronic record keeping]… to save money or to avoid having to hire the staff required. Cerner’s revenue from remote hosting and similar services grew 20% last year.

When the computers went down last week, some hospitals lacked backup sources for critical information. A new team of doctors and nurses came on duty, and for hours were unable to read the reports of the previous shift.

Doctors and hospitals interviewed said they weren’t immediately aware of any patients harmed by the disruption, but they expressed concern about how long the outage lasted and the failure of any Cerner backup system to kick in.

In an op-ed in the August 9th New York Times, Errant Code? It’s Not Just a Bug, former software engineer Ellen Ullman makes the following points: It is impossible to fully test any computer system and there is no such thing as a body of code without bugs. 

I don’t have a brilliant answer for what any of this means or what we must do in response. But I certainly want to raise questions. How vulnerable are we? How serious are the threats we face? How dangerous? Will we ultimately experience irreparable calamity? Are there obvious lessons or conclusions? What must we do to minimize the chances of something going horribly wrong? 

4 thoughts on “In Computers We Trust: The Thrill of Victory, the Agony of Defeat”

  1. Worthy subject; I wish it were more widely discussed. A few comments, based on a half-century of software development experience:

    You may remember that the space shuttle disasters were basically human-caused, even though it was technology that failed in the moment. In both cases, if I remember rightly, management had considered schedules and deadlines more important than caution, over the objections of engineers.

    “It is impossible to fully test any computer system and there is no such thing as a body of code without bugs.” True, and it’s impossible to test any human system, and there’s no such thing as an organization of humans without flaws.

    I think we’ve probably passed the level of complexity in some of our largest systems that can be effectively managed by any group of humans. The news about the push to automate all medical systems definitely worries me, especially since it probably won’t be done with enough attention to backup systems and ways to handle the things that *will* go wrong.

    Fortunately, our software system development organizations aren’t as badly afflicted with unwarranted optimism as our economic geniuses — yet.

    Here’s a book recommendation, about civil engineering rather than software, but quite relevant: “Design Paradigms: Case Histories of Error and Judgment in Engineering”, by Henry Petroski.

  2. But in the last few years, IPTV has dwarfed the
    quality of DVD and Cable TV. Instant availability is precisely what is triggering ebook piracy too.
    The most enjoyable part of Revenge of the Fallen is
    its sheer familiarity.

Leave a Reply

Your email address will not be published.