Category Archives: posts

Full text posts

The Internet-Enabled Eclipse

We saw the 2017 eclipse in Oregon. Here’s some quick thoughts, and why I think this one was a singular event.

  • I feel lucky on two accounts: 1) having the opportunity to be somewhere to see it with my family, and 2) having good weather. It would be easy to miss it for either of these reasons.
  • The eclipse was very cool, but totality was a whole different level of cool. (There have been lots of good write-ups, but I thought Jason Snell captured it well). The combination of the corona, being able to look at the sun directly (and also see Venus!), and the level of darkness were all a bigger deal then I would have guessed. If you have the opportunity to get to totality in 2024 then definitely do it!
  • The goofy eclipse glasses were totally worth it.
  • The change in temperature was noticeable where we were. Cliff Mass summarized the weather impact in the northwest.. Interesting that some of the stuff was a surprise…..
  • Cliff also mentioned the traffic. In general there were some delays, but quick to get through. HOWEVER, the state of Washington once again proved that they have the doubler whammy of bad roads and bad drivers. Serious traffic jams on I-5 in the middle of nowhere,10 hours after the eclipse and 100 miles north of totality.

This eclipse was “marketed” as the “Great American Eclipse”, but I think what really made it unique was that it was the first total US eclipse of the Internet age. Everything from websites to social media to camera phones to e-commerce played a role in this eclipse. Location selection, travel plans, weather, traffic and sharing – all of these involved the Internet. In short, the 2017 eclipse wouldn’t have been itself prior to the Internet.

Eclipse today: As we were driving away from totality my teenage daughter said “cool, one of my friends saw the eclipse from a plane and just posted the video!” (I don’t have a link to it, but here’s a similar one).

1970’s eclipses: it was interesting for me to go back through the historical list of US total eclipses. I remember being outside my grade school in Wisconsin with the paper-and-pinhole setup viewing an eclipse. From the timeline, I suspect it was in ’70 or ’72. The February of ’79 eclipse happened while I was in high school, and I can attest to the fact that it was a non-event (I actually don’t remember it at all).

In the 1970’s, how would you have known where to travel to? What time the eclipse would peak at that location? Would this info have come from a school? 6 o’clock news? Newspaper? How would you even have known to get excited about it? How would you have gotten eclipse glasses, let alone known about them? How would you have shared your experience? Would you have visited the AAA store to get a Trip-Tik so you had some maps?

As things increasingly happen at “Internet speed”, it gets harder and harder to remember what things were like before it existed. The 2017 eclipse provides a one-time opportunity to contrast the the world before and after the Internet.

Remembering Jim Canty

greatday


“… a first ballot Hall of Famer on the ice and in life.”

The quote above was from a mutual friend in reference to Jim Canty, who passed away in late April. If you’re racking your brain for memories of Jim’s professional career (“Wasn’t he on the ’84 Nordiques?”) you can stop now. Jim’s illustrious 22-year career was with Hippy Hockey, the Sunday night skate at the local rink.

It would only be partially accurate to characterize Hippy Hockey as a bunch of old guys reliving their glory days on the ice — many of us never had glory days to relive. Jim loved hockey. Like myself Jim came to Hippy Hockey via pond hockey, and enjoyed the magic of skating, the friendly competition and a beer or two in the parking lot afterward. In the summer Jim would bring fresh clams from the Cape and cook them up.

Jim and I initially connected through our kids. My two older kids were similar ages to the middle two of Jim’s four kids, and intersected in everything from play dates to confirmation to hockey games. We coached hockey together one year when the boys were young.

Jim was, at his inner core, a family man. I’d seen Jim in his husband/father role, but his family says it started early on with his parents and 8 brothers and sisters. When Jim and I saw each other we always caught up on how each other’s kids were doing. When some people talk about their kids and their accomplishments it comes across as bragging, partly about the kids, but often more about how great of a parent they are. With Jim it was different. He spoke with a sense of selfless wonderment and joy. I wish I could describe it better, but if you’d had the chance to talk to Jim you’d get it.

Professionally Jim was an lawyer and investor, so there was little overlap with my tech world. We did, however, interact about some of the energy investments he was looking at. I’d try to help out with some technical perspective, or tap into some of the science talent at work to analyze some startup’s claims. Over time I came to realize that Jim and I shared a similar optimism about human potential. Sure there’s lots of problems in the world, but there’s also a lot of smart, hard-working people, and humanity has the potential to overcome its challenges.

I wasn’t the only one who noticed this about Jim. At his memorial service a Franciscan monk made a connection between Jim’s optimism and the optimism at the core of Franciscan values. Jim had strong connections to the Franciscans, a Catholic religious order who follow the teachings of St. Francis of Assisi, with roots back to Jim’s college days at St. Bonaventure.

I’ll close with a statement from the Siena College mission/vision page, since it describes so well for me how I saw Jim live his life:

In our Franciscan community, optimism is a faith-filled affirmation of the basic goodness of life and of all men and women because, in the words of St. Francis, God our Creator is “good, all good, supremely good.” So:

  • be positive

  • be hopeful

  • be open to the future.

May God bless Jim and his family.

Remembering Ken Traub

Just over a week ago Ken Traub, my friend and colleague, suddenly passed away. Lots of people have shared their memories of Ken, and have captured what a wonderful person he was, his intellect, and his breadth of interests and skills. To say that Ken was exceptional was an understatement, but I wanted to highlight three specific aspects of Ken that stand out for me.

The first was Ken’s ability to organize complex very systems. To the average person this might not sound that “sexy”, and they’d be mostly right, but the foundations of our digital world depend on a small number of people with this unique ability.

An example is the system of numbers that show up in bar codes and RFID tags and are a foundation for our entire system of commerce. Over the years Ken made major contributions to these standards and technology. Creating a system like this that can work reliability on a global scale requires the synthesis of information from a wide range of topics, including physics (the ability to read a code accurately), to computer science, to business standards and processes. Knowledge of all of these need to be synthesized into a coherent design, and then combined with the ability to write down the design in a clear and complete matter.

This last point is particularly important. A key artifact of these projects are lengthy, detailed, precise documents that specify everything about the system. Most of us (and I definitely include myself) will get the basics right, then lose momentum when it comes time to work through all of the edge cases and exceptions that occur in the real world.

Successful projects in this space produce systems, like the bar code and RFID systems, that are so good, so reliable, that you forget they exist. Ken was among the world’s elite at creating and defining systems like this. (Side note: the other person who was amazing at this that I’ve had the privilege of working with was Guy Steele, who also happens to live in the same town as Ken’s family).

The second thing I’ll always remember about Ken was his intellectual honesty. When you design things in groups, lots of interesting dynamics appear. Individuals might get attached to their own idea and defend it even after it has been shown to be flawed. Or engineers defend idea A because it links to idea B, which they are really excited about. Or someone dismisses key information because it comes from someone they don’t consider to be as smart as they are. Or you’ll get competitive types who view the design process as a contest where there are winners and losers depending on who’s ideas get built.

These behaviors are so common, so widespread, that over time you listen to people with the assumption that there are hidden agendas at work. Quickly I learned this was not the case with Ken. Whether we agreed or disagreed, he was always driven towards an underlying beauty or truth, the belief that there is a “right” answer to any design challenge, and a willingness to incorporate new ideas or information that might help lead to that result. It was fanstastically refreshing to design things with Ken – you just knew you were going to end up with a better design, and the journey was going to be as rewarding as the destination.

Finally, my lasting image of Ken is mentoring a junior engineer. The act of mentoring embodies so much about Ken’s character: his modesty, his approachability, his teaching skills and his amazing thought process. Ken didn’t just help other engineers solve problems – he taught them to be better engineers.

And that was Ken in a nutshell. You thought you were sitting down to improve a design, but you got improved in the process.

God bless Ken, and give his family strength during this challenging time.

More on United…

Following up on the post from my good friend Snowman on Fire, I wanted to add a few notes. Like Snowman I have been a customer of United and Continental for decades, with over 2M miles. Long ago I came to a healthy point in my relationship with them – I understand when I take them I’m risking random crap like this happening, so except for extreme cases, I don’t let it bother me. I only fly them when they’ve made it well worth my while through a much better fare, travel times, upgrades, etc. All airlines have their warts and good sides, but its not a fluke that this happened on a United flight instead of someone else.

  • Snowman is on the money with his points: United was within their legal rights, but the situation was self-inflicted (caused by their own operational ineptitude), and of course they handled it all wrong.

  • Companies can develop personalities, and United is somewhat schizophrenic. They can have good days, but they also have really bad days where they are downright mean spirited. I believe them when they say they didn’t know what the security people would do to the gentleman, but I have no doubt that many of them were hoping that is exactly what would happen.

  • Much of the bad side of their personality comes from the pre-merger United organization. If you have a good flight crew on United and ask them which company they came from, 8 out of 10 times they’ll be from Continental.

  • The fact that they kicked a passenger off for an employee is totally within their culture, which is “employees come first”. United flight crews tend to cut in TSA lines without a “sorry” or “thank you” more often than the others, get their bags on board first before the overheads start to fill up (look for them next time you board), and heaven help you if you try to get some service while they are figuring out their schedule for next month.

  • I had something this bad happen on a United flight a good while back, before cell phones could capture and spread it. Tried to get United’s attention for two weeks after, but no one cared (and I know I was not alone in calling people there up – the main conversation on the flight was “did we really just see that?”).

Finally, I want to add one more point to Snowman’s observation that this was self-inflicted, because United had created the situation. Not only did they create the operational situation, but if the passenger was a regular United customer, they were in no mood to do any favors for the airline.

AI, Society and Citizen Engineers

Two recent Techcrunch articles highlight some of the challenges we’re in for with the increased use of machine learning (ML) and artificial intelligence (AI), to the extent these are separate. In [AI’s open source model is closed, inadequate, and outdated], Kumar Srivastana argues that we need a new kind of transparency (I’m avoiding his use of “open source” – more on that below) because of the complexity and unpredictability of these systems. Along similar lines, Jeremy Elman and Abel Castilla argue that we need to rethink liability and quality standards in [Artificial intelligence and the law].

These articles cover some interesting ground, but are also enlightening in the misconceptions that they reflect. Some specific points:

  1. “Open source” can technically mean one of two things: 1) an organization’s decision to make code or other intellectual property visible to others, and 2) a set of licenses organizations that allow to control how others can use material they choose to make visible. I agree with Kumar that AI and ML introduce some new elements that organizations can choose to make visible, but it’s not clear that the decision process of organizations or the licenses we use are the issue.

  2. Even without AI or ML we have many systems that people depend on every day where we have no idea how they work. Furthermore, these systems are complex enough that no one truly understands how they will react in different situations (e.g. cascading failures in our electric grid), and we have no visibility into how these systems are being tested, etc. AI and ML may further complicate this problem, but to portray the situation as brand new and unlike what came before it is not accurate.

  3. Both articles discuss AI’s as somehow disembodied beings. These are algorithms embodied in products and services, where they are generally replacing large, complex pieces of human-written software. Maybe I’m missing something, but I don’t understand how the replacement of one algorithm with another changes the liability of the companies for their products and services. If my product uses software and it fails in a damaging way, AFAIK the liability should be the same independent of how that software was created.

For me the bottom line is this: AI and ML are subtle and potentially powerful tools. As with any technology, it is the responsibility of product and service companies, and, importantly, their engineers, to understand these tools and use them in a responsible way. And since these are rapidly evolving technologies, it is incumbent on engineers who use them invest in the time to stay current and keep their systems up to date with the latest methods for testing and validating the algorithms they produce.

Elman and Castilla provide the excellent example of a traffic light that is run by an AI, and the AI decides that the most efficient mode requires the lights to change faster than normal, resulting in more accidents. The authors cite this example as an example of why we need the law to adapt. I disagree – this is a clearcut example of engineering culpability. Just because you use AI or ML for part of an algorithm doesn’t exclude you from putting in some good old-fashioned logic checks in addition. Think about it: if a human were controlling the light, wouldn’t we want some logic to make sure they stay within some safety parameters?

Artificial intelligence and machine learning are important new technologies, and they bring some interesting twists. But this is the next in a very long history of technologies, going back to fire if not earlier, where it is the responsibility of our engineers to translate them into safe uses. I know that it’s scary that we can’t look at a piece of code and see exactly what it’s supposed to do, but its foolhardy to believe that with today’s complex systems anyone fully understands what they do and can vouch for them anyway.
As far as I can tell our existing legal and transparency frameworks and practices haven’t been shown to be outdated by these new technologies, so let’s see how their use evolves and react when our systems break down.

Sustainability and Values, Birds and Wind

I have always maintained that every action justified by sustainability has side effects, and that it becomes a values-based judgement as to whether the value of the sustainability improvement outweighs negative impacts of the side effects. Being a values-based judgement means that not everyone will view the cost-benefit tradeoff the same way. Further complicating matters is the fact that the tradeoffs are apples versus oranges, as in the case of compact florescent light bulbs, that have much better energy efficiency than our historical incandescent bulbs, but also have embedded mercury that poses other threats if not handled appropriately.

One such cost-benefit tradeoff that deserves far more attention is the threat of wind power to birds and bats. The fact that giant wind turbines kill birds isn’t a surprise, but we (the US public) don’t know how many. The problem is that the data of has been kept secret, though its value is questionable since collecting the data has not been a requirement for wind energy providers. However, we can get a hint from US Government decisions, and what is revealed is not pretty.

On the 14th of December the Department of the Interior (DOI) finalized a rule that absolves wind generation companies from penalties and prosecution for killing protected birds for the next 30 years (news coverage by AP here), overriding the Bald and Golden Eagle Protection Act that us normal Americans are governed by. The rule allows up to 4,200 deaths per year of bald eagles, or roughly 3% of the total population. It also waives penalties for killing the rarer golden eagles.

While the number of eagles killed each year is kept secret by the DOI, we have some insight into the number, since this DOI rule raised the cap on bald eagles by nearly 4x, indicating that the wind turbine owners were nervous that they were at or above the current limits.

Personally I’m appalled by the death of these beautiful birds. However, more appalling are the lack of good, public data on bird deaths by wind farms, and the lack of an open discussion about the relative value of wind power and birds. How many dead bald eagles per year are an “acceptable” side effect of our growing wind power? How many golden eagles? The DOI has decided that this is a discussion they are not going to allow to happen.

Why We Want Biased News

All of the recent discussion of news, be it fake, unfair, slanted, etc, has gotten me thinking about the mechanics of news. I’ll define news as things or events that, as far as anyone can tell, happened or didn’t happen, and reported newsas how news is communicated, including websites, papers, newscasts on TV, radio, podcasts, blogs, tweets, etc. So any description of something that happened or didn’t happen, then I’m calling it reported news.

First, I assert that all reported news is filtered, meaning that it’s not the whole story. This is true at every level: any news show or website or paper cannot describe everything that happened that day, and for any individual story, be it a political speech or a car accident or a good deed, it’s impossible to provide all of the detail and background. In every case what’s reported is a small subset of what is being reported on, so is filtered.

Second, I assert that all useful filters are biased, i.e. they favor some things over others. There’s almost always a spatial (favor local news) and temporal (favor recent news); we’re so used to these we don’t even notice them. The bias can also favor one group over the other in sports, religion, race, politics, sexuality, age, eduction, etc, etc, etc. And the bias can be in attitude, favoring happy, sad, good, bad, etc.

Even though news has always been biased, our awareness of the bias seems to be going up. This would make sense – we have more news sources, many of which are targeting specific audiences, and aren’t even claiming to be unbiased. As a result, it is easier to see and compare the choices that specific news outlets are making.

The New York Times has had the motto “All the news that’s fit to print” since 1897, but in 2016 no one treats that motto as an absolute statement, i.e. that the New York Time has some magic filter that isn’t biased. Seeing that motto today people naturally read “All the news that’s fit to print, as determined by some folks at the New York Times”.

Techies may tell you that algorithms can be unbiased, but that’s generally not true. The hand-coded algorithm will embody the bias of its developer, and the machine learning algorithm will learn the biases of its trainers. Beyond that, there’s the question of what inputs the algorithm has – does it have access to all of the details of everything that’s going on, or just a subset? So sensing system of our algorithm can be biased as well.

The clever coder might now ask: what if I had cameras everywhere, and created an algorithm that randomly creates a stream of news from all of the world’s events? My response is that it would be super boring: minor traffic accidents in far off countries, youth soccer scores for children we don’t know, misbehaving mayors of cities we’ve never heard of, etc . While our natural inclination is to think of bias as bad, the truth is that we want bias. We want an editor, human or algorithm, to find things that will interest us.

The key, then, is to embrace the bias, and acknowledge and understand it. The only time bias is really bad is when pretend that it isn’t there..


Self-Driving Cars: Advantage Pedestrians

In Boston, two warring factions have been partnered in an unlikely, decades-long truce, their aggressive tendencies controlled by an unspoken, yet universally understood set of rules of engagement. That truce may soon be shattered.

Boston pedestrians will brazenly cross busy streets in the middle of a block, but deep down they understand that getting hit by a car could really suck. Boston drivers are aggressive, using intimidation to navigate rotaries and keep pedestrians in line. But they also understand that hitting a pedestrian would be a major hassle.

The result of this mutual understanding might look horribly chaotic and dangerous to an outsider, but it generally works, making near-optimal use of the road for both groups. Of course a mini van with Ohio plates might get stuck for a few minutes by some commuters coming out of a subway stop. Or a foreign family may wait 3 or 4 light cycles before summoning the nerve to trust the “walk” signal, but that’s the price of stepping into the middle of a conflict without understanding the rules of engagement.

Now picture this scene from a not-too-distant future: a mutual fund manager wants to grab a Starbucks before digging into company valuations for the morning. Before stepping off the curb mid-block, she looks down the street to assess the state of oncoming traffic, and the only car with a real shot at her has no driver – a freshly programmed autonomous vehicle. Instantly she recognizes that lawyers in some glass office building have ensured that the AI in that car will do absolutely anything to avoid hitting her, so she confidently steps off the curb.

As more autonomous vehicles hit the streets, this becomes scene occurs more and more often. Self-driving cars take an hour to go two blocks on Boylston St during commute time. On Saturday afternoon on Newbury St, fancy electric cars are stalled until their batteries run out, blocking traffic for hours afterwards.

It’s not clear what the answer to this emerging problem is. Rehire all of the out-of-work cab drivers to hand out $500 jay walking tickets? Blanket the city with advanced facial recognition cameras that try to shame repeat offenders on popular social networks? Underground roads? Overhead walkways?

It’s possible that nothing will work until we’re willing to make the artificial intelligent cars just a little big crazy.

Hidden Science, Real News

Publishing in science used to mean submitting your work to a scientific journal, a process involving rigorous scientific review (usually). Lately some people haven’t bothered to wait for that process to run its course, and just put out a press release. That’s fine unless the press and policy crowds start treating the results as the truth.

An example from the eclectic and highly skeptical statistician, Michael Briggs:

So the WHO, the UN’s medical bureaucracy, trumpeted a press release claiming processed meats cause cancer. “News” organizations dutifully repeated what they were told and said a paper with the proof of these extraordinary claims was in the Lancet. There was no paper in that journal. Instead, there was an abstract which repeated the assertion and claimed that proof was on its way from the International Agency for Research on Cancer. And did that august group have the evidence? No, sir, they did not. Instead, they have a promise that it will be forthcoming. Any day now. Probably.

This is brilliant, doom fans. Tell the world it is in need of regulation and assure it that Top Men have called upon Science and Science is on their side. What effect does this have on honest, suspicious critics? Can the critics call “BS!”? No, sir, they cannot. Because why? Because the critics cannot examine the evidence. The critics must wait, else they will sound petulant. By the time the evidence surfaces, and even if it is entirely typical, which is to say flawed, overblown, and misleading, because it is based on classical statistical analyses time will have passed and few will care to listen to what the critics have to say.

Program Idea = Great; Name = ?

Last week a long list of distinguished technology, business and government leaders published a letter urging support for a new program in NYC called “Computer Science For All” (#CS4All).

This program, and programs like it, are critically important to our national competitiveness, and the future of today and tomorrow’s students. Can you imagine a successful, future career path that can be navigated without a basic level of computational knowledge? It gets harder and harder to do imagine day. On the other end of the spectrum, big, life-changing opportunities open up daily for those that have truly mastered some aspect or application of computers.

Since this announcement I’ve caught myself thinking about the program on a daily basis. While the program’s leadership wishes I was thinking about how to help, I got stuck on a more mundane issue:

Is “computer science” the right phrase for this program?

At this point you might be thinking “Dave, don’t be a shmuck, it’s just a name, so leave it alone”, but I think there are two important reasons to have a serious discussion about this question.

What do we want students to learn?

I suspect I’m not alone when I say that the phrase “computer science” evokes images of things like computer language design, compiler optimizations, algorithm efficiency, artificial intelligence and cache-coherent multiprocessor architecture. And I can say with certainty that these aren’t the things that #CS4All has in mind for kids.

One explanation is that the meaning of the phrase has evolved, and it just means “anything to do with computers”. That would be fine with me, but I’m not sure that’s really happened. Or maybe its a projection of a serious science down to an age-appropriate scale, such Physical Science in elementary school, which would also be reasonable.

But neither of these explanations get to the point of what we’d like these students to learn. Do we want them to be comfortable with the guts of computers, or just confident computer users? Is it OK if they never learn a formal programming language before college? Do we have a preference for whether they control a robot, create a dynamic presentation, graph a science dataset or play with the color spectrum of a digital photo? If a student is a wizard with iOS but can’t find their way around Windows a success, or a failure?

The answers to these questions are not at all obvious, but it is also not clear who is in a good position to answer them. Given that we’re looking to prepare students for the use of computation throughout our economy and daily lives, the two obvious groups, educators and computer scientists, don’t seem to have any more or less insight than any number of other groups.

Summary: We can call this whatever we want, but I’m not sure we know what we want students to learn.

How should we market to students?

While the last question was poked at some deeper issues, this one is more straightforward: the fraction of people who are motivated by the idea of learning “computer science” is miniscule. Worse, the only people who would find the concept appealing are the types who’d disappear into their room and teach themselves, so we don’t need to market to them anyway. (Note: “computer literacy” may be even worse, so don’t go there).

The saving grace is that, in my experience, kids naturally want to interact with computer programs and the Internet, that just makes the lack of an appealing name even more lame.

Here’s my modest proposal if we can’t come up with something better: “Computing For All”. But I’m not good at this stuff, and I hope someone can do better.