Category Archives: posts

Full text posts

Hidden Science, Real News

Publishing in science used to mean submitting your work to a scientific journal, a process involving rigorous scientific review (usually). Lately some people haven’t bothered to wait for that process to run its course, and just put out a press release. That’s fine unless the press and policy crowds start treating the results as the truth.

An example from the eclectic and highly skeptical statistician, Michael Briggs:

So the WHO, the UN’s medical bureaucracy, trumpeted a press release claiming processed meats cause cancer. “News” organizations dutifully repeated what they were told and said a paper with the proof of these extraordinary claims was in the Lancet. There was no paper in that journal. Instead, there was an abstract which repeated the assertion and claimed that proof was on its way from the International Agency for Research on Cancer. And did that august group have the evidence? No, sir, they did not. Instead, they have a promise that it will be forthcoming. Any day now. Probably.

This is brilliant, doom fans. Tell the world it is in need of regulation and assure it that Top Men have called upon Science and Science is on their side. What effect does this have on honest, suspicious critics? Can the critics call “BS!”? No, sir, they cannot. Because why? Because the critics cannot examine the evidence. The critics must wait, else they will sound petulant. By the time the evidence surfaces, and even if it is entirely typical, which is to say flawed, overblown, and misleading, because it is based on classical statistical analyses time will have passed and few will care to listen to what the critics have to say.

Program Idea = Great; Name = ?

Last week a long list of distinguished technology, business and government leaders published a letter urging support for a new program in NYC called “Computer Science For All” (#CS4All).

This program, and programs like it, are critically important to our national competitiveness, and the future of today and tomorrow’s students. Can you imagine a successful, future career path that can be navigated without a basic level of computational knowledge? It gets harder and harder to do imagine day. On the other end of the spectrum, big, life-changing opportunities open up daily for those that have truly mastered some aspect or application of computers.

Since this announcement I’ve caught myself thinking about the program on a daily basis. While the program’s leadership wishes I was thinking about how to help, I got stuck on a more mundane issue:

Is “computer science” the right phrase for this program?

At this point you might be thinking “Dave, don’t be a shmuck, it’s just a name, so leave it alone”, but I think there are two important reasons to have a serious discussion about this question.

What do we want students to learn?

I suspect I’m not alone when I say that the phrase “computer science” evokes images of things like computer language design, compiler optimizations, algorithm efficiency, artificial intelligence and cache-coherent multiprocessor architecture. And I can say with certainty that these aren’t the things that #CS4All has in mind for kids.

One explanation is that the meaning of the phrase has evolved, and it just means “anything to do with computers”. That would be fine with me, but I’m not sure that’s really happened. Or maybe its a projection of a serious science down to an age-appropriate scale, such Physical Science in elementary school, which would also be reasonable.

But neither of these explanations get to the point of what we’d like these students to learn. Do we want them to be comfortable with the guts of computers, or just confident computer users? Is it OK if they never learn a formal programming language before college? Do we have a preference for whether they control a robot, create a dynamic presentation, graph a science dataset or play with the color spectrum of a digital photo? If a student is a wizard with iOS but can’t find their way around Windows a success, or a failure?

The answers to these questions are not at all obvious, but it is also not clear who is in a good position to answer them. Given that we’re looking to prepare students for the use of computation throughout our economy and daily lives, the two obvious groups, educators and computer scientists, don’t seem to have any more or less insight than any number of other groups.

Summary: We can call this whatever we want, but I’m not sure we know what we want students to learn.

How should we market to students?

While the last question was poked at some deeper issues, this one is more straightforward: the fraction of people who are motivated by the idea of learning “computer science” is miniscule. Worse, the only people who would find the concept appealing are the types who’d disappear into their room and teach themselves, so we don’t need to market to them anyway. (Note: “computer literacy” may be even worse, so don’t go there).

The saving grace is that, in my experience, kids naturally want to interact with computer programs and the Internet, that just makes the lack of an appealing name even more lame.

Here’s my modest proposal if we can’t come up with something better: “Computing For All”. But I’m not good at this stuff, and I hope someone can do better.

USA Hockey’s Girl Problem

This month’s USA Hockey Magazine celebrated the national tournament winners at each age group. Unfortunately, the print edition went out with the boys teams pictured as the victors for both the boys’ and girls’ tournaments. While this was surely an unintentional oversight by some staffer, it unfortunately is symbolic of an ongoing problem in USA Hockey’s approach to girls in hockey.

Before going into the details, let me say up front that I’m a big fan of USA Hockey. I’ve coached youth teams of boys, girls, and mixed for last 13 years or so, and have benefited extremely from the organization and training that USA Hockey provides for (and rightfully demands of) their coaches. Overall its been an extremely rewarding experience, and like many youth coaches I can only hope that the players have gotten as much out of it as I have.

Furthermore, USA Hockey’s progress in building female participation in hockey is fantastic. From their own report, USA Hockey now has more than 65,000 registered female players, representing over 10% of all players, and in states with strong programs, such as Massachusetts and Minnesota, over 20% of registered players are female. The success of the USA Women’s Olympic teams is not a fluke – there is great participation from teenagers to young girls that is feeding the program.

The problem is that USA Hockey can’t get over the hump and give the girls equal status with the boys. Take the national championships, for instance. Go to the web page and click on “2015 Nationals”, and you’ll see categories like “Youth Tier II”, followed later by “Girls Tier II”. You’ll also see a “High School Division”, and clicking through “Girls Tier I” you’ll find “Tier I Girls 19U”. The message is clear: there’s Hockey, and then there’s Girl’s Hockey.

I suspect this was left over from a time when the only option for girls was to play on boys teams, and that still happens sometimes. But take part in USA Hockey coach’s training and you’ll see this problem goes deeper:

  • since I was coaching a girl’s team this year I needed to complete the standard series of coaches training videos for my age group, and then also complete a special module for girl’s coaches (there’s Hockey, and then there’s Girl’s Hockey).
  • one standard module started out “U14 is an important age, since it is when most players will first experience body checking”. Boy, I sure hope not, since its not allowed in girls hockey.
  • at my in-person training last summer, a USA Hockey video meant to inspire players and coaches to become officials featured exactly zero women, and every reference to a referee was “he”.

Hockey is a great sport, whether it played by boys, girls or a mix of both. USA Hockey has done a great job of fostering the growth of participation among girls, but needs to make the final step and give its girls and women a first-class status.

Great Job Opportunities


There’s a lot going on at Applied Invention, and I wanted to highlight some specific talent that we’re currently in the market for. If you have any interest or questions please email me at ‘dd’ at this domain.

The first two positions are with our partner, Dark Sky (web-version is currently here). Adam, one of the Dark Sky co-founders, wrote an awesome blog post when we cemented our partnership a couple of months ago. Adam and Jay are great to work with, and if anything the potential of our partnership feels even bigger than the day we inked the deal.

We’re looking to fill two Dark Sky positions in the broader Boston area: a mobile app developer with a near-term focus on Android, and a versatile back-end developer who can also help out with operations. More info is available at the Dark Sky jobs page. This is a great chance to join the company that is revolutionizing how weather predictions are created and communicated.

As part of another partnership we are working with software entrepreneur Adam Omansky, who has a proven track record in the construction industry. While the details of Adam’s new venture are still under wraps, he’s looking for a lead engineer/architect for the new endeavor. The offering will have a core of big data and predictive analytics, with the usual browser and mobile front ends. This position will also be in Boston, and its a great chance to get in on the ground floor of a new startup.

Finally, we’ve got needs over the upcoming months within Applied Invention itself. If you’ve got experience with complex web applications, mobile apps, big data or predictive analytics, please reach out and we can discuss specific opportunities.

Is a 2014 Global Temperature Record Significant?


The media have gotten the message out big time: the 2014 global temperature was a record high (here’s the Huffington Post version and AP articles, but you can find the story everywhere).

There have been three valid criticisms of the record temperature claim:

  1. it’s widely agreed that the the earth has been warmer in the past, so this is only a record during the period of recorded temperature, which starts at 1880 (in fact some are saying that 2014 was in the coldest 3% of the past 10,000 years).

  2. among different organizations that publish temperature series, only two of them, NOAA and NASA, show that 2014 was the highest among recent years (Mike Smith).

  3. NOAA has a low confidence level in their own claim that 2014 temperature was a record (Daily Mail article, WUWT)

I’d like to put these aside and focus on a higher level question: Even if 2014 was a record, is that significant? Specifically, are these rare events, or are they commonplace?

IPCC 1 Chart

Marcotte Chart

The estimated temperature of the last thousand years provides important context for understanding record highs. The two charts above show the original IPCC 1 chart and one of the more recent versions which downplay the Midieval Warming Period (example here). In either case the current warming trend that started with the Little Ice Age is obvious.

During this period, extending well before the start of the NASA and NOAA temperature record in 1880, the temperature trend has been steadily up, though individual years and decades were sometimes flat or slightly down. As the chart below from NASA shows, since 1880 there have been four distinct roughly 30 year periods:

  1. slightly down (1880-1910)
  2. steady increases (1910-1940)
  3. flat (1940-1970)
  4. steady increases (1970-2000)

Finally, the last 15+ years have been generally flat, having plateaued near the high established in 1997.

NASA GISS Temperatures

Think about it: if you start measuring something and it generally goes up, you’re going to frequently get record highs. And that’s exactly what we have in the NASA and NOAA temperature histories, which each have 18 records in their 134 year histories, a record every 7 years or so on average. The charts below show the number of records per decade starting at 1880 and working forward (note: I did the same experiment starting in 2014 and working backward in case there was something about the start date that inadvertently skewed the accumulated data, but it looked the same).

Record Highs Per Decade

As you can easily see, high temperature records are commonplace since 1880, with many prior to the period of CO2 increases (that’s not to say that CO2 isn’t helping increase the temperature, just that records can happen regularly without elevated levels). In addition, the frequency of recent records doesn’t appear to be unique.

My conclusion is that the recent claims of high temperature add nothing to the data behind the current temperature run-up, which extends back into the 1700’s. If there was a record high in 2014, the fact that it happened is totally consistent with other record highs in the NOAA and NASA’s data.

Note: here’s a spreadsheet with the NOAA and NASA global land-ocean data in one place, and the records calculated. Note that a second tab has the land-only data.

Never, Ever Change Your Apple Password.

I admit it, I’ve had the same password on my Apple account for a few years. “Pretty stupid”, you might say, and I’d agree, but I’ll do it again given the hassle involved with changing it.

Last night I made a change to my Apple account, and to make the change Apple forced me to pick a more secure password (yeah, it wasn’t that great of a password, either). So I choose something longer, with caps, numbers, special characters, etc. Then the fun begins.

Mac: “Your iCloud password for your Apple account is incorrect”. No problem, I’ll just type that in.

Mac: “Your FaceTime password for your Apple account is incorrect”. Annoying, but I can type fast so a minor inconvenience.

Mac: “Loser, your App Store password for your Apple account is incorrect”. Come on, these are all stored separately?

Mac: “Hey bonehead, type in your Apple account password twice if you want iTunes to work.”

So I’m a little annoyed, but get over it until I pick up my iPad.

iPad: “I believe that you have stolen Dave Douglas’s iPad. Please type his password multiple times to return it to full functionality.” A little more annoyed, until I realize that I have to type this complex password into a touch keyboard. Of course I have to do this multiple times, also.

iPhone: “I see from our records that you were able to type a complex string into an iPad keyboard multiple times. Now try it on a phone keyboard, sucker.” This sucks.

Finally, I think I’m all back on-line, and I turn on my TV.

AppleTV: “Congratulations, you have advanced to the final challenge. You must now attempt to type your new password in using a child-like TV remote. Hahahahaha!”

Two conclusions:

  1. Apple is still not serious about security. Making it painful for people to change their password will have the expected result.

  2. Apple is going to fast. We’ve all seen the increase in buggy software (e.g. text messages barely work on my phone right now), and this is the kind of thing that they used to identify early and come up with a clever solution for. An opportunity for multi-factor auth, maybe? Losers.

Yesterday Was Cool…Until We Tell You Otherwise


Meterologist Mike Smith’s recent blog post provides a good reminder that the concept of global temperature is far more complex than a casual observer might suspect.

If you think about it for awhile, you’d probably come up with two of the bigger problems. The first one is the poor distribution of reliable weather stations. Coverage is decent in the economically advanced countries, but goes downhill from there. And as you’d expect, the poles and oceans have incredibly limited coverage.

Of course this leads to a question of how to create a coherent global average, especially given the fact that stations come and go over time. How much weight do you give a station whose nearest neighbor is hundreds or thousands of miles away, as opposed to one with many neighbors tens of miles away? What kind of algorithm makes the most sense to fill in the big empty areas? Some of these problems are being addressed by satellites, but these have blank spots as well (this post by Dr. Roy Spencer gives a great sense of the complexity of satellite temperature sensing).

The second problem is the big changes in technology. We’re increasingly reliant on satellites and other modern technologies for measuring temperature, but these are (obviously) recent inventions, so leaves the problem of getting comparable temperatures to compare against in the past. This becomes especially tricky when thermometers of the past are far less accurate than the temperature trends we’re trying to detect. (This problem really hasn’t ever gone away. To get a feel for the issue take a look at Anthony Watts’, the greatest citizen science project of all time, imo).

Going way back in time we don’t even have the benefit of human temperature records, so are left with proxies, i.e. natural indications of temperature over time such as tree rings or ice cores.

Add all of this up, and you find that you need tons of complex math when calculating global temperature records. Of course anything that complex can be done in more than one way, and different approaches means different results. When you’re trying to figure out if this year is a fraction of a degree warmer or cooler than some other year, small changes in algorithm can give you very different answers.

As Mike Smith indicates, concern surfaces when the same people trying to predict future temperatures are also involved in computing how global temperature is determined, and when the temperature adjustments are in a direction that helps past predictions look better. Personally I can’t judge whether there’s any validity to these concerns, but as a user of weather data I share Mike’s annoyance whenever a temperature record is retroactively recomputed.

An 18-Year Climate Anniversary


Today, October 1, 2014, marks 18 years without a measurable increase in the earth’s temperature according to the RSS temperature series (note: other series have different durations since warming stopped, some longer, some shorter).


This is not a reason to declare victory, as the side effects of our energy usage, including CO2 dumped in the atmosphere, add uncertainty to our future. But that doesn’t mean that this isn’t good news, as it certainly is. We have been given a reprieve from rising temperatures, and there’s an increasing chance that earlier estimates about the climate sensitivity as a result of atmospheric CO2 isn’t as bad as we thought.

Though the lack of warming is slowly getting more visiblity, there’s still a huge disconnect given the catastrophic narrative that many people continue to use when discussing the global warming. If rising temperatures are so horrible, aren’t these folks rejoicing that they’ve stopped for now? No they aren’t, and the reasons are varied. For some its a reputation issue, for some there’s big money involved in their campaign of fear, and for others global warming was just a lever to push for revolution in the economy, the government, or society overall, and a little pesky data won’t disrupt the crusade.

My advice is simple: be thankful that we’ve been given a break, take some time to look at real information about what’s going on and the options we have going forward, and tune out the fear mongers for awhile.

Mac Hacking: Sending Email From the Command Line


In moving an automated process from a linux box over to a mac, I found myself missing cron’s built-in ability to send an email as I switched over to OSX’s launchd. The obvious answer, sending an email from within my bash script, send me looking for an easy way to send an email from the command line, without needing to install sendmail or postfix. Ideally I’d use Gmail as an SMTP service, but not a hard requirement.

A number of articles cited msmtp as a good solution, and this article provided a great overview of how to install it. Furthermore, it supports Gmail and can use the OSX keyring for account info, which are nice. However, in the 5 years since it was written, enough things had changed that I couldn’t follow it directly. Specifically:

  1. I ended up using Macports, since I was missing pkg-config (and probably other stuff) that would have let me build msmtp from source.

  2. Thawte made some cert changes in 2010, and the described config doesn’t work as written.

So here was my process:

  1. Installed msmtp via Macports, including the openssl variant (‘port install msmtp +openssl‘).

  2. Installed the curl-ca-bundle from Macports to replace Thawte.

  3. Created an msmtprc file as described in the earlier instructions, and store at /opt/local/msmtprc. The only change is the location of the cert file, which should now be:

    tls_trust_file /opt/local/share/curl/curl-ca-bundle.crt

  4. Manually added the account to the OSX keyring and added a .mailrc as described in the earlier instructions.

Works great!

United Airlines Crosses a Line


I usually don’t bother to write about airlines, since its not news to anyone that crappy experiences happen all of the time. But things crossed the line on United Airlines flight 6314 from Des Moines to O’Hare on July 23rd.

Part way through boarding, a young man with some clear physical challenges sat down in the bulkhead seat. He had a smile on his face, but couldn’t have weighed more than 100 pounds, had his arm was in a sling, and one of is legs was immobilized by a brace. The passport in his hand was foreign, and I noticed a small American flag poking out of his backpack.

Being the bulkhead, the flight attendant came over and told him that he needed to put his backpack in the overhead for takeoff. His English clearly failed him, and he looked confused. She follows with “That can’t be there” in a loud, stern voice. He understands that, so picks up the pack as if to hand it to her. Stern and loud again: “No. You need to get up and put that in the overhead bin yourself.”

At that point a couple of fellow passengers jumped up and took care of it for him, since it was clearly going to be a physical challenge. I said “geez, that’s some customer service”. The attendant turned to me and angrily said “I would have expected some more courtesy out of the other passengers to begin with”. At that point she did her safety talk, went on the intercom to declare that the flight was too short for any service, and she sat down and read magazine for the rest of the flight.

Upon landing a couple of us got up and collected the fellow’s items, and helped him out to his wheel chair on the jetway. Not a lifted finger or word from the attendant the whole time. We got a friendly “thank you” from the young man as he was wheeled off.

In the terminal I sought out a United service rep and explained that I felt our flight attendant had been abusive to a disabled passenger. Without looking up from her computer she said “That’s too bad, please go to our website if you’d like to report it.” At home I did that, asking to speak with someone. A few days later I got an email saying that if I had something to say, I needed to write it up on the website and they’d get back to within 7 to 14 days.

So I tried to talk to United, but since they insist I type everything up, I’ll just put it on my blog.

Over the year’s I’ve met some great United employees, and I’ve had bad experiences with specific employees at other airlines. But United has, by far, the highest fraction of employees who just don’t give a shit, and make no effort to hide the fact that they consider customers a major nuisance.

In an ideal world I could just stop flying United, but there’s a few trips I do where there aren’t any practical alternatives. So I work hard to fly someone else whenever I can, and use them as a last choice.