Tag Archives: innovation

US Data Investments to Support the Next Generation of Ag Innovation

While we all know about the US government’s big investments in agriculture research, their role as a big data and analytics provider is less well appreciated.

Seeing the phrase ‘big data and analytics’ we think of machine learning, artificial intelligence, cloud computing, and other popu;ar 21st century concepts, but the US government has been providing important data and analytics to researchers, farmers and their advisors, and ag tech companies for over a century. The US has been the innovation leader of global agriculture for most of this period, and weather and soil data, nutrient models, geospatial information and crop yield data have all beem important components.

Soils data is a great example, from formal soil surveys starting in 1899, to today’s USDA soils resources wealth of data and models. If you want to dive in, check out the amazing SSURGO dataset, which you can access (in a kind of clunky way) through the Web Soil Survey.

These data and analytics assets are an amazing resource and a huge, national competitive advantage. Importantly, they are all available free of charge to anyone, which is not the case in many other places around the world.

The US strategy for innovation-enabling ag data and analytics warrants a longer discussion, but let me kick it off with a list of 5 opportunities for the the USDA to have an even bigger impact:

  1. Farm Efficiency. An array of new practices have emerged since the turn of the century, including new tillage strategies and cover crops. On the surface these seem sensible, but do they achieve their goals? Is there a solid cost-benefit win, or are they a mixed bag? Helping to inform grower decisions with real data and analysis has been a key role of the USDA, and it is an ideal time for a deeper look at the impact of these emerging practices.

  2. Soils, Topography. The US soils data is great, and it is a critical tool for managing water usage and resources. Small changes in the sand content of soil or shifts in the shape of the terrain can make dramatic changes to how much irrigation is required in a given season. As we need to manage water more carefully in the future, any improvement in this data, especially in high irrigation regions, will be invaluable.

  3. Weather. Weather data is the single most important factor in nearly every aspect of crop modeling, including fertility, irrigation, yield, disease and pests. The US gets big points for making all weather data free for all uses, but there are opportunities to improve the spatial density of precipitation-related forecasts and readings, and to improve inputs and calculation of key derived values, such as evapotranspiration.

  4. Satellite Imagery. I predict we will look back on the Twenty-Teens as the start of the golden age of satellite data in ag. Fly-bys will increase from every 10+ days to every day (cloud cover permitting), and our ability to leverage what we’re seeing will increase radically with more data. Most of these new capabilities are private (namely PlanetLabs), which is , but there is an important role for the government to play in providing freely available data of increasing quality and scope.

  5. Genetic Data. When we think of agricultural genetics we naturally think of GMO corn, but that’s a tiny fraction of what will matter in the future. Other crops, the soil microbiome, and the genetics of disease and pests will all be impactful areas for research and applications. Historically the big ag companies have taken the lead in genetics, but between with their capture of intellectual property and international ownership, there is a strong argument that the USDA should secure the core genetics data to enable a broader base of research and private sector innovation.


Data is increasingly viewed as an asset, and that has been true in agriculture for decades. The US government deserves credit for recognizing this before most, and now its time to look forward, continue to invest, and modernize their approach to match today’s technology and priorities.

Innovation Lessons of the Internet

In the midst of a trend to credit the federal government with all of the great things that have been invented or built since WW II, it is important that we study the history and the lessons it has to offer. The WSJ has a useful op-ed about the start of the Internet that untangles some of innovation myths of the oft-cited endeavor.

Government Successes in Innovation

I give the US government massive credit for their role in fundamental research. With the decline of major corporate labs (e.g. Bell Labs, IBM Research, etc), the feds are left as the primary funder of basic research in the US, with positive effects that span the globe.

I also believe the government has done amazing work in what I call “mission-driven innovation”, where the government is driving for a specific outcome in order to fulfill a focussed mission that it has. I would count supercomputing in this category as well as many other defense, health and agriculture-driven innovations. In these cases the government is funding innovation, but is also a major customer of the innovation, helping to create an early market. [Note: while many people put the current clean energy push in this category, I don’t count the current manifestation as mission-driven. There is no articulated, focussed mission or strategy; “Let’s try everything” doesn’t count.]

The Early Internet

In addition to the notes in the WSJ article, it is important to understand the state of computer networking during the early days of the Internet in the 80’s. First, there were already widespread computer network operating, and early versions of network services, such as email, already existed. What was missing was agreement on a protocol that could span networks in a scalable way. This is what the Internet Protocol (IP) proved to be great at. Furthermore, the Internet Protocol was an open protocol (i.e. not proprietary and controlled by a commercial organizations), so implementations of the protocol were able to spread freely.

But the ascendance of IP to its current role in global networking was not at all clear at that time. However, it is easier in retrospect to look back and see these, and other features as critical to its success. The government deserves credit for funding the research that led to IP, and for supporting the open design that allowed it to spread so rapidly.

The Government’s Key Role in the Internet?

But I believe that the government’s key contribution to the Internet happened in the 90’s. While a couple of government agencies ran their own IP networks (e.g. ARPA, NSF), and the government continued to fund basic research in computer networks, the bulk of the design and buildout of the Internet that we know today were happening outside of the government.

By the middle of the decade, Netscape’s meteoric rise had finally awoken Bill Gates at Microsoft, who’s famous letter to the company served to alerts its vast array of businesses to the potential of the emerging network, and the threat to Microsoft if it didn’t embrace the technology. For me this letter marks the start of a 5-year period of broad awareness, where individuals, mainstream companies and governments gradually came to understand the Internet as an open platform for innovation, with its potential to transform nearly every aspect of our business and government. [I consider the end of this period is marked by the pets.com $82.5M IPO in February of 2000; at that point the broader market had clearly taken the potential of the Internet to heart.]

You may feel like you missed the role of government in the Internet in my description of this decade, and you’d be right. Aside from some mission-driven organizations operating their own IP networks (and helping to prime the pump for early IP-based networking gear), the formative years of the Internet as an unprecedented platform for innovation were largely free of government involvement. Yes key inventions like HTML involved government funding, but the government was yet to be consciously involved in the design, build out and operatin of the Internet.

To this day the Internet is governed by a independent non-profit organization, with the backbone network operated and managed by commercial companies. To this day you can register a domain name and put an innovative new service on the Internet, all within a period of hours, and all without a transaction of any kind with the government.

Of course it can only be a thought experiment, but one can’t help wondering what the Internet would be like today if the federal government had awoken to its full potential in 1990. How open would it be? How cheap would it be to use? How far and how fast would it have grown? What would our economy look like?

Another thought experiment would be to ask what the Internet look like if the government had set out with the explicit goal to build it. What if Reagan, Carter or Bush had decided to recreate the mission to the moon and build the mother of all networks by the year 2000? Is there any chance the result would look like the Internet we know today?


The US government is the primary driver of basic research today, and drives broader innovation through some of its mission-driven endeavors. It deserves a piece of the credit for the creation of the Internet.

But many are extrapolating from these facts and overestimating the role of government in innovation, and underestimating the innovation-driven role of the private sector in bringing basic science to market. Furthermore, many are overlooking the ways that government intervention can, and does, stunt the growth of valuable, new technologies.

The Internet presents a wonderful opportunity to study the role of government in innovation. Let’s make sure we’re open to all of the lessons it has to offer.

In Search of Energy Innovation Role Models

We have compelling reasons to drive for clean, cheap energy, but we lack the technology to get there today. Threats of climate change, national competitiveness and energy security (OK, “clean, cheap, domestic energy”) all contribute to the urgency of this innovation challenge. Given the scale of the challenge, coupled with the dire consequences of not succeeding, it is only natural that we’d look for reassurance and guidance from historical success stories of large-scale innovation.

Most frequently mentioned are the Apollo Project (“land a man on the moon by the end of the decade”), and the Manhattan Project (“develop nuclear weapons before our enemies”). They are attractive because they had urgent time tables, required outside-the-box innovation, and most importantly, as measured by their stated goals, were wildly successful. They provide some confidence that we (or maybe even just the President) need only to make the decision, and it will happen!

Many have also pointed out the flaws in these analogies. Characterizing them as self-contained projects that didn’t require deep changes to our national infrastructure, economy and behavior patterns, many have cautioned that we need to be careful viewing these as models for economy-wide energy transformation (for a good discussion check out the intro to “Technology Policy and Innovation” by Mowery, Nelson and Martin).

As an alternative, Prof. John Sterman at MIT has pointed to the Civil Rights movement as a better analogy, especially related to the climate change . While it offers some important lessons of the interplay between shifting public sentiment, leadership and government policy, it doesn’t offer us any guidance on how to address the underlying energy innovation challenge.

While we could start a process to catalog the problems of every historical precedent, its probably simpler to step back and observe that as a nation we’ve never before consciously undertaken an effort:

  • to overhaul something that touches every part of our economy
  • on a fixed schedule
  • lacking the required technology

So we’re left with looking for guidance from imperfect matches, which is OK as long as we understand that’s what we’re doing. And since we’re in uncharted and dangerous waters, I believe its still important to take any help we can get.

Tom Friedman has talked about needing a “Million Manhattan Projects”, capturing the need to invest in a many technical approaches in parallel. And this mental framework nicely complements the work done by many groups to analyze historical US innovation successes in agriculture, health and IT (again, see Mowery, et al for a good summary). In other words, lets not think of this as a single thread from R&D to production, but a number of parallel threads, each of which needs to be optimized.

Interestingly, this mindset brings us back to WW II, where the US government pursued a wide array of approaches to different innovation challenges. An amazing collaboration with the auto companies produced new airplane designs, which were produced at record rates. A public-private-university collaboration housed at MIT developed RADAR, fundamentally altering the effectiveness of German submarines. Partially backed by public funding, Goodyear created the first high-volume, low-cost synthetic rubber. And with the US military as a driving customer, Merck and other private pharmaceutical companies massively improved the effectiveness of penicillin, while ramping production over 100 times previous levels. What’s nice about these example is that they each use a different model of public-private partnership, and that they run the full lifecycle from R&D through large-scale production.

And, of course, we return to the Manhattan Project, which, viewed through this lens, is no longer an innovation strategy unto itself, but is a specific approach to one innovation challenge in a broad portfolio of approaches and challenges.

The suggests an interesting approach to our situation, where we have a large number of separate technologies, each with their own innovation lifecycle. Instead of looking for a historical analogy to the whole problem, we can take each technology and look for historical guidance on how to move it forward. We can look at solar, wind, smart grid, CCS, electrical storage, nuclear, automotive power, jet power, etc, and for each one examine the state of the technology, public and private investment, and hurdles to broader production and adoption. We can look for similar historical situations and what worked or didn’t, or we may decide that some are without historical precedent.

The result wouldn’t be a million Manhattan Projects, but maybe something like two Manhattan Projects, four Polio Eradication Campaigns, three IT Revolutions, a WW II Aircraft Miracle, and a few hundred thousand Internet Startups, four of five of which might hit the big time.

Through this process we can approach the type of roadmap that Weiss and Bonvillian describe in Structuring an Energy Technology Revolution. With such a roadmap our policy and public investments can become more direct and organized, and we will have a better framework for measuring our progress. In an upcoming post I’ll give some examples of how this might look.