Too Much Dairy Data!

Justin Howes

No shortage of data exists on modern dairy operations. But, you need more than just data to unlock a herd’s potential

The author is the strategic dairy marketing lead for Cargill Animal Nutrition.

Justin Howes

Think about all the available data a herd manager of a dairy operation has at their disposal today. Daily milk weights, reproductive records, health events, activity reports, and so forth. We collect that data in a multitude of places and formats. Some data is in spreadsheets, other in herd management software. On some farms, some of it is still recorded on paper while others via a mobile device. The challenge facing the dairy industry today isn’t a lack of available data, it’s a lack of ability to use that data in a way that is meaningful for a producer. By some estimates, we only use 10 to 15 percent of the power of the farm data we have today, and most of that is used for daily tasks rather than making strategic, long-term decisions.

Unlocking the power behind the data is a complex challenge facing our industry. But unlike other challenges like animal welfare or other consumer pressures, this data dilemma has the chance to have a powerful impact on the ability of a producer to drive their business forward. Before we dig into the challenge of unwinding data and making it usable, it’s important to understand the difference between data and data analytics.

  • Data is factual information, usually in a numerical form. It is most commonly outputted from a sensing device. Examples would include pounds of milk produced, daily steps recorded, or a case of mastitis.
  • Data analytics is the process of examining data sets to draw conclusions about the information they contain. One example would be monitoring for dramatic changes in milk production or daily activity to identify an illness to be treated.


What are all the challenges?

The data and analytics we have available on our farms today tend to be used primarily for daily management work. This would be things like a list of cows to be checked for illness, possibly in heat to be bred, or sorted into the wrong pen. These action lists and performance monitors do save time for producers, but they only scratch the surface of the power of the data within each system.

Several challenges face our ability as an industry to unlock the power of our data:

Increasing availability of data – we have more of it available than ever before.

Not in optimal form – the data we have is not in a usable form.

Lack of time – a huge limiting factor is not having enough time to go through it.

Prone to misinterpretation – different conclusions can be drawn from the same data set by different people.

Lack of collaboration – systems won’t talk to each other and/or consultants may not share data with other dairy influencers.


These challenges are not insignificant and go across multiple stakeholders. However, the opportunity for a dairy producer, and the companies that support them, is huge. Our goal is to use data analytics to make better decisions, and to make them while they are happening. Today, too many of our decisions must be made after an event has happened. Through the development of predictive tools within some of the sensing technologies, we have gotten more timely. But, how timely can we realistically be, without a limitless number of hours and talent available on most farms to look at the data in the first place?

Building the ideal system

Technology should help you make the right decision at the right time. By building a holistic dairy management system that integrates data from multiple sources including the nutrition formulation system, dairy producers will be able to have 24/7 access to seamless, structured and actionable info, allowing them to improve operational efficiencies. To build that kind of system, we need to integrate data from four different types of data sources:

  • Input data – What is going into the animals and the dairy? This includes nutrition formulation, feed ingredients, animal health protocols and other things we do to the cow.
  • Output data – How are the cows doing? This would include milk production, component levels, somatic cell score and other cow performance measures.
  • Farm environment – What is happening on the dairy that might affect cow performance? Is it hot, cold, windy, wet, dry? Are pens overcrowded, or is there enough bunk space? A system should include these types of environmental factors.
  • Animal behavior – What is going on inside the cow? This dataset would include activity monitoring, rumination scoring, and outputs from other behavioral sensors, including the new camera systems ready to hit the market.

Now that we know what data we have; the challenge is to consolidate it from multiple systems and turn it into relevant and actionable information to anticipate problems and plan a brighter future. Fortunately, we see examples of these kinds of systems in other industries that give us reason to believe they could be employed within agriculture.

One example is the GE Predix Platform which is used to optimize the operation of large industrial machines such as power plants and wind turbines. The Predix Platform is built to integrate data from internal and external data sources ranging from the machine itself to the environment within which its operating. This data is used by data analytics and complex machine learning to provide a user interface to manage the system. The benefits are more productive machine operation, less downtime by anticipating repairs before they’re needed, and a higher efficiency for the operation.

Now, managing a wind turbine is a whole lot less complex than managing a biological system like a herd of cows. Machines are built to do the same task over and over again. Cows, while creatures of habit, we know have minds of their own! But, that doesn’t mean we can’t use what other industries have created as an inspiration for a system from which a modern dairy operation could benefit.

We know at its core, this integrated system must have a data collection function, a data science ability, and a farm-level dashboard for optimal use and decision-making. With this in mind, Cargill has been developing a platform for producers called Dairy Enteligen®. Dairy Enteligen can integrate data from all herd management systems, including nutrition formulation, to offer a complete, real-time dashboard of operation insights. As the figure shows, the core input areas for this system include cow comfort, animal health, nutrition, and cow productivity measures. Information is collected across these measures by both people and sensing technologies and uploaded to a cloud. Placed alongside the Cargill nutrition formulation software, the MAX™ system for Dairy, it provides an all-inclusive picture of what’s happening on the dairy.

Dairy Enteligen is currently being trialed on dairies around the world and is one of several platforms aiming to build the system of data management and analytics that the dairy industry will need to compete in the future. As margins get tighter, producers will need to make more timely and valuable decisions. They will need access to information to make informed choices.

The right decision at the right time. That’s the answer to the dairy data overload facing our industry. Unlocking the power of farm data will allow us to double the impact of the available data, and to use it in a way that goes beyond simple daily tasks to real-time, confident decision making. The future for a dairy producer is using analytics for strategic management decisions. It’s the opportunity to make proactive management decisions, as opposed to reactive problem fixing.

As an industry, it’s an exciting challenge ahead of us.

For more information, click here

Figure 1