Output forecasting solution
Overview
Production of raw materials or inputs where you are dependent on many internal and external influences can be unpredictable. Especially if you are in the production of perishable goods. Having a solution that could predict outcomes, such as the quantity and quality of your outputs, can be beneficial for you as the producer and for the next person in the supply chain, the buyer.
For our clients, it was important to predict quantities and quality so buyers could plan their production and that sellers can anticipate their revenue. It is important to say that this solution needed to be created on such principles that it could be applied in any industry or production that collects or generates data.
The starting point was to delve deep into data from production to determine what affects production the most. If you look at any productive unit, determining what influences productivity was a priority. From internal and external influences, we combined those that are directly linked with the output itself. Each influence that plays a role in output production had to be examined to see if the impact is relevant or not. The historical data also had to be taken into account to see how production fluctuated over time.
The first step was to approach data with due diligence to understand its origin, structure, and quality. Each set of data came from different sources, in different formats, and with different meanings. For the analysis, data had to be prepared to fill out all the requirements. Only when data, historical and current, was in the right directory and the correct format, could we apply data science methods and machine learning to predict future outcomes in production.
After determining the right data sources that have the most impact on the desired results, we chose our technology stack carefully to accompany a solution that will evolve and progress through time. An optimal choice had to be made, so sellers and buyers can both get easy-to-understand results that will drive their decision-making. In the end, we created a self-evolving solution that uses machine learning and daily data intake to predict product quantity and quality. That way producer can anticipate their production and prepare themselves and their operations in line with expected product volume. This solution gets more accurate with time. The more data it consumes, the preciser the results are, and with that, planning of production will be more efficient.
Project scope
Discovery:
- Client interviews
- Data validation
- Project planning
Procedure:
- Data cleansing
- Data analysis
- Baseline model implementation
- Feature engineering
- Model selection
- Deployment to production
Team:
- Data scientists
- Data engineers
Methods:
- Machine Learning
Technologies used:
Python
Pandas
NumPy
Scikit-learn
Vertica
AWS Sage Maker
What we wanted to accomplish:
What we wanted to answer:
What we needed to accomplish:
- To use historical and current data with machine learning to predict trends and outcomes in output quantity
- To create a fully automated, self-evolving cloud solution
- To develop a system that learns through daily data intake and becomes better at predicting
What we needed to answer:
- How much product can you sell in weeks or even months in the future?
- How much product will be on disposal to be processed?
- What are the trends in overall input/output production?
Our process:
- Making sure that the necessary data is in place
- re-engineering of parts of the existing data pipeline is made in order to make necessary data available
- Understanding the data and making new discoveries
- Discovering what does a certain piece of data represent, where and how to extract information for predictions, why are there anomalies in the process
- Giving attributes to certain information subjects
- Creating a tailor-made data prediction solution in accordance with the uniqueness and characteristics of the data source (subject) – every business has a different process (execution time, usage period of resources, etc.)
- Making adjustments according to errors, missing information, bad data, or non-quality data
- Unlocking the possibilities of a self-learning system as new data is added daily
- Building in systems for continuous performance tracking
Solution:
Our goal was very simple. We needed to create a solution that will forecast the quantity and quality of milk. A solution that can be used across a greater number of farms.
For that, we needed to understand what drives milk production. We discovered that milk production is dependent on the animal and its life cycle, feed intake, and external influences such as weather. Cows are nowadays milked by milking machines and sensors that measure how long is the milking time, how much milk could you get from one animal, the quality of the milk, and such. Based on that we generated data for each animal and how much milk will it produce. We attributed numbers to each animal and recorded historical data based on their lactation periods. This could lead to determining an animal’s production cycle and what to expect from a certain animal in the future.
What we did:
A machine learning forecasting solution was applied to determine how much milk can be produced, how much could be sold, and what trends could be expected. This is valuable for both the producer and the processor since these solutions could level the supply and demand throughout certain periods.
Every herd is different and it will give different results and show different discrepancies and anomalies. Different farms hold different breeds, they prolong lactation periods or breed cows at different rates. Each situation affects the lactation period and how much milk could a certain cow produce. Various external influences that producers can’t influence, such as diseases and cow death, also affect the milk quantity. Each factor needs to be used in the solution to reach maximum efficiency and accuracy of predictions. Using daily data (daily milk production) in the solution leads to more accuracy and a more stable model. Bigger quantities of data and a solution that is continuously learning (with the usage of machine learning), lead to trends overview, forecasts, and data visualizations on which producers can base their strategic decisions and reach a certain level of security.
Results:
+10%
Better predictions compared to the baseline
The ability of predictions accuracy to increase based on continuous information intake and machine learning
Fully automated and self-evolving cloud solution
Accurate forecasting that enables faster decision making, regulates costs and improves production management
Admired by
