Delivered for a large dairy manufacturer, through Platform to Platform and Data to Data.
The challenge. Product costs were re-estimated manually against a raw material whose price moves with fat content, quality, season and supply, so pricing decisions across a 100+ SKU catalogue were made on stale cost assumptions.
Context
Between 2021 and 2022 we built a price forecasting and cost optimisation platform for a large dairy manufacturer with a catalogue of more than 100 products. It recomputes product cost daily from recipe level, forecasts where input cost is heading, and recommends a price per SKU.
The problem
Dairy pricing is a cost problem disguised as a pricing problem.
Milk is the dominant input for most of the catalogue, and its price is not a single number. It moves with fat content, with quality grade, with season, and with local supply. It is also perishable, which means the cost of holding it is not storage cost alone — it is storage cost plus a spoilage probability that rises with time.
Every product in the catalogue is a recipe drawing on that input plus others: sugar, packaging, transport, energy. So a movement in the milk price does not change one cost, it changes a hundred costs by a hundred different amounts, depending on how much milk each recipe uses and in what form.
The manufacturer was estimating this manually. Manual estimation across 100+ SKUs has a predictable failure mode: it happens infrequently, it uses last known input prices, and it converges on round numbers. In a stable season that is tolerable. In a volatile season the prices being charged reflect a cost structure that stopped existing weeks ago, in one direction or the other.
Why it is hard
The optimisation is not linear and it is not separable. Price affects volume, volume affects how fast stock moves, how fast stock moves affects spoilage loss, and spoilage loss is part of cost. Setting price to maximise margin per unit does not maximise margin. Products also compete with each other for the same input, so pricing one SKU changes the correct price for another.
Wastage is the number nobody has. Recipe cost is knowable and often already documented. What actual usage was, against what the recipe says it should have been, is the difference between theoretical and real cost — and it lives in inventory logs that were not written to be analysed.
Perishability makes the forecast horizon short and the penalty asymmetric. Underpricing loses margin. Overpricing loses volume on a product that will be written off if it does not sell. Those are not symmetrical errors and an optimiser has to be told so.
Seasonality is real but not clean. Milk supply seasonality interacts with demand seasonality, and the two do not peak together.
What we built
Daily ingestion with a purpose-built ETL layer. Java ETL scripts pull and cleanse POS transactions, inventory and wastage logs, supplier pricing for milk and other ingredients, and seasonal supply and demand data. Landed daily into a central NoSQL store sized for fast recomputation rather than for reporting.
Recipes as structured formulas. Each product’s recipe is stored as a formula with raw material quantities and dependencies captured in relational form, so cost is computed rather than looked up. A Java back end recomputes cost across the entire catalogue when inputs move. This is the piece that turns one price movement into a hundred correct cost updates.
Milk price and cost forecasting. Python time-series models fed daily, forecasting milk price from seasonality and market data and projecting production cost over short and mid-term horizons, with historical volatility analysis behind the confidence on each forecast. Built with Statsmodels, scikit-learn and XGBoost.
Non-linear price optimisation. SciPy and Pyomo, optimising price per SKU against raw material price volatility, storage cost, spoilage risk and market competitiveness together rather than sequentially. The optimiser generates scenarios rather than a single answer, because the right price depends on how aggressive the manufacturer wants to be that month.
A dashboard that shows the recommendation and the reasoning. ReactJS with TypeScript, presenting recommended price by SKU, margin projections under varying cost scenarios, and seasonal and inventory-driven alerts. A recommendation nobody understands does not get used, so margin projection under alternative scenarios sits next to the number.
Results
100+ SKUs on one costing engine, recomputed from recipe level rather than re-estimated.
5-12% margin improvement across products during volatile seasons. The range is the honest answer: the gain depends on how milk-heavy the recipe is and how much the input moved. Products with stable inputs saw little change, which is the expected result.
Forecast accuracy above 90% on the daily and weekly cost and price forecasts.
Manual estimation came out of the loop. Pricing updates during seasonal shifts became a review of a recommendation rather than a recalculation exercise.
What we would do differently
We would model wastage as a first-class input from the start rather than deriving it from inventory reconciliation. Wastage was initially treated as a correction applied to theoretical recipe cost. It is better understood as its own time-series, correlated with weather, stock age and how fast a SKU is moving — and forecasting it directly is more accurate than backing it out afterwards.
We would also resist daily reoptimisation of every SKU. The optimiser can produce a new price for everything every day, and a sales organisation cannot execute that. Price changes have a cost that is not in the model: channel friction, customer trust, and retailer relationships. Constraining the optimiser to change fewer prices by larger amounts would have produced recommendations that were acted on more often.
Where this transfers
Formula-based cost modelling over a volatile input, with a forecast horizon set by perishability and an optimiser that balances margin against spoilage, applies directly to FMCG manufacturing with seasonal inputs, to formulation-based industries such as cosmetics and food ingredients, and to any retail estate doing dynamic pricing against live sales and inventory. The distinctive part is not the forecasting. It is holding recipe, wastage and spoilage in one cost model so the optimiser is arguing with real numbers.