How to Measure and Interpret Forecast Error: Bias and Percentage Error Metrics

In brief: Forecast error is the difference between the demand forecast and the actual demand; in practice the same meaning is also expressed as "forecast deviation". Inventory building up in the warehouse while reports show low error is a common situation in practice. One common explanation is that the two separate dimensions of error, namely its size and its direction, are not monitored together. Mean absolute percentage error (MAPE) and weighted mean absolute percentage error (WMAPE) show the size, while bias shows the direction. This article covers how error is calculated, which metric is used in which situation, and how results are interpreted; it also examines the role of technology in collecting and analyzing data and monitoring errors. The general framework of forecasting methods can be reached on the What Is Demand Planning? page.

What Is Forecast Error and Why Is It Measured?

Forecast error is the difference between the forecast made for a given period and the demand realized in the same period. Error is calculated as Error = Actual − Forecast. When actual demand is above the forecast, the error is positive; when it is below, it is negative. In this article the expression "forecast error" is used; "forecast deviation" can be used interchangeably in the same sense.

The reason for measuring error is that planning decisions rely on the forecast. When the forecast is too high, materials, capacity, and inventory are planned in excess; when it is too low, customer orders cannot be met, and lost sales and urgent shipment costs arise. Knowing at what level and in which direction the error lies is the first condition for improving both the forecasting method and the planning process.

Consider an example of two planners forecasting the same product group. Both have a mean absolute percentage error of 9%. The first errs upward one month and downward the next; the second gives a forecast above demand every month. In terms of size, their performance is the same. The picture in the warehouse is different: the first's errors balance each other out in stock, while the second's errors add up every month and accumulate as permanent excess stock. For this reason, the size and direction of error must be monitored separately.

Which Decisions Should Be Made Before Calculating Error?

Before moving to the calculation, the sign of the error must be clarified. One of the confusions frequently seen in practice is one department using "Actual − Forecast" and another using its opposite. This direction should be fixed in writing within the organization. In this article error is defined as Error = Actual − Forecast; a positive value shows that demand came in above the forecast (the forecast was too low), and a negative value shows that the forecast was above demand.

The second decision is the level of measurement. Error should be measured at the level at which the decision is made. If production and capacity decisions are made at the product family level, error is also calculated at that level; if distribution decisions are made at the product-location level, at that level. As the aggregation level rises, error becomes smaller because positive and negative errors of products partially offset each other; for this reason, values calculated at different levels are not compared with each other.

The third decision is which forecast version will be compared with actuals. If the production plan is frozen two weeks ahead, the actual is compared with the forecast given two weeks earlier; a comparison with the forecast updated at the end of the period does not reflect the forecast on which the decision was actually made. For this reason, versions of the forecast at different lags must be stored for each period.

The fourth decision is what the term "actual" refers to. Shipment data does not show demand that could not be met because of stockouts; order data may include cancellations and postponements. Where possible, a demand series to which unfulfilled demand is also added should be used, and the definition used should be fixed in writing. Finally, the measurement period is determined: weekly data contains more volatility, while monthly data smooths the error; if decisions are made weekly, measurement should also be weekly.

How Is the Size of Error Measured? Percentage Error Metrics

Size metrics measure how wrong the forecast is without looking at the direction of the error. For this, the absolute value of the difference between forecast and actual is used; thus, positive and negative errors do not cancel each other out.

Mean Absolute Deviation (MAD) is the average of absolute errors and is expressed in the product's own unit (units, cases, kilograms). It is easy to understand for a single product; however, because units differ, it is not suitable for comparison across products.

Mean Absolute Percentage Error (MAPE) is the average of the percentages found by dividing the absolute error for each period by the actual of that period. Because it is unit-independent, it allows different products to be compared. It has two limitations. When actual demand is close to zero, the percentage error becomes very large or undefined. In addition, a high percentage error on a low-volume product carries the same weight in the calculation as the same percentage on a high-volume product.

Weighted Mean Absolute Percentage Error (WMAPE) is obtained by dividing the sum of absolute errors by the sum of actual demand: WMAPE = Σ|Actual − Forecast| / Σ Actual. Because high-volume products affect the result more, it presents a more balanced picture than MAPE in portfolio-level reporting. Forecast accuracy is mostly reported as 1 − WMAPE; therefore, accuracy percentage and error percentage are two different expressions of the same information.

Root Mean Square Error (RMSE) is calculated by taking the square root of the mean of the squared errors. Because it gives more weight to large errors, it is preferred when errors that occur rarely but are costly are important. The error measure used in safety stock calculation is also based on this logic (see 5 Common Mistakes in Safety Stock Calculation and the Correct Approach).

Metric selection depends on the purpose. MAD or RMSE can be used for diagnosis at the product level, WMAPE for portfolio-level reporting, and MAPE for products where percentage comparison is needed and zero demand is rarely seen. The metric definition should not be changed within the same report, and which metric is used should be clearly stated in the report.

Bias: How Is the Direction of Error Measured?

Bias is the sum of errors taken together with their signs: Bias % = Σ(Actual − Forecast) / Σ Actual. A positive value shows that the forecast is systematically low, and a negative value that it is systematically high. When error is random, positive and negative values cancel each other out and bias approaches zero.

Assume that the six-month forecast of a product group is 100, 110, 105, 120, 115, and 110 units, and its actual sales are 92, 101, 99, 108, 104, and 103 units (figures are for illustration purposes). The monthly errors (Actual − Forecast) are −8, −9, −6, −12, −11, and −7 respectively, and the total error is −53 units. Since total actual is 607 units, WMAPE is calculated as 8.7% and bias as −8.7%. The two values being equal in absolute terms shows that all of the error is in one direction: the forecast stayed above demand every month, and about 9% more material and capacity was planned for this product group.

It can be assumed that in the same period another product group's monthly errors were −8, +9, −6, +12, −11, and +7, with total actual again 607 units. The sum of absolute errors is again 53 units and WMAPE is again 8.7%; however, since the net sum of errors is +3 units, bias is about +0.5%. The size metric gives the same value for the two groups, while bias reveals that the behavior of the two groups is completely different.

The importance of this distinction is that the solutions for the two error types differ. Random error can be covered with safety stock, whereas systematic error (bias) can be eliminated only by correcting the forecasting process. Compensating bias with safety stock does not solve the problem; it increases inventory cost without removing the cause.

Bias is assessed not over a single period but over a rolling window. The total of the last 6-12 periods should be used; otherwise, errors from years ago suppress the current picture. A large error in a single period (for example, an unexpected large order) is treated not as bias but as an event. To be able to speak of bias, the error must repeat in the same direction for several consecutive periods.

How Are Error Results Interpreted?

Error values are not interpreted as good or bad on their own. A high-volume product with stable demand and a low-volume product with fluctuating demand cannot be evaluated with the same error value. For this reason, error is interpreted within segments determined by the product's volume and demand variability, and by comparison with the organization's own past periods.

When size and direction are read together, four situations can be distinguished. If error is large and bias is small, the forecast errs randomly; the solution is to improve the forecasting method and its inputs, and safety stock for the remaining uncertainty. If error is large and bias is of the same magnitude, the error is systematic; the solution is to find the source of the error in the forecasting process. If error and bias are both small, the process is healthy. If error is small but bias is close to the error, there is a small but persistent drift; since this will accumulate in stock over time, it requires early intervention.

The course of error over time also carries information. If error is rising, the demand structure may have changed (a new customer, pricing policy, a competitor move); if bias has changed direction, the assumptions on which the model relies may have become outdated. As the forecast horizon lengthens, error naturally grows; for this reason, the errors of one-week and three-month forecasts are not evaluated on the same scale.

An error that looks small at the total level can be large in sub-breakdowns. Looking at error by breakdowns such as product family, customer, channel, and planner shows in which subset the error is concentrated and narrows the search for the root cause.

What Are the Most Common Sources of Error?

• Entering the target instead of the forecast: When the sales target or budget is entered into the system as if it were expected demand, the forecast systematically shifts upward.
• Adding a "safety margin" to the forecast: Protection against uncertainty is the function of safety stock; inflating the forecast distorts both inventory and the capacity plan.
• Double-adding the campaign effect: If the statistical model already includes past campaigns, adding a campaign effect on top of it leads to demand being counted twice.
• Not counting lost sales as demand: Sales realized during stockout periods are lower than true demand; models built with this data systematically under-forecast demand.
• New product optimism and trends reaching saturation: The assumption that the growth curve will continue with its past slope leads to high forecasts in the maturity period.
• Data quality problems: Product code changes, unit conversion errors, and late-processed return or cancellation records can produce an error that does not exist in reality.

How Is Error Managed in Planning Meetings?

Measurement is only one part of the process; the real difference emerges in which decision the result is converted into at the meeting. It is recommended that product groups whose bias goes outside the defined band be a standing agenda item in the demand review meeting. For each of them, the root cause, owner, and action (model change, input correction, process rule) should be recorded as a decision. An organization-specific acceptance band at the product family level (for example, ±5%) should be defined, and families going outside the band should be carried to the management S&OP meeting. The decision logic of the meetings is covered in the article Which Decision in Which Meeting in the S&OP Cycle? Calendar, Decision Logic, and Scenarios.

The random component of error is transferred separately to inventory planning because it determines the variability used in safety stock calculation. In this way, systematic error is handled in the process and random error in stock.

Technology and Digitalization: Collecting Data, Analysis, and Interpreting Errors

None of the calculations given above is complicated; they can be done in a few minutes on a spreadsheet. The real difficulty is repeating the same measurement consistently every period across thousands of product-location combinations and interpreting the results meaningfully. The contribution of digitalization on this subject can be considered in three stages: collecting data, calculating errors, and interpreting results.

Collecting and Consolidating Data

The basis of error measurement is storing forecast and actual together at the same product, location, and period level. Enterprise Resource Planning (ERP) systems are the main source of actual data because they record order, shipment, inventory, and price transactions in a common database. Sales and inventory data received automatically from distributors and retail chains through Electronic Data Interchange (EDI) create a series closer to true demand than shipment data. Connected points of sale, vending machines, and smart shelves within the Internet of Things (IoT) transmit consumption without human input, making systematic error in shipment-based forecasts visible.

Record discipline is also needed on the forecast side. In every planning cycle, the relevant versions of the forecast (statistical forecast, forecast after sales adjustment, approved forecast) must be stored as snapshots for different lags; in systems that keep only the current forecast, past error cannot be calculated afterwards. Automatically collecting these records from source systems and consolidating them in a common data warehouse allows measurement to be made from a single source of truth. A data warehouse is an analytical database in which data from different source systems is stored in a single, consistent structure.

Calculating and Analyzing Errors

Once data is collected in one place, metrics such as MAD, WMAPE, RMSE, and bias can be calculated automatically across product, product family, customer, channel, and planner breakdowns. Calculation can be done according to the lag at which the decision is made and can be rolled up consistently between aggregation levels. In this way, the formula and scope differences that are inevitable in manual work as the number of products grows are also eliminated.

In the analysis stage, Business Intelligence (BI) tools form the analytics layer that combines data from different sources and presents it to decision makers through dashboards and reports. When error results are monitored with heat maps and drill-down dashboards, one can see in which product family the error is concentrated and in which direction and for how long bias has persisted. Presenting items that exceed the threshold to the planner as an exception list makes it possible to focus on only those that require intervention among thousands of items.

The Contribution of Artificial Intelligence in Interpreting Errors

Artificial Intelligence (AI) is the general name for methods that produce forecasts, classifications, and recommendations by learning patterns from data; Machine Learning (ML) is its sub-field that learns from data. These methods are used in error analysis in two ways. The first is automatically scanning for patterns in error: bias that repeats in a particular customer, channel, or period, calendar effects, or structural breaks can be found in a level of detail that the human eye might miss. The second is identifying the variables that explain error (price, campaign, weather, calendar).

Generative Artificial Intelligence (GenAI) is a type of artificial intelligence that produces summaries and draft explanations in natural language based on large language models. It can turn these findings into readable text for the error review meeting; verifying the rationale and making the decision remain the planner's responsibility. Because an artificial intelligence model can also learn the systematic errors in the data it is trained on, it is recommended that the model's output be monitored with the same metrics (WMAPE and bias) and that the planner be able to see which variables influence the forecast (explainability).

Error measurement begins with accurate and consistent data;
the more detailed the data is collected, the faster the cause of the error is found

Implementation: Checklist

1. Fix the direction of the error sign (Actual − Forecast) in writing in the procedure.
2. Determine the level and lag at which error will be measured according to the point at which the decision is made.
3. Put the definition of "actual" demand (shipment, order, whether unfulfilled demand is included) in writing.
4. Present the size metric (WMAPE) and bias showing direction together in the same report.
5. Report the rolling 6-12 period bias monthly by product family.
6. For families exceeding the acceptance band, decide the root cause, owner, and action in the demand review.
7. Consolidate ERP, point-of-sale, and shipment data in a common data warehouse; store forecast versions as snapshots for different lags.
8. Monitor artificial intelligence or machine learning output with the same error metrics.

Related Articles

• Which Decision in Which Meeting in the S&OP Cycle? Calendar, Decision Logic, and Scenarios
• 5 Common Mistakes in Safety Stock Calculation and the Correct Approach
• How to Forecast Promotion Impact: Campaign Types and Uplift Calculation
• Time Series Analysis for Demand Forecasting: Stationarity, Trend, and Seasonality

Frequently Asked Questions

Are forecast error and forecast deviation the same thing?

Yes, they are two expressions describing the same difference and are often used interchangeably. The distinction lies in which dimension of the error is measured: metrics such as MAPE and WMAPE show the size, while bias shows the direction.

Should MAPE or WMAPE be used?

In portfolio-level reporting, WMAPE generally presents a more balanced picture because high-volume products affect the result more. MAPE is useful for comparing products one by one because it is unit-independent; however, it produces misleading values in periods when actual demand is close to zero.

Above what value is bias considered a problem?

There is no universal threshold. The acceptance band is set in writing by the organization according to the product segment and the cost of the decision (for example, ±5% at the product family level). How many consecutive periods the error repeats in the same direction matters as much as the size of the value.

At what level should error be measured?

Error should be measured at the level at which the decision is made. If production and capacity decisions are made at the product family level, error is also calculated at that level; if distribution decisions are made at the product-location level, at that level.

Does artificial intelligence eliminate forecast error?

No. Artificial intelligence can also learn systematic errors in data and assumptions. What reduces error is regularly measuring the model and human adjustments with metrics such as WMAPE and bias and intervening according to the result.

References

• Hyndman, R. J., & Koehler, A. B. (2006). Another look at measures of forecast accuracy. International Journal of Forecasting, 22(4), 679-688.
• Kolassa, S., & Schütz, W. (2007). Advantages of the MAD/MEAN ratio over the MAPE. Foresight, 6, 40-43.
• Hyndman, R. J., & Athanasopoulos, G. (2021). Forecasting: Principles and Practice (3rd ed.). OTexts.
• Silver, E. A., Pyke, D. F., & Thomas, D. J. (2017). Inventory and Production Management in Supply Chains (4th ed.). CRC Press.

Related Optimum Planlama Trainings

Companies That Trust Us

Akkim, 2023

Alarko Holding, 2011

Anadolu Efes, 2021

Barilla Food, 2015

BVN (Bahcivan), 2020

Cilek Furniture, 2017

Civil Retail, 2019

Eczacibasi Esan, 2021

EssilorLuxottica, 2022

Global Bilgi, 2021

Global Pharmaceutical Company, 2023

Haribo, 2017

Hascelik, 2015, 2022

Ibavalresa, 2025

Istikbal Furniture, 2012

Jantsa, 2019

Kaleseramik, 2015

Kastamonu Entegre, 2017

LCWaikiki, 2010, 2012

Makbul Nuts, 2021, 2022

Migros, 2021

Nivea Türkiye, 2019

Opet Fuchs, 2014

Panasonic Electric Works, 2022

Red Bull Türkiye, 2023, 2025

Rengarenk Furniture, 2017

Sarsilmaz Firearms, 2023, 2024

Sika Türkiye, 2018

Sisecam Holding, 2013–2016

SKT Spare Parts, 2019, 2023

Turk Telekom, 2013–2018

Zorluteks, 2023

Comments

“We had a productive training for two days. As a company, we have seized opportunities to see our shortcomings and make improvements. Thank you very much.”

Yiğit Doruk Turguter, Senior Engineer in Production and Demand Planning, Petrol Ofisi (Demand Planning, 2024)

“It was a training that was useful for our sales and planning group. Thank you.”

Murat Işık, Planning & ERP Manager, Sarsilmaz Firearms (Demand Planning, 2023)

“It is a training that every company operating in a geography like Turkey, where uncertainty is increasing day by day, should receive. I think that this training will provide a great advantage in our business processes, as it is very critical to make the right decisions in an aggressive competitive environment and to make the right predictions in parallel.”

Hale Topaloğlu, Senior Product Manager, Global Pharmaceutical Company (Demand Planning, 2023)

To view all testimonials, click here.

Request a quote for training or consulting tailored to your organization.