Purpose

In the fast-paced world of supply chain management, accurate delivery predictions can make or break a company's operational efficiency. We're excited to introduce our groundbreaking new feature that harnesses the power of machine learning (ML) to predict purchase order delivery dates with unprecedented accuracy.

What does it do?

Our new ML-powered PO Delivery Risk feature predicts the likely delivery date for each purchase order in your system. It's like having a crystal ball for your supply chain, but one based on data and advanced algorithms rather than magic.

How does it work?

The feature uses a sophisticated ML model trained on historical delivery data. It takes into account various factors, including:

• Details about the item being ordered
• Information about the supplier
• Other relevant order details

What's unique about our approach is that we use natural language processing (NLP) techniques to analyze text data such as supplier names and item descriptions. This allows the model to make intelligent predictions even for new items or suppliers that share similar characteristics with those in our historical data.

How will you see it?

You'll notice a new ⚠️ icon appearing next to purchase orders in your report. Simply hover over this icon to see a breakdown of likely delivery dates and their probabilities. No need to decipher complex charts or data – we've made it as simple as possible.

Why does it matter?

Imagine knowing in advance that a critical component will likely be delayed. With our new feature:

1. Buyers can proactively manage at-risk orders.
2. Planning teams can adjust production schedules based on more accurate information.
3. Customer service can provide more reliable delivery estimates to customers.

In essence, this feature turns potential surprises into manageable situations, helping you stay ahead of the curve.

Imagine knowing in advance that a critical component will likely be delayed. With our new feature:

1. Buyers can proactively manage at-risk orders.
2. Planning teams can adjust production schedules based on more accurate information.
3. Customer service can provide more reliable delivery estimates to customers.

In essence, this feature turns potential surprises into manageable situations, helping you stay ahead of the curve.

How accurate is it?

During our initial tests, we found that our ML model reduced the average prediction error to just 9.6 days, compared to 23.8 days when using standard ERP lead times. That's a 60% improvement in accuracy!

When can you start using it?

We're putting the finishing touches on this feature and expect to roll it out within the next few months. Stay tuned for the exact release date.

What key elements are taken into account?

Predicting exact delivery dates when a purchase order (PO) is placed is extremely difficult. This model focuses on predicting the likelihood of delivery within a range of dates, rather than predicting a single date. For each PO, the model provides probabilities (e.g., 25%, 50%, and 75%) for delivery by specific dates. 

To land on these probabilities, the machine learning model uses natural language processing (NLP) techniques to represent suppliers historical performance and delivery behavior and items past delivery timelines, building a model tailored to each and trained using historical delivery data.

This approach reduces the risk of overfitting, ensuring the model generalizes well to different suppliers without focusing too narrowly on specific cases or outliers.

What's next?

This is just the beginning. As we gather more data and feedback, we plan to expand the feature's capabilities. Future updates may include:

• Shortage risk score
• New visualizations to help you understand and act on the predictions
• And more!

With this new ML-powered feature, we're not just predicting the future of your supply chain – we're helping you shape it.

FAQs

For those unfamiliar with ML

1. What is "machine learning" in this context?
   • It's a system that learns from historical delivery data to make predictions about future deliveries.
2. How reliable are these predictions?
   • Initial tests show a 60% improvement in accuracy compared to standard ERP lead times.
3. Will this replace our current estimation methods?
   • It's designed to work alongside existing methods, providing additional insights.
4. Do we need to input additional data?
   • No, the model uses existing data about items, suppliers, and order details. However, the PO risk indicator could help in revealing data health issues if they are present.
5. Will the system learn and improve over time?
   • Yes, as it processes more data, its predictions should become more accurate.

For those familiar with ML

1. What type of ML model are you using?
   • The model predicts a probability distribution across delivery dates for items on order. It is a regression-based approach that learns how an order’s supplier, timeframe, size, and other properties interact to shift the likelihood of delivering on time.
2. How are you handling text data?
   • Vectorization techniques derived from NLP systems are used to process text data like supplier names and item descriptions to maximize generalizability across related entities.
3. What evaluation metrics are you using?
   • Our metrics change as we explore the space of tradeoffs in model performance
   • Some examples of model evaluation criteria:
     1. Mean days of error in the model versus a baseline of using the ERP lead time
     2. Correlation between our predicted outcomes and actual outcomes
     3. Precision and recall in predicting late deliveries
4. How are you dealing with concept drift?
   • The model is retrained frequently, and we track its performance using ML Ops tools to ensure the training data is representative of what is being seen in production.
5. What's your approach to interpretability?
   • We predict a distribution over expected delivery dates, allowing exploration of different scenarios
   • We use analysis based on Shapley values to understand the contributions of different features to the model’s decisions
   • We track experiments to understand how changes to the model’s hyperparameters impact its performance.

For the more experienced

1. Is this using deep learning?
   • Yes, we used historical deliveries to train a deep learning model to predict future deliveries based upon: the item, supplier, timing, and other order details.
2. Are you using supervised or unsupervised learning?
   • The model is trained on historical data with known outcomes, making it a supervised system.
3. What kind of big data infrastructure are you using?
   • We make use of the existing LeanDNA data stack to provide source data via AWS data lake technology
   • AWS Sagemaker runs the training and inference jobs
   • MLFlow is used for experiment tracking and model registration.
4. How are you handling the bias-variance tradeoff?
   • The model is designed to reduce text inputs to a parsimonious vector representation
   • We track the impact of different features on the model’s decisions and may discard features that are not impactful
   • We experiment with model sizes, regularization and dropout, target variable definitions, and other methods of ensuring that we have right-sized the model’s complexity.