Feedback loop management This helps prevent model performance from spiraling downwards and allows human intervention using HITL.
Security and compliance The MM solution should provide encryption, access controls, and audit logs, among other security features.
By incorporating these crucial aspects, an MM solution empowers organizations to proactively monitor, detect, and mitigate potential issues, enabling them to make informed decisions and maintain the integrity of their ML initiatives.
Applying MM in the real world — A use case example
Imagine you have an ML model that predicts the sales volume of a particular product based on several features, one of which is the product's price. This model has been trained on historical data where the product price was relatively stable. In response to external factors such as rising raw material costs, shifts in market competition, and inflation, the product’s price experiences notable fluctuations over time. Consequently, the correlation between price and sales volume deviates from the pattern observed in the training data, illustrating the concept of concept drift. Left unchecked, the model’s sales volume predictions may become inaccurate, leading to poor business decisions and potentially significant revenue loss. Therefore, detecting this drift in the data (in this case, the change in product price) is critical. Once the drift is detected, it signals that the predictive model may need to be updated or retrained to maintain its accuracy. Discovering drift detection methods entails recognizing alterations in data distribution, such as employing the KS test, utilizing ML methods capable of detecting distribution changes, or employing straightforward techniques like monitoring the mean or variance of the price. Subsequently, an alert will be triggered if significant changes are detected. Standard statistical tests employed in this scenario include the KS test, Page-Hinkley, PSI/PPS, and CSI.
© 2023 Fractal Analytics Inc. All rights reserved
06
Made with FlippingBook - PDF hosting