Case Study
Methodology Use differentially private data with normal model and vice versa Use differentially private data with a differentially private model Test the model for different epsilon values Implementation details: • • • •
Customer: Leading bank in the United Kingdom
Problem: Application of machine learning in home loan lending with personal data protection. Predictive analytics can be effectively employed to minimize human intervention and automate decision-making. The dataset used for training the ML models includes personal data. Hence strict measures to protect user privacy are needed. Solution: Fractal applied differentially private algorithms to protect the user’s identity while using the data for analytics. Employed Random Forest Classifier (RFC) that was made differentially private by adding noise (using Exponential Mechanism) to the prediction probability of labels.
Open-Source library Diffprivlib from IBM; scikit-learn; Pandas, numpy and Matplotlib; Python v3.9 Model performance metrics: Accuracy: 76.72% (DP Data + DP Model); 76.36% (Raw Data + DP Model); 79.42% (DP Data + Normal Model). Note: Not a pplying differential privacy to data and/or model accuracy is 79.98% Prediction probability with different ‘ε’ •
•
The charts below illustrate how a differentially private model predicts the outcome for the same customer. While iterating a DP model multiple times with the same epsilon, probabilities fluctuate slightly, but the outcome (prediction) is the same.
prediction of probability with queries, cust = 1372, e=3
prediction of probability with queries, cust = 1372, e=1
0.8
1.0
Probability variation Probability variation
Probability variation Probability variation
0.7
0.95
0.85
0.6
0.5
0.80
Query number
2.5
5.0
7.5 10.0 12.5 15.0 17.5 20.0
Query number
2.5
5.0
7.5 10.0 12.5 15.0 17.5 20.0
Figure 5: DP model predictions epsilon=3
Figure 6: DP model predictions epsilon=1
4
© 2023 Fractal Analytics Inc. All rights reserved
Made with FlippingBook - PDF hosting