LEEANDSEO
14of 19
FIGURE 6 Performance comparison with AUC and F-measure. AUC, area under the ROC curve
0.8
0.12
0.724
0.715
0.694
0.7
0.1
0.097
0.601
0.594
0.6
0.08
0.5
0.378
0.057
0.4
0.06
0.052
0.3
0.038
0.04
0.03
0.028
0.2
0.02
0.1
0
0
Autoencoder LSTM-Autoencoder Decision Tree
SVM
CL-LNN
RD-LNN
AUC F measure
TABLE 4 Comparison between Euclidean and DTW distance
Item
Euclidean distance DTW distance Remark
TN (True negative) FN (False negative) TP (True positive) FP (False positive)
1514
1528
19
19
6
6
Detect failures
282
268
F-measure
0.038 0.601
0.040 0.618
(2 × TP)/(2 × TP + FP + FN)
AUC
Running time 5.13hours Note: The boldfaced font was used to emphasize the difference of running time between two methods. (the other metrics are similar). Abbreviations: AUC, area under the ROC curve; DTW, dynamic time warping. 4.6 minutes
FIGURE 7 The effect of the window size in RD-LNN for the training process. RD-LNN, relative distance of the local nearest neighbor
0.10
8
0.08
7
0.06
0.04
6
0.02
5
0.00
20
40
60
80
100
Window Size
Another parameter we need to carefully determine is the number of normal instances randomly selected in the train- ing dataset. We examined the effectiveness of the number of normal instances with the performance depicted in Figure 8. Note that 99 failures are included in the training dataset and the class distribution between failures and normal instances needs to be balanced to handle imbalanced dataset. It shows that F-measure increases when the number of normal instances for training increases from 100 to 200, and then significantly decreases after 200 while running time keeps ris- ing over the number of normal instances. This indicates that 200 normal instances we randomly selected in the training dataset provide better performance than others.
Made with FlippingBook - Online catalogs