H. Liu et al. : Effluent Quality Prediction of Papermaking WWTPs Using SEL
A. TRAINING AND TESTING PROCESS For the training process, three base-learning algorithms including PLS, SVR, and ANN are defined as ζ 1 , ζ 2 , and ζ 3 , respectively. As the meta-learning algorithm, MLR is defined as ξ . In the training process, five-fold cross validation approach is adopted. Original training set D ={ ( x 1 , y 1 ) , ( x 2 , y 2 ) , · · · , ( x m , y m ) } is randomly divided into five data sets with the same size as D 1 , D 2 , D 3 , D 4 , D 5 and x i represents sample feature, y i represents the target value. Among which, four data sets are used for training base-learners, and the remaining data set is used to verify the predicted performance, then optimizing the specific parameters of base-learner h ( j ) i fromthe i -th learning algorithm ζ i . The above-mentioned process needs to be car- ried out five times. Then, the well-trained base-learner h ( j ) i is used to predict the samples which not participate in modeling, and the prediction result can be expressed as h ( j ) i ( x i ). The secondary training set produced by the three base-learning algorithms can be expressed as D = { ( x i , y i ) } m i = 1 . Among them, x i = ( h ( j ) 1 ( x i ), h ( j ) 2 ( x i ), h ( j ) 3 ( x i )), and y i is still the target value in the original training set D . The D is used for training the meta-learner h = ξ ( D ) by the meta-learning algorithm ξ . For the testing process, five base-learners h ( j ) i get corre- sponding prediction results for original test data set D test , which can be described as h (1) i ( x t ), h (2) i ( x t ), h (3) i ( x t ), h (4) i ( x t ), and h (5) i ( x t ). By averaging all of the prediction results, a pre- diction vector is obtained as follows : h i ( x t ) = h ( j ) i ( x t ) / 5 (1) Three base-learning algorithms produce three prediction vec- tors, which can be described as x i = ( h 1 ( x t ) , h 2 ( x t ) , h 3 ( x t )). Because y i in D test has not changed, the secondary test set can be expressed as D = { ( x i , y i ) } n i = 1 . By applying the D into the meta-learner h , the final prediction result H ( x ) = h ( D ) is obtained. The implementation process of SEL and the formation process of the secondary data set is shown in Figures 1 and 2, respectively. n = 5 j = 1
FIGURE 1. Scheme of stacking ensemble learning.
FIGURE 2. Formation process of secondary data set.
where P and Q are loading matrices, T and U are latent matrices which carry enough variation information in X and Y , E and F represent residual matrices. 2) SUPPORT VECTOR REGRESSION Support vector regression algorithm is the modified version of support vector machine (SVM) which can be used to solve linear and non-linear regression tasks. It performs better for small-scale data set by using the structural risk minimization principle. The key to SVR lies in finding suitable mapping function ϕ ( x ) between input vectors and output vectors. With the help of mapping function ϕ ( x ), the training data T = { ( x 1 , y 1 ) , ( x 2 , y 2 ) , · · · , ( x i , y i ) , · · · , ( x n , y n ) } is mapped into a high dimensional feature space. Then, an optimized regression function can be constructed in the new feature space as follows : f ( x ) = w T ϕ ( x ) + b (3) where w is the weight vector and b represents the bias term. By introducing insensitive loss function ε , the errors within a specified tolerance range can be neglected. Then, the slack variables ξ i and ξ ∗ i are added into Equation (3). The regression task can be transformed into an optimization problem as follows : ⎧⎪ ⎨ ⎪⎩ y i − w T ϕ ( x ) − b ≤ ξ i + ε, i = 1 , 2 , · · · , n w T ϕ ( x ) + b − y i ≤ ξ ∗ i + ε, i = 1 , 2 , · · · , n ξ i ξ ∗ i ≥ 0 , i = 1 , 2 , · · · , n (4) VOLUME 8, 2020
B. BASE-LEARNING ALGORITHMS 1) PARTIAL LEAST SQUARES
Partial least squares algorithm has been widely used in regression, mainly by finding the reasonable latent variables. Assume that input matrix is X = [ x 1 , x 2 , · · · , x q ] n × q and output matrix is Y = [ y 1 , y 2 , · · · , y p ] n × p . Among them, n is the number of the samples, q and p represents the number of input variables and output variables, respectively. To better exploit variance structures of process, X and Y are projected into a lower dimensional space as follows : X = TP T + E Y = UQ T + F (2)
180846
Made with FlippingBook flipbook maker