A T-CNN time series classification method based on Gram matrix | Scientific Reports – Nature.com

Sep 22, 2022 by The Beat Maker - 0 Comments

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
Advertisement
Scientific Reports volume 12, Article number: 15731 (2022)
Metrics details
Time series classification is a basic task in the field of streaming data event analysis and data mining. The existing time series classification methods have the problems of low classification accuracy and low efficiency. To solve these problems, this paper proposes a T-CNN time series classification method based on a Gram matrix. Specifically, we perform wavelet threshold denoising on time series to filter normal curve noise, and propose a lossless transformation method based on the Gram matrix, which converts the time series to the time domain image and retains all the information of events. Then, we propose an improved CNN time series classification method, which introduces the Toeplitz convolution kernel matrix into convolution layer calculation. Finally, we introduce a Triplet network to calculate the similarity between similar events and different classes of events, and optimize the squared loss function of CNN. The proposed T-CNN model can accelerate the convergence rate of gradient descent and improve classification accuracy. Experimental results show that, compared with the existing methods, our T-CNN time series classification method has great advantages in efficiency and accuracy.
A time series is the result of a set of sequence data observed for a potential process at a given sampling rate in an equal period1. In recent years, with the development of computer technology, time series have become more and more widely applied, such as disaster monitoring, safety analysis, climate change, stock funds and other fields. How to classify time series2,3,4 has been a difficult point in the field of data mining. For example, mine disaster monitoring and early warning systems use microseismic sensors deployed around mines to store real-time data. These mine disaster events usually last from a few seconds to more than 10 s. Classifying these events helps to summarize the feature of various types of disasters5, which is of great significance for data analysis and disaster prevention.
The existing time series classification methods mostly use symbolic aggregation approximation (SAX)6 and convolution neural network (CNN)7, 8, but ignore the time attribute and the classification accuracy is not high. Therefore, to solve the above problems, this paper proposes a T-CNN time series classification method based on the Gram matrix9. CNN network replaces large-size convolution kernels by cascading multiple small-size convolution kernels, so that neurons of multiple small convolution kernels and one large convolution kernel have the same perception, which not only effectively reduces the number of network parameters, but also increases the nonlinear transformation and network depth of the network. It can extract more complex and abstract advanced features and improve the accuracy of model classification. The proposed method preserves the time attribute, and improves the square loss function10 of the CNN model in the full connection layer to enhance classification accuracy and efficiency. The main contributions of this paper are as follows:
Aiming at the problem of loss of time attribute after transformation of time series matrix, a transformation method based on the Gram matrix is proposed, which converts time series into time-domain images without loss, and uses the wavelet threshold denoising method to filter out normal background noise11, 12.
In order to increase the network convergence rate, a Toeplitz convolution kernel matrix13 is introduced into the CNN convolution layer, and the product of the Toeplitz matrix is used to replace the traditional convolution operation.
A T-CNN model is proposed, which introduces a triplet network14 in the fully connected layer to calculate the difference function between the same class and different classes, and optimizes the square loss function of the CNN model, to improve the classification accuracy.
At present, researchers have studied the time series classification and obtained some research achievements.
Zhong et al.15 proposed SAX, which divided the series into segments and converted each segment into a corresponding letter for classification. This method used the idea of aggregation to effectively reduce the dimension and increase the classification efficiency. However, this method lost a large number of data points in the time series, and the classification accuracy of this method was not high. Xing et al.16 proposed a trend turning point (TTP) method, which extracted the trend feature of the time series itself and required statistical analysis of the extreme points and inflection points of each time series. This method had a good effect on the data sets with specific rules, but was not good for the data sets with large trend fluctuations.
Marion et al.17 proposed a time domain distance (TDD) method, which reflected the similarity by calculating the Euclidean distance between different sequences. The closer the distance, the higher the similarity. The classification speed of this method was fast, but the classification accuracy was low. Marco et al.18 proposed an interval classification method, which divided time series into equal-length intervals, calculated the mean value and standard deviation of each interval, and used support vector machines for classification. This method regarded time series as a set of discrete data points and lost the time attribute.
Francisco et al.19 proposed a Shapelet method to find the Shapelet from the time series of known class labels to reflect the corresponding classification features. This method has high classification accuracy, but the time complexity of Shapelet extraction is high, and the classification efficiency is low. Chen et al.20 proposed a classification method based on convolution neural networks, which used CNNs to effectively classify time series. However, the training process of this method was complicated, and did not consider the time attribute.
To solve the above problems, this paper proposes a T-CNN time series classification method based on the Gram matrix, which converts time series into time-domain images without loss and retains time attributes. The proposed T-CNN model based on convolution neural networks inputs the time-domain images into the model, improving the classification accuracy.
Time series events are not one or a few abnormal perception data points, but a series of discrete data points that meet the threshold range in the time domain, and the same class events have similar features.
Definition 1 Time series events: A collection of continuous abnormal data initiated by the first outlier in the time series that exceeds the threshold range and lasts for a period of time. It can be represented as:
where (e_{i}) represents an outlier in an event, and (phi) denotes the given threshold range.
According to Definition 1, each time series event has time attributes. Aiming at the lack of time attribute in the existing time series classification methods, we introduce the Gram matrix conversion method to realize the lossless conversion of time series.
After Gram matrix conversion, the time-domain image histogram presents normal distribution. The existence of normal noise directly affects the conversion of images in the time domain. Therefore, before the Gram matrix is used to convert the time series, the data is preprocessed first, and then the wavelet threshold denoising method is used to filter out the normal background noise carried by the data.
The steps of the wavelet threshold denoising method are as follows:
Wavelet decomposition of signals. A wavelet is selected to determine the S-level wavelet decomposition, and then the S-level wavelet decomposition calculation is carried out for the signals.
Threshold quantization of high-frequency coefficients in wavelet decomposition. From layer 1 to layer S, a threshold is selected for the high-frequency coefficients of each layer for threshold quantization. The threshold function used in this paper is calculated by:
where (sigma = M/0.6745) denotes the median value of the absolute value of the first-level wavelet decomposition coefficient. Because the gaussian distribution function P(|x|< 0.6745σ) is approximately equal to 0.5, therefore, 0.6745 is the adjustment coefficient of the Gaussian noise standard deviation, and is a fixed value. N denotes the length of the signal.
Wavelet reconstruction of signals. According to the low-frequency coefficients of the S layer and the quantized high-frequency coefficients of the first layer to the S-th layer of the wavelet decomposition, the signal is reconstructed by wavelet.
Similar time series events have the same features, but the locations and intensities of the events are different, and the perceived event data are not at the same scale. Therefore, the normalization equation is used to keep the weight of the data feature dimension on the objective function consistent. The data normalization equation is as follows:
where x denotes the data to be normalized, x´ denotes the normalized result, xmin represents the minimum value in the time series data, and xmax represents the maximum value in the time series data. Through data normalization conversion, all data can be normalized to [0,1]. For example, Fig. 1 shows a normalized time series T´ = {t1´, t2´, t3´,…, tn´} calculated by Eq. (3), the abscissa is time t and the ordinate is threshold Tv.
Time series.
The time attribute is an important attribute to determine the classification. Due to the different time attributes, different regular distributions are presented in chronological order. Therefore, this paper proposes a time-domain image conversion based on the Gram matrix. This method can retain the time attributes and convert a time series into an N × N two-dimensional matrix without loss.
The Gram matrix G is a matrix composed of the inner product of each pair of vectors in the form of Eq. (4):
where < ai, aj > denotes the inner product of two vectors.
The matrix G is a positive definite matrix. The proof of G is as follows:
Let (Z = (x_{1} ,x_{2} ,…x_{n} )^{T} ne 0), then
Thus,
where Z´GZ represents a positive definite quadratic form, and G denotes a positive definite matrix. A positive definite matrix can retain the feature of the matrix by calculating the eigenvalues. After converting the time series by the Gram matrix, the time attributes can be retained. The result of conversion to Gram matrix form is shown in Eq. (5):
where Gt denotes the time series after Gram matrix conversion, < ti, tj > is the inner product pair of the time series, and n is the length of the time series. An inner product represents the correlation between two points, Gt a real symmetric matrix. From left to right of the first row and from top to bottom of the first column in the Gt, the correlation between the first point and the subsequent points of the time series increases with time. Similarly, the second row and the second column are the correlation between the second point and the subsequent points. Therefore, the Gt matrix from the upper left to the low right represents the order of correlation arrangement between two points with time increasing. The time attribute of the time series is retained in the Gram matrix.
Since the time series T is one-dimensional, and in the rectangular coordinate system, the inner product pair calculation requires two-dimensional information of the abscissa and the ordinate. Therefore, to better preserve the time attribute of the time series, we use the polar coordinate system to calculate the inner product pairs of the time series.
The time series T can be encoded into polar coordinates by Eqs. (6 and 7):
where (t_{i}^{^{prime}}) denotes the normalized time series, i is the time stamp in the time series, n is the length of the time series, (theta_{i}) is the angle of a point in the time series in the polar coordinate system, and (r_{i}) is the radius of a point in the time series in the polar coordinate system. Thus, each point in the time series can be represented by Eqs. (6 and 7).
Figure 2 shows the polar coordinate coding of T´ in Fig. 1. The encoding process for 0.5 is:
Polar coordinates of time series.
According to Eq. (6), arccos(0.5) = 60º; i = 3; N = 5; according to Eq. (7), the radius is 0.6. Thus, (0.6, 60º) is the coding result.
As time goes on, the radius of the point becomes larger and larger, and gradually moves away from the center of the circle. Polar coding can completely preserve the time attribute by increasing the radius, and the numerical change can be represented by the angle change. Therefore, as shown in Eq. (8), a new Gram matrix can be obtained by using the polar coordinate angle relationship between two points in the time series:
where (cos (theta_{i} + theta_{j} ) = cos theta_{i} cos theta_{j} – sin theta_{i} sin theta_{j} = t_{i} times t_{j} – sqrt {1 – t_{i}^{2} } times sqrt {1 – t_{j}^{2} }), and (theta_{i}) denotes the angle obtained by Eq. (6). When i = j is formed the diagonals (cos (theta_{i} + theta_{i} ) = 2t_{i}^{2} – 1) of the Gt matrix, and the diagonal of the Gt matrix is arranged in order.
As the position of the Gram matrix moves from the upper left corner to the lower right corner, the time series values are arranged into the matrix sequentially. In other words, we preserve the time attribute of the time series, and encode the time dimension into the geometric structure of the matrix. Each value of the matrix is equivalent to the pixel of the image, and each time series is converted into a time-domain image by the Gram matrix.
As mentioned above, time series are converted to Gram time-domain images, and the Gram time-domain images are used as the input matrix of convolutional neural networks for classification. In order to solve the problems of complex computation and slow training speed of convolutional neural networks, we propose a method based on the Toeplitz matrix product to replace the convolution operation of the convolution layer, and introduce the idea of triplet network into the loss function to improve the efficiency and accuracy of classification.
The convolution operation based on the Toeplitz matrix product is shown in Fig. 3. In Fig. 3, the dark blue square represents the convolution kernel, and the light blue square represents the matrix being convoluted. The convolution kernel is 2 × 2, the unconvoluted matrix is 3 × 3, and the step size is 1. The traditional convolution is shown in the upper part of Fig. 3. The convolution kernel moves successively on the matrix to be convoluted according to the step size of 1, and it requires four traversals of the complete matrix to be convoluted. After each traversal, the convolution kernel and the matrix part with its repeated sum are multiplied and accumulated, and the obtained value is the local convolution result at the corresponding position. Since the traditional convolution needs to traverse the whole image, the computational complexity is high.
Convolutional computation.
As shown in the lower part of Fig. 3, based on the Toeplitz matrix product convolution, each 3 × 3 process matrix obtained by the convolution kernel traversal matrix is expanded in line with row order to obtain a 4 × 1 × 9 row matrix, forming a large matrix H. Then, the convolution matrix is expanded into a 9 × 1 column vector X in line with row arrangement order. The product of the large matrix H constructed by the convolution kernel and the column vector X to be constructed by the convolution matrix effectively replaces the convolution computation. Specifically, the convolution kernel matrix H is composed of 6 small matrices, which are respectively the matrix in the red box and the matrix in the yellow box, as well as the zero matrix in the two white parts.
In Fig. 3, the matrix in the red box conforms to the definition form of the Toeplitz matrix. Similarly, the matrix in the yellow box and the zero matrix are Toeplitz matrices. Therefore, the convolution kernel matrix constructed is a large Toeplitz matrix composed of several small Toeplitz matrices. The product of the Toeplitz matrix is used to replace the traditional convolution operation. The convolution kernel is directly constructed into the convolution kernel matrix without traversing the image in order of step size, and the product of the two matrices is calculated to reduce the computational complexity.
Definition 2 Toeplitz matrix: A matrix with the same elements on each diagonal line from the top left to the bottom right is a Toeplitz matrix, which has the properties (A_{i,j} = A_{i + 1,j + 1} = a_{i – j}). Mathematically,
In order to replace the convolution calculation with the Toeplitz matrix multiplication operation, the convolution kernel matrix H is constructed as the Toeplitz convolution kernel matrix Ht. Given any convolution kernel matrix as follow:
The corresponding construction steps of the Toeplitz convolution kernel matrix are as follows:
A small Toeplitz matrix is generated from each row element of the convolution kernel matrix. Since the size of the convolution kernel matrix is C × D, the convolution kernel matrix H is divided into C Toeplitz matrices: H0, H1, H2, H3, …, Hc-1, where H0 is the zero interpolation of the element h11 in the first row and first column of H, the number of inserted zeros is the number of columns in the convolution kernel matrix H minus 1, and the interpolation result is taken as the first row of H0. Then h12 is interpolated as the second row according to the properties of the Toeplitz matrix until the 2 × D-1 rows are formed and the H0 construction is completed. By analogy, Hi is the matrix obtained by interpolating the (i – 1) row elements of H. For example, the convolution kernel matrix is (H = left[ {begin{array}{*{20}c} 1 & 2 \ 3 & 4 \ end{array} } right]), then H is divided into two matrices (H_{0} = left[ {begin{array}{*{20}c} 1 & 0 \ 2 & 1 \ 0 & 2 \ end{array} } right]) and (H_{1} = left[ {begin{array}{*{20}c} 3 & 0 \ 4 & 3 \ 0 & 4 \ end{array} } right]).
The small Toeplitz matrix obtained in Step (1) is formed into a large Toeplitz matrix:
In the example in Step (1), (H_{t} = left[ {begin{array}{*{20}c} {H_{0} } & 0 \ {H_{1} } & {H_{0} } \ 0 & {H_{1} } \ end{array} } right]) is obtained by Eq. (11), where 0 represents a zero matrix of 3 × 2.
After obtaining the Toeplitz convolution kernel matrix from “Toeplitz convolution kernel matrix construction” section 8, the traditional convolution can be replaced by the Toeplitz matrix multiplication using Eq. (12).
where (X = left( {begin{array}{*{20}c} {x_{11} } & {x_{12} } & cdots & {x_{1B} } \ {x_{21} } & {x_{22} } & cdots & {x_{2B} } \ vdots & vdots & vdots & vdots \ {x_{A1} } & {x_{A2} } & cdots & {x_{AB} } \ end{array} } right)) denotes the matrix to be convolved, (H = left( {begin{array}{*{20}c} {h_{11} } & {h_{12} } & cdots & {h_{1D} } \ {h_{21} } & {h_{22} } & cdots & {h_{2D} } \ vdots & vdots & vdots & vdots \ {h_{C1} } & {h_{C2} } & cdots & {h_{CD} } \ end{array} } right)) denotes the convolution kernel, Ht is the Toeplitz convolution kernel matrix in “Toeplitz convolution kernel matrix construction” section, and XT is the column vector obtained by arranging all the elements of X in row order. Using the full convolution method, the matrix to be convolved is filled with zeros, and the result returns all the data after convolution. The row number of the convolution result matrix is M = A + C − 1 and the column number of the convolution result matrix is N = B + D − 1.
For example, when (X = left[ {begin{array}{*{20}c} 5 & 6 \ 7 & 8 \ end{array} } right]), then (X_{T} = left[ {begin{array}{*{20}c} 5 & 6 & 7 & 8 \ end{array} } right]^{T}), and the results that use convolution calculation is (X*H = left[ {begin{array}{*{20}c} 5 & 6 \ 7 & 8 \ end{array} } right]*left[ {begin{array}{*{20}c} 1 & 2 \ 3 & 4 \ end{array} } right] = left[ {begin{array}{*{20}c} 5 & {16} & {12} \ {22} & {60} & {40} \ {21} & {52} & {32} \ end{array} } right]). The result that uses convolution operation based on the Toeplitz matrix is.
(H_{t} times X_{T} = left[ {begin{array}{*{20}c} {H_{0} } & 0 \ {H_{1} } & {H_{0} } \ 0 & {H_{1} } \ end{array} } right] times left[ {begin{array}{*{20}c} 5 & 6 & 7 & 8 \ end{array} } right]^{T} = left[ {begin{array}{*{20}c} 5 & {16} & {12} & {22} & {60} & {40} & {21} & {52} & {32} \ end{array} } right]^{T}).
Then the calculated column vector is rewritten into a 3 × 3 matrix according to M = A + C − 1 = 3 and N = B + D − 1 = 3, which is the same as the results of the convolution calculation.
We use the Toeplitz matrix multiplication to effectively replace the convolution operation. In terms of time complexity, the input time-domain image size is A × B, and the convolution kernel size is C × D. The convolution operation requires the convolution kernel to continuously traverse the time domain image and calculate A × B × C × D times multiplication.
When using the Toeplitz matrix multiplication, it only needs to calculate the matrix multiplication once. It is realized from Fig. 3 that there are a large number of zeros in each row of the matrix which does not need to be calculated. Thus, the actual calculation of each row is C × D, the row number is the time of convolution kernel traverses, and approximately multiply A × B × C × D times. Therefore, during a calculation, the calculation amount of the two methods is roughly the same. However, when a new time-domain image is inputted into the traditional convolution each time, there are a large number of shift operations in the calculation, which greatly increases the calculation time.
Although it takes some time to construct the Toeplitz matrix, Toeplitz matrix multiplication only needs to construct the corresponding Toeplitz matrix once according to the given convolution kernel, and then can directly perform the matrix multiply calculation on all the input time-domain images to obtain the convolution result. In this way, for the datasets with a large number of sample sets and test sets, the convolution operation time will be greatly reduced.
When a CNN model is used for classification, its fully connected layers perform convergence operations, and a given loss function is required. In this paper, the Triplet network is introduced into the loss function, and then the T-CNN model is proposed.
Let the sample set of m samples is (left{ {left( {x^{left( 1 right)} ,y^{left( 1 right)} } right),left( {x^{left( 2 right)} ,y^{left( 2 right)} } right),…,left( {x^{left( m right)} ,y^{left( m right)} } right)} right}), there are n classes in these samples, which (y^{left( i right)}) represents the expected output of (x^{left( i right)}), and the loss function of CNNs is shown as Eq. (13):
where (omega) is the weight of each neuron, (b) is the bias, and (p_{omega ,b} left( {x^{i} } right)) is the actual output of the sample. The CNN model continuously adjusts the parameter (omega) and (b) by training to minimize (Rleft( {omega ,b} right)). Equation (13) is the square loss function of the traditional convolutional neural network model, which only considers the category of the image itself and does not consider the differences between different categories. Therefore, we will improve it later.
The CNN uses the gradient descent method to adjust the parameter (R(omega ,b)), as shown in Eqs. (14) and (15):
where a is the learning rate and (Rleft( {omega ,b} right)) is the CNN loss function. Equations (14) and (15) are used to update the values of network parameters (omega) and (b). The calculation method is the gradient descent method. In other words, the value of (omega) and (b) can be obtained when the derivative of the loss function is 0.
In order to improve the classification accuracy, the Triplet network is introduced into the CNN loss function for constraint, and a T-CNN model based on the Triplet loss function is proposed. The idea of the T-CNN model is to input three-time domain images at a time, two of which belong to the same class and one belongs to another class. The T-CNN model can obtain the feature of the time domain images by training and can obtain the feature difference function (L_{1}) of two-time domain images from the same class and the feature difference function (L_{2} ) of two-time domain images from different classes. Then (L_{1} ) and (L_{2} ) are used to adjust the parameters of the T-CNN model. (L_{1} ) and (L_{2}) are shown in Eqs. (16 )and (17) respectively:
where (p_{omega ,b}^{{left( {l_{i} } right)}}) is the output value of the same class and (n_{omega ,b}^{left( l right)}) is the output value of the different classes. The image feature difference functions are shown in the adjustment Eq. (18).
where (gamma) represents the minimum distance of the difference function between different classes and between classes (set to 0.1 in this paper). In the experiment of this paper, the comparison experiment was conducted by changing the value of (gamma), and the value of (gamma) was 0.01, 0.05, 0.1, 0.2 and 0.5 respectively. The experiment found that 0.1 was the best experimental result. In each reverse iteration, LT gradually approaches zero. As shown in Fig. 4, when the feature difference function L1 of the same class of images is greater than the feature difference function L2 of different classes of images minus the parameter α, LT is greater than zero, and the model is adjusted in reverse to make L1 smaller and L2 larger. Reference 21 has verified that the Triplet loss function can make samples of the same kind close to each other and samples of different kinds far from each other.
Adjustment chart of feature difference. (It is used to illustrate the triple Loss function proposed in this paper. The software is Microsoft Office Visio).
In Fig. 4, A and P belong to the same class, while N does not belong to the same class as A and P. Before the adjustment, the distance between A and P is greater than that between A and N, and the difference function LT is greater than zero. Thus, the model parameters need to be adjusted in reverse. After the adjustment, the distance between A and N becomes larger, while the distance between A and P becomes smaller.
According to Eqs. (16) and (17), in each reverse iteration, it can be seen that L1 will make the feature difference of the same class smaller, while L2 will make the feature difference of different classes larger. On this basis, a Triplet loss function is proposed as shown in Eq. (19):
where (Rleft( {omega ,b} right)) denotes the CNN square loss function, (alpha) and (beta) are the weight proportion coefficients greater than zero. In the experiment, we tested the values of (alpha) and (beta). The values of (alpha) were 0.1, 0.01, 0.3, 0.4 and so on, and the values of (beta) were 0.9, 0.99, 0.7, 0.6 and so on. After several experiments, it was found that the values of (alpha) and (beta) were 0.4, 0.6 respectively, and the experimental effect was the best. L1 is the feature difference function of the same class, and L2 is the feature difference function of different classes. Therefore, the new residual error of each layer by the backpropagation algorithm is as follows:
The T-CNN model based on the Triplet network adds the feature difference function between the same class and the feature difference function between different classes into a cross-entropy loss function, which is conducive to allow the parameters to extract features with larger differences more quickly in the process of parameter weight adjustment. The partial derivative of (Lleft( {omega ,b} right)) can make the backpropagation residual calculation to obtain new parameters (omega) and (b). Each iteration is more inclined to the direction of gradient descent, which can make the model converge faster and improve the classification efficiency.
The T-CNN model structure used in this paper is 5 × 5 convolution of 128 neurons in the first layer, 5 × 5 convolution of 128 neurons in the second layer, maximum pooling layer of 2 × 2 in the third layer, 3 × 3 convolution of 256 neurons in the fourth layer, 3 × 3 convolution of 256 neurons in the fifth layer, maximum pooling layer of 2 × 2 in the sixth layer, 1024 neurons in the full connection layer in the seventh layer. The loss function is the Triplet-based loss function, the activation function is the sigmoid function, and the value range of the function is (0,1). Figure 5 is the model structure. T-CNN time series classification algorithm is shown in Algorithm 1.
The structure of the T-CNN classification model.
The experimental data set adopts 6 data sets of UCR2018 and microquake data sets, which contains three types of time series event waveforms of mine microseismic signals. The dataset includes time series data of different sizes, lengths and categories. The size of the training set accounts for 60% of the total data set, and the size of the test set accounts for 40% of the total data set, and the model training framework is Tensorflow. The hardware and software environment of the experiment are shown in Table 1.
The experiment data sets are shown in Table 2. After applying the wavelet denoising method in this paper, the type and length of the microquake dataset containing noise will not change, but the length itself is unequal, so it will not affect the structure of the dataset.
In order to prevent local over convergence, the convolutional neural network structure is equipped with two convolutional layers. The convolutional layer 1 uses a convolution kernel of 5 × 5 size, and the maximum pooling window size is 3 × 3. The convolutional layer 2 uses a convolution kernel of 3 × 3 size, and the maximum pooling window size is 2 × 2. The convolutional layer 2 is followed by a standard fully connected layer. The activation function of this layer uses ReLU which makes the CNN model converge quickly. The loss function uses the Triplet loss function. The time-domain image output by the T-CNN model is p = {p1, p2, p3}, where p1, p2, and p3 represent the probabilities of three classes in the data set respectively. class = T(max(p)) is used for judgment classes, where the function T is a threshold function, and the threshold is set to 0.8 after repeated experiments. When max(p) is greater than the threshold set by the function T, the class of time-domain image is output. In order to make the experimental results more reliable, the experiment uses a tenfold cross validation, and the final result is an average of 10 times. The T-CNN model adjusts itself to the optimal state through continuous forward conduction, in which the main adjustable parameters are the learning rate and the iteration times. During this experiment, the learning rate is set to 0.005, and the iteration time is adjusted.
The accuracy is the proportion of the number of samples correctly classified to the total number of samples. Figure 6 is the comparison of classification accuracy by eight different classification models. The compared methods are Transformer, symbolic aggregation approximation (SAX), Shapelet, Dynamic Time Warping (DTW), Collective of Transformation-based Ensembles (COTE), Complexity Invariant distance (CID) and CNN classification method.
Comparison of classification accuracy.
The experimental comparison results of classification accuracy under different data sets are shown in Table 3.
From Fig. 6 and Table 3, it is known that the T-CNN model converts the time series into the time domain images by using the Gram matrix, it can completely retain the time feature of the time series, and its classification accuracy is significantly better than other methods.
The precision is the proportion of samples predicted to be one class that is truly in that class, which is given by:
where (n_{ij}) is the number of samples predicted as the j-th class in class i, and (n_{c}) is the number of sample classes. Then calculate the average precision of all classes to get the average precision of the classification. Figure 7 is the comparison of classification precision.
Comparison of classification precision.
The experimental comparison results of classification precision under different data sets are shown in Table 4.
From Fig. 7 and Table 4, due to the improved loss function of the T-CNN model, the classification precision is significantly better than other methods.
The recall is the proportion of samples detected in one class, which is given by:
where (n_{ij}) is the number of samples predicted as the j-th class in class i, and (n_{c}) is the number of sample classes. Then calculate the average recall of all classes to get the average recall of the classification. Figure 8 is the comparison of classification recall.
Comparison of classification recall.
The experimental comparison results of classification recall under different data sets are shown in Table 5.
From Fig. 8 and Table 5, the T-CNN model uses a Gram matrix to convert the time series without loss and improves the loss function, therefore, the classification recall is close to that of the COTE method and superior to other methods.
In this section, the accuracy of convolution based on the Toeplitz matrix product and traditional convolution is compared. In the experimental process, the size convolution kernel is taken to calculate the convolution, and the time series length is successively intercepted from 10 to 100. The time domain image size transformed based on the Gram matrix is from 10*10 to 100*100. A comparison of the accuracy of Toeplitz convolution and traditional convolution for convolution of time-domain images of different sizes is shown in Fig. 9. The abscissa is the number of iterations, and the ordinate is the accuracy.
Comparison of convolution running time.
From Fig. 9, after several experiments, after 70 iterations, the training set can achieve a good effect, and the accuracy on the test set reaches 90% and tends to be stable. the accuracy of both traditional convolution and Toeplitz convolution changes gently between the image size of 10 and 40 in the time domain. When the image size of more than 40 in the time domain, the accuracy of the two convolutions increases faster. On the whole, when the image size is between 10*10 and 100*100, the accuracy of Toeplitz convolution is higher than that of traditional convolution operation, so the convergence of Toeplitz convolution is faster. Therefore, the Toeplitz convolution proposed in this paper can speed up the training of the model.
This paper proposes a T-CNN time series classification method based on the Gram matrix. The method firstly denoises the time series by using a wavelet threshold, and then converts the loss-free time series into time images by the Gram matrix, time attribute is retained to improve classification accuracy. And on this basis, the T-CNN model is proposed to introduce the Toeplitz convolutional kernel matrix into the convolution layer, replaces the convolutional operation with the multiplication of two matrices, and introduces the feature difference function of the same class and different classes to improve the CNN loss function of the fully connected layer, and the classification efficiency is improved. Experimental results show that the proposed T-CNN time series classification method is suitable for processing time series classification, and the classification accuracy, precision and recall are better than the existing methods.
Mori, U., Mendiburu, A., Miranda, I. M. & Lozano, J. A. Early classification of time series using multi-objective optimization techniques. Inform. Sci. 492, 204–218 (2019).
MathSciNet  Article  Google Scholar 
Zhao, C. et al. Rapid shapelet discovery algorithm for fusion selection extraction and subclass clustering. J. Softw. 31(03), 763–777 (2020).
MATH  Google Scholar 
Darabi, H. et al. Guest editorial for special issue on time series classification. IEEE/CAA J. Autom. Sin. 6(06), 1291–1292 (2019).
MathSciNet  Article  Google Scholar 
Li, G. L., Yan, W. H. & Wu, Z. D. Discovering shapelets with key points in time series classification. Expert Syst. Appl. 132, 76–86 (2019).
Article  Google Scholar 
Luo, H. et al. Multistage identification method for real-time abnormal events of streaming data. Int. J. Distrib. Sens. Netw. 15(12), 1550147719894544 (2019).
Article  Google Scholar 
Yahyaoui, H. & Al-Daihani, R. A novel trend based SAX reduction technique for time series. Expert Syst. Appl. 130, 113 (2019).
Article  Google Scholar 
Zhou, F. Y., Jin, L. P. & Dong, J. A review of convolutional neural networks. Chin. J. Comput. 40(06), 1229–1251 (2017).
MathSciNet  Google Scholar 
Zhao, B. D. et al. Convolutional neural networks for time series classification. J. Syst. Eng. Electron. 28(01), 162–169 (2017).
CAS  Article  Google Scholar 
Wang, W. G., Yang, P. & Jiang, Y. L. Reduction method of bilateral H2 optimal model based on cross gram matrix. Comput. Eng. Sci. 12, 2203–2209 (2017).
Google Scholar 
Zhao, J. P. et al. A coupled convolutional neural network for small and densely clustered ship detection in SAR images. Sci. China 62(04), 111–126 (2019).
MathSciNet  Google Scholar 
Zhou, X. C. et al. A wavelet transform image denoise method based on curvature variational regularization. Acta Electron. Sin. 46(03), 621–628 (2018).
Google Scholar 
Bayer, F. M., Kozakevicius, A. J. & Cintra, R. J. An iterative wavelet threshold for signal denoising. Signal Process. 162, 10–20 (2019).
Article  Google Scholar 
Zheng, Y. P., Shon, S. G. & Fu, Z. W. A gohberg-semencul type formula for the inverse of conjugate-Toeplitz matrix and applications. Acta Math. Appl. Sin. 34(02), 293–303 (2018).
MathSciNet  Article  Google Scholar 
He, G. C. & Liu, X. B. Unsupervised visual representation learning based on image triplet mining. Chin. J. Comput. 41(12), 2787–2803 (2018).
MathSciNet  Google Scholar 
Zhong, Q. L. & Cai, Z. X. An algorithm for symbolization of temporal data based on statistical features. Chin. J. Comput. 10, 1857–1864 (2018).
Google Scholar 
Xing, J. et al. Data trend turning point extraction algorithm for time series. Comput. Eng. 44(01), 56–61 (2018).
Google Scholar 
Morel, M., Achard, C., Kulpa, R. & Dubuisson, S. Time-series averaging using constrained dynamic time warping with tolerance. Pattern Recognit. 74, 77–89 (2018).
ADS  Article  Google Scholar 
Villegas, M. A., Pedregal, D. J. & Trapero, J. R. A support vector machine for model selection in demand forecasting applications. Comput. Ind. Eng. 121, 1–7 (2018).
Article  Google Scholar 
Baldán, F. J. & Benítez, J. M. Distributed fastShapelet transform: A Big Data time series classification algorithm. Inf. Sci. 496, 451–463 (2019).
Article  Google Scholar 
Chen, W. & Shi, K. A deep learning framework for time series classification using relative position matrix and convolutional neural network. Neurocomputing 359, 384–394 (2019).
Article  Google Scholar 
Schroff, F., Kalenichenko, D. & Philbin, J. FaceNet: A unified embedding for face recognition and clustering. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) 2015, 815–823 (2015).
Google Scholar 
Download references
This work was funded by the National Key R&D Program of China (No.2021YFF0901004), the National Natural Science Foundation of China (No.62072220, 61502215), the Natural Science Foundation of Liaoning Province of China in 2022(No.191), the Major Science and Technology Plan of Liaoning Province of China in 2022(No.28), the Scientific Research Project of the Educational Department of Liaoning Province (No.LJKZ0094), the Central Government Guides Local Science and Technology Development Foundation Project of Liaoning Province (No.2022JH6/100100032).
School of Information, Liaoning University, Shenyang, 110036, China
Junlu Wang, Su Li, Wanting Ji, Tian Jiang & Baoyan Song
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
J.W., S.L., W.J., T.J. and B.S. wrote the main manuscript text . All authors reviewed the manuscript.
Correspondence to Baoyan Song.
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Reprints and Permissions
Wang, J., Li, S., Ji, W. et al. A T-CNN time series classification method based on Gram matrix. Sci Rep 12, 15731 (2022). https://doi.org/10.1038/s41598-022-19758-5
Download citation
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-022-19758-5
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
Advertisement
Advanced search
Scientific Reports (Sci Rep) ISSN 2045-2322 (online)
© 2022 Springer Nature Limited
Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

source

Leave a Comment