Essay on Multiscale Principal Component Analysis (MSPCA) and Neural Networks

Multiscale Principal Component Analysis (MSPCA)

Is your time best spent reading someone else’s essay? Get a 100% original essay FROM A CERTIFIED WRITER!

Order now

Multiscale Principal Component Analysis (MSPCA) combines the features of PCA on approximations and provides the details in the wavelet domain in addition to a final PCA. MSPCA computes principal components through maximization of the sum of the pairwise distances that exist between data projections for only those pair of data points at distances within the preferred scale. The Multiscale nature of MSPCA makes it the most suitable method of modelling data that varies considerably over time and frequency. Process monitoring involves combining the PCA models at important scales in a scale-recursive manner so as to come up with the model for all the scales together. This approach is essentially similar to adaptively filtering the values of the scores and residuals using a filter that is efficient in separating the deterministic changes from normal process variation. In this approach, there is no need to use time-series modelling or matrix augmentation to monitor autocorrelated measurements hence making it effective. Apart from MSPCA offering an efficient method of detecting deterministic changes, it also makes it possible to extract features that promote abnormal operation, without the need of any pre-filtering.

MSPCA is also popular for multivariate extraction of empirical models from multiscale data. Since the quality of data determines the quality of empirical models extracted, it is important to have efficient methods to ensure quality results after the extraction process. This is usually achieved by preprocessing the variables by either filtering or compression which eliminates the less important components thus reducing the undesired effects of errors on the measured data. Thus, multiscale methods offer quality empirical models than other methods since most processes are multivariate in nature. The most efficient way to realize the benefits of multiscale representation is to pre-treat each variable independently using an appropriate filtering technique such as wavelet thresholding. The empirical model is then obtained from the filtered signal. Despite the benefits of this approach, preprocessing of each independent variable is still not efficient since it normally compromises the multivariate nature of the data due to lack of integration of the filtering and modelling tasks. Additionally, some heuristic knowledge may be needed to extract relevant features from the process. Furthermore, the inability of the univariate denoising approach to solve the empirical modelling problem of processes that are multivariate in nature makes it possible to reduce the effect of unwanted signals by using an approach that possesses both multiscale and multivariate features. Latent variable regression methods usually reduce unwanted signals by retaining only those coefficients that relate the variables to each other. Unfortunately, this method does not eliminate the unwanted signals completely since the errors will have already contaminated the scores and loadings retained. Thus, while PCA may reduce the effect of errors on the measured variables, it does not entirely eliminate the errors while capturing the relationship between the variables.

There is a direct correlation between the imbedded error and the real error together with the number of retained components. Thus, when using PCA, the error can be completely eliminated only if no principal components are present in the measured data. However, the latent variable regression method would be useless since it will not capture the relationship between variables. A combination of the latent variable regression model and wavelet thresholding overcomes this drawback as it improves the quality and noise removal ability of a latent variable regression model. The integration of the two approaches increases efficiency since they complement each other in the noise removal technique they use. Latent variable regression model uses the relationship between variables as a way of detecting and removing noise while wavelet thresholding extracts features from the measured variables before removing noise. Several extensions of multiscale empirical omodelling methods make use of the compatibility between latent variable models and wavelet thresholding to come up with different approaches of modelling and analysis of data matrices. This approaches take into consideration the input and output variables and their representation in the wavelet domain as well as the method of selecting the relevant coefficients from the data matrices. The multiscale modelling approach is among the most common types of error-in-variables modelling since it performs the task of error removal and determination of model parameters simultaneously. Multiscale PCA combines wavelet transform and PCA to enable feature extraction of signals in in the temporal and spatial domains. The variables on a selected wavelet are decomposed after which the PCA of the matrix coefficients is computed. PCA decorrelates the relationship between variables while wavelet decomposition decorrelates the relationship between stochastic measurements. When using scale dependent threshold values for non-white noise, or one which is scale-dependent, better removal of noise can be achieved through using a different PCA model for the matrix coefficients at each scale. The process involves choosing a subset of wavelet coefficients and principal components using a thresholding criteria that will depend on the nature of application. For instance, if MSPCA is applied in multivariate noise removal, any univariate thresholding technique can be used to determine the threshold for the latent variables. The number of principal components depends on how the variables are correlated. Since wavelet transform does not alter the correlation of the variables, each scale will have the same number of components. An independent PCA model at each scale implies that MSPCA will combine the results of the different scales at the final step. This final step is made possible through the reconstruction of the signal depending on the given coefficients of each scale followed by computing the PCA of the reconstructed signal and finally eliminating any irrelevant components. This will result in an improvement of noise removal while maintaining orthonormality of the loadings and scores of the reconstructed signal. Thresholding of the latent variables may be used to select the most relevant wavelet coefficients. Alternatively, sorting of the coefficients can be on the basis of their importance using various criteria such as correlation or mutual information between the wavelet coefficients. Despite multiscale modeling by thresholding of the latent variables can offer better performance than other existing methods, it does not integrate the modeling and thresholding steps efficiently. Additionally, it lacks a comprehensive statistical basis. Therefore, proper integration of the two steps needs better solutions of the optimization problems.

Artificial Neural Networks (ANN)

The first wave of interest in artificial neural networks (ANN) emerged when psychologists and neurobiologists sought to model the computational abilities of neurons. In simple terms, ANN can be thought of as an interconnection of input/output units with each of the connections having a specific weight. Learning capability of the network is achieved by adjustments of these weights in accordance with the preferred learning algorithm. ANN learning is also known as connectionist learning because of the connections between the input/output units. Training of artificial neural networks takes a lot of time hence are suitable for applications that can accommodate the long and diverse trainings. They require sufficient representative examples that are usually obtained empirically. A common criticism of artificial neural networks is that they have poor interpretability. For instance, humans may find it quite challenging to interpret the symbolic meanings of the hidden units and the learned weights in the network. At one point, these two features made artificial neural networks undesirable for data mining. Despite the drawbacks of artificial networks, they are still preferable in certain areas due to some of their advantages. The advantages include high tolerance to noise and their ability to model given patterns regardless of the training that they received. Thus, the neural network in scenarios where there is little knowledge of how attributes and classes are related. Additionally, unlike most decision tree algorithms, they handle continuous-valued inputs and outputs very well. There are a wide variety of real-life applications of artificial neural networks including areas such as handwritten character recognition, computer pronunciation of English text, and pathology and laboratory training. Artificial neural network algorithms essentially possess fine-grained parallelism. Therefore, employing efficient parallelization techniques can improve the computational speed. Additionally, recently developed techniques have resulted in an improvement of rule extraction from trained artificial neural networks. All these features make the neural network quite useful in the classification and numeric prediction involved in data mining. Various types of neural networks and neural network exits. The most common neural network algorithm, which gained recognition in the 1980s, is the backpropagation algorithm.

A Multilayer Feed-Forward Neural Network

Multilayer feed-forward neural networks use various learning techniques with the most popular one being the backpropagation algorithm. It iteratively approximates an appropriate set of weights to allow for better prediction of the training dataset. A multilayer feed-forward neural network is made up of three layers with each containing various units. The first layer is the input layer while the last one is the output layer. In between these two layers are the hidden layers. The neural network receives training patterns via the input layer. The inputs are presented simultaneously into the units forming the input layers. The input layer then weighs and communicates the inputs to one or more hidden layers. The output of one hidden layer can be an input to another hidden later. The hidden layers then perform processing by using a system of weighted connections then link to the output layer which then emits the prediction of the given dataset. The input later contains input units while the hidden layer and the output layer contains output units which are at times called neurodes since they have a symbolic biological background. Thus, the network can be considered a two-layer network since the main purpose of the input unit is only to pass the inputs to the next layer. Similarly, a neural network with two hidden layers is referred to as a three-layer network, and so on. A neural network is referred to as a feed forward network when all the data flow in a single direction. The connections are in such a way that each unit provides input to the next unit in the forward layer. Thus, each output unit only receives a weighted sum of outputs from the units in a previous layer. Multilayer feed-forward neural networks can perform nonlinear regression since they are able to model class predictions as nonlinear combinations of inputs. With enough hidden units and training samples, these networks can efficiently approximate any function.

Dening a Network Topology

Network topology is an important factor in the learning and functioning of an artificial neural network. Therefore, for training to be possible, the user must define the network topology by specifying the number of units in each layer and the number of hidden layers if it is more than one. One way of speeding up the learning phase is to normalize the input values for each attribute in the training tuples. Usually, the input values are...