Time and date: 1 February 2023 at 2:00 pm | Location: Abacws 1.04 | Speaker: Ben Jones
In high-dimensional spaces, classical statistical methods often break down for theoretical reasons such as matrix non-invertibility or ill-conditioning. A standard approach to resolving this issue is to reduce the dimension (p) of the data by projecting it onto a d-dimensional (d < p) manifold in some optimal way. Principal Components Analysis (PCA), for example, seeks the subspace spanned by the first d normalised eigenvectors of the covariance matrix of the variables (the matrix whose ijth entry is given by the covariance of the ith and jth variables). It is optimal in the sense that the ith normalised eigenvector maximises the variance of the dot product of some normalised vector with the random variable corresponding to the data, subject to the normalised vector being orthogonal to the previous solutions to the optimisation problem. In a regression context, PCA is often applied to the predictor variable X (which takes values in R^p) before then using the components of its projection in regression for some response variable Y (taking values in R) which we are interested in predicting. The fundamental issue with such an approach, and other unsupervised dimension reduction procedures such as t-SNE and Locally Linear Embedding, is that the components are not guaranteed to retain the predictive power for Y that X has (because Y is not used in the derivation). It is natural then to devise supervised dimension reduction methods, using both X and Y, which have this desired property. The purpose of this talk is to overview a framework, called Sufficient Dimension Reduction (SDR), for a class of such methods. The talk will cover the basic concepts when X is multivariate and Y is univariate, it will cover two of the earlier techniques for this situation which seek linear subspaces, and - time permitting - it will cover a modern measure-theoretic definition of SDR which allows for X and Y to be more general objects and which allows for nonlinear approaches. If, after that, time still permits then there will be a discussion of SDR methods, including one of my own, which are capable of handling when a categorical predictor W is present in addition to X.
