Independence tests in high dimensions: asymptotics, limitations, and enhancements
作者：
时间： 2024-04-26
阅读：
Title: Independence tests in high dimensions: asymptotics, limitations, and enhancements
报告人：张耀武
Abstract: matchMeasuring the nonlinear dependence between random vectors and testing for their statistical independence is a fundamental problem in statistics and has attracted increasing attention in recent years. However, most existing works have focused on either fixed or very high-dimensional covariates. To bridge this gap, we offer statistical insights into the behavior of popular independence measurements, such as DC and HSIC, as the dimensions vary at diverse rates. We first show that, under the null hypothesis, the rescaled measurements converge in distribution to the standard normal distribution. Subsequently, we provide a general condition that ensures the tests based on these measurements possess nontrivial power in high dimensions. By decomposing this condition, we illustrate how the ability of the measurements to detect nonlinear dependence changes with increasing dimensions. Moreover, we demonstrate that, depending on the sample size, the covariate dimensions and the dependence structures within covariates, these measurements can capture different types of associations between random vectors. Finally, to enhance the power of the tests in sparse high-dimensional scenarios, we introduce a cross-validation procedure which incorporates feature screening with the independence tests. Extensive numerical studies are conducted to validate our theoretical results.
报告人简介：
张耀武，上海财经大学教授，主要从事高维复杂模型的统计推断。在统计学领域国际顶级期刊《The Annals of Statistics》、《Biometrika》，经济学顶级期刊《 Journal of Econometrics》、《Journal of Business & Economic Statistics》，以及权威期刊《Journal of Machine Learning Research》上发表研究论文多篇。
