FWER and FDR controlling procedures for multiple correlation tests

Description

The package compiles some multiple testing procedures which theoretically control asymptotically the FWER in the framework of correlation testing. Four tests statistics can be considered: the empirical correlation, the Student statistics, the Fisher's z-transform and the usual Gaussian statistics considering random variables (X_i-mean(X_i))(X_j-mean(X_j)). Four methods are implemented: Bonferroni (1935)'s, Šidák (1967)'s, Romano & Wolf (2005)'s bootstrap and (Drton & Perlman (2007)'s procedure based on the asymptotic distributions of the test statistics, called MaxTinfty. The package also includes some multiple testing procedures which are related to the control of the FDR : Cai & Liu (2016)'s procedures called LCT-N and LCT-B -which have been proven to control the FDR for correlation tests- and Benjamini & Hochberg (1995)'s -which has no theoretical results in correlation testing.

Details

Consider \lbrace \bold{X}_\ell = (X_{1\ell},\dots X_{p\ell}),\; \ell=1,...,n\rbrace a set of n independent and identically distributed R^p-valued random variables. Denote data the array containing \lbrace\mathbf{X_\ell},\; \ell=1,\dots,n\rbrace, with observation indexes l in row. The aim is to test simultaneously

(H_{0ij})~ Cor(X_i,X_j)=0 {~~against~~} (H_{1ij})~ Cor(X_i,X_j)\neq 0,~~ i,j=1,..., p,~ i<j.

Four tests statistics are implemented: the empirical correlation, the Student statistics, the Fisher's z-transform and the usual test statistics on expectancy considering the product of random variables. They are available in function eval_stat. Next, two main types of procedures are available:

Asymptotically FWER controlling procedures:

Bonferroni (1935)'s method, Šidák (1967)'s procedure, Romano & Wolf (2005)'s bootstrap procedure and Drton & Perlman (2007)'s procedure. A description of these methods can be found in Chapter 5 of Roux (2018). To apply these procedures, function ApplyFwerCor can be used as follows:

ApplyFwerCor(data,alpha,stat_test,method), with alpha the desired level of control for FDR and stat_test, method respectively the kind of test statistic and the FDR controlling method. The function returns the list of indexes \lbrace (i,j), i < j \rbrace for which null hypothesis (H_{0ij}) is rejected.

Asymptotically FDR controlling procedures:

Cai & Liu(2016)'s two procedures and Benjamini & Hochberg (1995)'s procedure (with no theoretical proof for the latest). To apply these procedures, use function ApplyFdrCor as follows: ApplyFdrCor(data,alpha,stat_test,method) with alpha the desired level of control for FWER and stat_test, method respectively the kind of test statistic and the FDR controlling method. The function returns the list of indexes \lbrace (i,j), i < j \rbrace for which null hypothesis (H_{0ij}) is rejected.

Functions SimuFwer and SimuFdr provide simulations of Gaussian random variables for a given correlation matrix and return estimated FWER, FDR, Power and true discovery rate obtained applying one of the procedure above. Some example of results obtained can be found in Chapter 6 of Roux (2018).

Author(s)

Irene Gannaz

Maintainer: Irene Gannaz <irene.gannaz@insa-lyon.fr>

References

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society. Series B (Methodological), 289-300, https://doi.org/10.1111/j.2517-6161.1995.tb02031.x.

Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

Cai, T. T., & Liu, W. (2016). Large-scale multiple testing of correlations. Journal of the American Statistical Association, 111(513), 229-240, https://doi.org/10.1080/01621459.2014.999157.

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449, https://doi.org/10.1214/088342307000000113.

Romano, J. P., & Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100(469), 94-108, https://doi.org/10.1198/016214504000000539.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Šidák, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62(318), 626-633.

Examples

# Parameters for simulations
Nsimu  <- 100                # number of Monte-Carlo simulations
seqn   <- seq(100,400,100)   # sample sizes
p      <- 10                 # number of random variables considered
rho    <- 0.3                # value of non-zero correlations
seed   <- 156724
 
corr_theo <- diag(1,p)       # the correlation matrix
corr_theo[1,2:p] <- rho
corr_theo[2:p,1] <- rho               

# Parameters for multiple testing procedure
stat_test <- 'empirical'     # test statistics for correlation tests
method <- 'BootRW'           # FWER controlling procedure
SD <- FALSE                  # logical determining if stepdown is applied
alpha  <- 0.05               # FWER threshold 
Nboot  <- 100                # number of bootstrap or simulated samples

# Simulations and application of the chosen procedure
res <- matrix(0,nrow=length(seqn),ncol=5)
for(i in 1:length(seqn)){
    temp <- SimuFwer(corr_theo,n=seqn[i],Nsimu=Nsimu,alpha=alpha,stat_test=stat_test,
           method='BootRW',Nboot=Nboot,stepdown=SD,seed=seed)
    res[i,] <- temp
}
rownames(res) <- seqn
colnames(res) <- names(temp)

# Display results
par(mfrow=c(1,2))
plot(seqn,res[,'fwer'],type='b',ylim=c(0,max(alpha*1.1,max(res[,'fwer']))),
    main='FWER',ylab='fwer',xlab='number of observations')
plot(seqn,res[,'sensitivity'],type='b',ylim=c(0,1.1),
    main='Power',ylab='sensitivity',xlab='number of observations')

Applies multiple testing procedures built to control (asymptotically) the FDR for correlation testing.

Description

Applies multiple testing procedures built to control (asymptotically) the FDR for correlation testing. Some have no theoretical proofs for tests on a correlation matrix.

Usage

ApplyFdrCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  method = "LCTnorm",
  Nboot = 1000,
  vect = FALSE,
  arr.ind = FALSE
)

Arguments

Value

Returns either

• logicals indicating if the corresponding correlation is significant, as a vector or a matrix depending on vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society. Series B (Methodological), 289-300.

Cai, T. T., & Liu, W. (2016). Large-scale multiple testing of correlations. Journal of the American Statistical Association, 111(513), 229-240.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

See Also

ApplyFwerCor

LCTnorm, LCTboot, BHCor, BHBootCor

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
res <- ApplyFdrCor(data,stat_test='empirical',method='LCTnorm')
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)

Applies multiple testing procedures controlling (asymptotically) the FWER for tests on a correlation matrix.

Description

Applies multiple testing procedures controlling (asymptotically) the FWER for tests on a correlation matrix. Methods are described in Chapter 5 of Roux (2018).

Usage

ApplyFwerCor(
  data,
  alpha = NULL,
  stat_test = "empirical",
  method = "Sidak",
  Nboot = 1000,
  stepdown = TRUE,
  vect = FALSE,
  logical = stepdown,
  arr.ind = FALSE
)

Arguments

Value

Returns either

• the adjusted p-values, as a vector or a matrix, depending on vect

• logicals indicating if the corresponding correlation is significant if logical=TRUE, as a vector or a matrix depending on vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Romano, J. P., & Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100(469), 94-108.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Šidák, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62(318), 626-633.

See Also

ApplyFwerCor_SD, ApplyFdrCor

BonferroniCor, SidakCor, BootRWCor, maxTinftyCor

BonferroniCor_SD, SidakCor_SD, BootRWCor_SD, maxTinftyCor_SD

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
(res <- ApplyFwerCor(data,stat_test='empirical',method='Bonferroni',stepdown=FALSE))
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)

Applies an oracle version of MaxTinfty procedure described in Drton & Perlman (2007) for correlation testing.

Description

Applies oracle MaxTinfty procedure described in Drton & Perlman (2007) which controls asymptotically the FWER for tests on a correlation matrix. It needs the true correlation matrix.

Usage

ApplyFwerCor_oracle(
  data,
  corr_theo,
  alpha = c(),
  stat_test = "empirical",
  method = "MaxTinfty",
  Nboot = 1000,
  stepdown = TRUE,
  vect = FALSE,
  logical = stepdown,
  arr.ind = FALSE
)

Arguments

Value

Returns either

• the adjusted p-values, as a vector or a matrix, depending on vect (unavailable with stepdown)

• logicals indicating if the corresponding correlation is significant if logical=TRUE, as a vector or a matrix depending on vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

Oracle estimation of the quantile is used, based on the true correlation matrix

References

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

See Also

ApplyFwerCor

maxTinftyCor, maxTinftyCor_SD

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values:
(res <- ApplyFwerCor_oracle(data,corr_theo,stat_test='empirical',Nboot=1000,stepdown=FALSE))
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)

Benjamini & Hochberg (1995)'s procedure for correlation testing with bootstrap evaluation of p-values.

Description

Benjamini & Hochberg (1995)'s procedure on the correlation matrix entries with bootstrap evaluation of p-values (no theoretical proof of control).

Usage

BHBootCor(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  Nboot = 100,
  vect = FALSE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• a vector or a matrix of logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, if arr.ind=FALSE,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society. Series B (Methodological), 289-300.

See Also

ApplyFdrCor, BHCor

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BHBootCor(data,alpha,stat_test='empirical',arr.ind=TRUE)

Benjamini & Hochberg (1995)'s procedure for correlation testing.

Description

Benjamini & Hochberg (1995)'s procedure on the correlation matrix entries (no theoretical proof of control).

Usage

BHCor(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  vect = FALSE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society. Series B (Methodological), 289-300.

See Also

ApplyFdrCor, BHBootCor

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BHCor(data,alpha,stat_test='empirical',arr.ind=TRUE)

Bonferroni multiple testing procedure for correlations.

Description

Bonferroni multiple testing procedure for correlations.

Usage

BonferroniCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• the adjusted p-values, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

See Also

ApplyFwerCor, BonferroniCor_SD

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
corr_theo <- diag(1,p)
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- BonferroniCor(data,stat_test='empirical')
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
BonferroniCor(data,alpha,stat_test='empirical',arr.ind=TRUE)

Bonferroni multiple testing method for correlations with stepdown procedure.

Description

Bonferroni multiple testing method for correlations with stepdown procedure.

Usage

BonferroniCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

See Also

ApplyFwerCor, BonferroniCor

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BonferroniCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- BonferroniCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})  

Bootstrap multiple testing method of Romano & Wolf (2005) for correlations.

Description

Multiple testing method based on the evaluation of quantile by bootstrap in the initial dataset (Romano & Wolf (2005)).

Usage

BootRWCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• the adjusted p-values, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Romano, J. P., & Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100(469), 94-108.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

See Also

ApplyFwerCor, BootRWCor_SD

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- BootRWCor(data,stat_test='empirical',Nboot=1000)
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
BootRWCor(data,alpha,stat_test='empirical',Nboot=1000,arr.ind=TRUE)

Boootstrap multiple testing method of Romano & Wolf (2005) for correlations, with stepdown procedure.

Description

Multiple testing method based on the evaluation of quantile by bootstrap in the initial dataset (Romano & Wolf (2005)), with stepdown procedure.

Usage

BootRWCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Romano, J. P., & Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100(469), 94-108.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

See Also

ApplyFwerCor, BootRWCor

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
BootRWCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- BootRWCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})  

Bootstrap procedure LCT-B proposed by Cai & Liu (2016) for correlation testing.

Description

Bootstrap procedure LCT-B proposed by Cai & Liu (2016) for correlation testing.

Usage

LCTboot(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  Nboot = 100,
  vect = FALSE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Cai, T. T., & Liu, W. (2016). Large-scale multiple testing of correlations. Journal of the American Statistical Association, 111(513), 229-240.

See Also

ApplyFdrCor, LCTNorm

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
LCTboot(data,alpha,stat_test='empirical',Nboot=100,arr.ind=TRUE)

Procedure LCT-N proposed by Cai & Liu (2016) for correlation testing.

Description

Procedure LCT-N proposed by Cai & Liu (2016) for correlation testing.

Usage

LCTnorm(
  data,
  alpha = 0.05,
  stat_test = "2nd.order",
  vect = FALSE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Cai, T. T., & Liu, W. (2016). Large-scale multiple testing of correlations. Journal of the American Statistical Association, 111(513), 229-240.

See Also

ApplyFdrCor, LCTboot

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
LCTnorm(data,alpha,stat_test='empirical',arr.ind=TRUE)

Sidak multiple testing procedure for correlations.

Description

Sidak multiple testing procedure for correlations.

Usage

SidakCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• the adjusted p-values, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Šidák, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62(318), 626-633.

See Also

ApplyFwerCor, SidakCor_SD

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- SidakCor(data,stat_test='empirical')
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
SidakCor(data,alpha,stat_test='empirical',arr.ind=TRUE)

Sidak multiple testing method for correlations with stepdown procedure.

Description

Sidak multiple testing method for correlations with stepdown procedure.

Usage

SidakCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Šidák, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62(318), 626-633.

See Also

ApplyFwerCor, SidakCor

Examples

  
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
SidakCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- SidakCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})  

Simulates Gaussian data with a given correlation matrix and applies a FDR controlling procedure on the correlations.

Description

Simulates Gaussian data with a given correlation matrix and applies a FDR controlling procedure on the correlations.

Usage

SimuFdr(
  corr_theo,
  n = 100,
  Nsimu = 1,
  alpha = 0.05,
  stat_test = "empirical",
  method = "LCTnorm",
  Nboot = 1000,
  seed = NULL
)

Arguments

Value

Returns a line vector containing estimated values for fwer, fdr, sensitivity, specificity and accuracy.

References

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society. Series B (Methodological), 289-300.

Cai, T. T., & Liu, W. (2016). Large-scale multiple testing of correlations. Journal of the American Statistical Association, 111(513), 229-240.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

See Also

ApplyFdrCor, SimuFwer

Examples

Nsimu <- 1000
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
alpha <- 0.05
SimuFdr(corr_theo,n,Nsimu,alpha,stat_test='empirical',method='LCTnorm')

Simulates Gaussian data with a given correlation matrix and applies a FWER controlling procedure on the correlations.

Description

Simulates Gaussian data with a given correlation matrix and applies a FWER controlling procedure on the correlations.

Usage

SimuFwer(
  corr_theo,
  n = 100,
  Nsimu = 1,
  alpha = 0.05,
  stat_test = "empirical",
  method = "Sidak",
  Nboot = 1000,
  stepdown = TRUE,
  seed = NULL
)

Arguments

Value

Returns a line vector containing estimated values for fwer, fdr, sensitivity, specificity and accuracy.

References

Bonferroni, C. E. (1935). Il calcolo delle assicurazioni su gruppi di teste. Studi in onore del professore salvatore ortu carboni, 13-60.

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Romano, J. P., & Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. Journal of the American Statistical Association, 100(469), 94-108.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

Westfall, P.H. & Young, S. (1993) Resampling-based multiple testing: Examples and methods for p-value adjustment, John Wiley & Sons, vol. 279.

See Also

ApplyFwerCor, SimuFwer_oracle, SimuFdr

Examples

Nsimu <- 1000 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
alpha <- 0.05
SimuFwer(corr_theo,n,Nsimu,alpha,stat_test='empirical',method='Bonferroni',stepdown=FALSE)

Simulates Gaussian data with a given correlation matrix and applies oracle MaxTinfty on the correlations.

Description

Simulates Gaussian data with a given correlation matrix and applies oracle MaxTinfty (i.e. Drton & Perlman (2007)'s procedure with the true correlation matrix) on the correlations.

Usage

SimuFwer_oracle(
  corr_theo,
  n = 100,
  Nsimu = 1,
  alpha = 0.05,
  stat_test = "empirical",
  method = "MaxTinfty",
  Nboot = 1000,
  stepdown = TRUE,
  seed = NULL
)

Arguments

Value

Returns a line vector containing estimated values for fwer, fdr, sensitivity, specificity and accuracy.

References

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

See Also

ApplyFwerCor_Oracle, SimuFwer

Examples

Nsimu <- 1000
n <- 50
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
alpha <- 0.05
SimuFwer_oracle(corr_theo,n,Nsimu,alpha,stat_test='empirical',stepdown=FALSE,Nboot=100)

Uncorrected testing procedure for correlations.

Description

Uncorrected testing procedure for correlations.

Usage

UncorrectedCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• the non-adjusted p-values, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# p-values
res <- UncorrectedCor(data,stat_test='empirical')
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
UncorrectedCor(data,alpha,stat_test='empirical',arr.ind=TRUE)

Returns the theoretical covariance of empirical correlations.

Description

Returns the theoretical covariance of empirical correlations.

Usage

covD2nd(r)

Arguments

Value

Returns the theoretical covariance of 2nd order statistics, \sqrt{n}*mean(Y)/sd(Y) with Y=(X_i-mean(X_i))(X_j-mean(X_j)).

See Also

covDcor

Examples

p <- 10
corr_theo <- diag(1,p)
corr_theo[2:p,] <- 0.3
corr_theo[,2:p] <- 0.3
covD2nd(corr_theo)

Returns the theoretical covariance of empirical correlations.

Description

Returns the theoretical covariance of empirical correlations.

Usage

covDcor(r)

Arguments

Value

Returns the theoretical covariance of empirical correlations.

References

Aitkin, M. A. (1969). Some tests for correlation matrices. Biometrika, 443-446.

See Also

covDcorNorm

Examples

p <- 10
corr_theo <- diag(1,p)
corr_theo[2:p,] <- 0.3
corr_theo[,2:p] <- 0.3
covDcor(corr_theo)

Returns the theoretical covariance of test statistics for correlation testing.

Description

Returns the theoretical covariance of test statistics for correlation testing.

Usage

covDcorNorm(cor_mat, stat_test = "empirical")

Arguments

Value

Returns the theoretical covariance of the test statistics.

See Also

covDcor, covD2nd, eval_stat

Examples

p <- 10
corr_theo <- diag(1,p)
corr_theo[2:p,] <- 0.3
corr_theo[,2:p] <- 0.3
covDcorNorm(corr_theo,stat_test='student')

Evaluates the test statistics for tests on correlation matrix entries.

Description

Evaluates the test statistics for tests on correlation matrix entries.

Usage

eval_stat(data, type = "empirical")

Arguments

Value

Returns the test statistics for correlation testing.

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
stat <- eval_stat(data,'fisher')

Multiple testing method of Drton & Perlman (2007) for correlations.

Description

Multiple testing method based on the evaluation of quantile by simulation of observations from the asymptotic distribution (Drton & Perlman (2007)).

Usage

maxTinftyCor(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  OmegaChap = covDcorNorm(cor(data), stat_test),
  vect = FALSE,
  logical = FALSE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• the adjusted p-values, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

See Also

ApplyFwerCor, maxTinftyCor_SD

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
# adjusted p-values
res <- maxTinftyCor(data,stat_test='empirical',Nboot=1000)
round(res,2)
# significant correlations with level alpha:
alpha <- 0.05
whichCor(res<alpha)
# directly
res <- maxTinftyCor(data,alpha,stat_test='empirical',Nboot=1000,arr.ind=TRUE)

Multiple testing method of Drton & Perlman (2007) for correlations, with stepdown procedure.

Description

Multiple testing method based on the evaluation of quantile by simulation of observations from the asymptotic distribution (Drton & Perlman (2007)), with stepdown procedure.

Usage

maxTinftyCor_SD(
  data,
  alpha = 0.05,
  stat_test = "empirical",
  Nboot = 1000,
  OmegaChap = covDcorNorm(cor(data), stat_test),
  vect = FALSE,
  logical = TRUE,
  arr.ind = FALSE
)

Arguments

Value

Returns

• logicals, equal to TRUE if the corresponding element of the statistic vector is rejected, as a vector or a matrix depending of the value of vect,

• an array containing indexes \lbrace(i,j),\,i<j\rbrace for which correlation between variables i and j is significant, if arr.ind=TRUE.

References

Drton, M., & Perlman, M. D. (2007). Multiple testing and error control in Gaussian graphical model selection. Statistical Science, 22(3), 430-449.

Roux, M. (2018). Graph inference by multiple testing with application to Neuroimaging, Ph.D., Université Grenoble Alpes, France, https://tel.archives-ouvertes.fr/tel-01971574v1.

See Also

ApplyFwerCor, maxTinftyCor

Examples

 
n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
alpha <- 0.05
# significant correlations:
maxTinftyCor_SD(data,alpha,stat_test='empirical', arr.ind=TRUE)
# successive p-values
res <- maxTinftyCor_SD(data,stat_test='empirical', logical=FALSE)
lapply(res,FUN=function(x){round(x,2)})
# succesive rejections
lapply(res,FUN=function(x){whichCor(x<alpha)})  

Returns an upper-triangle matrix, without the diagonal, containing the elements of a given vector.

Description

Returns an upper-triangle matrix, without the diagonal, containing the elements of a given vector.

Usage

unvectorize(vect)

Arguments

Value

Returns an upper-triangle matrix where each entry is given by the vector containing the upper triangle of a matrix, without the diagonal.

See Also

vectorize

Examples

unvectorize(1:10)

Returns a vector containing the upper triangle of a matrix, without the diagonal.

Description

Returns a vector containing the upper triangle of a matrix, without the diagonal.

Usage

vectorize(mat)

Arguments

Value

Returns a vector containing the upper triangle of a matrix, without the diagonal.

See Also

unvectorize

Examples

vectorize(matrix(1:9,3,3))

Returns the indexes of an upper triangular matrix with logical entries.

Description

Returns the indexes of an upper triangular matrix with logical entries.

Usage

whichCor(mat)

Arguments

Value

Returns the indexes of the upper triangular part where the entries are TRUE

Examples

n <- 100
p <- 10
corr_theo <- diag(1,p)
corr_theo[1,3] <- 0.5
corr_theo[3,1] <- 0.5
data <- MASS::mvrnorm(n,rep(0,p),corr_theo)
res <- ApplyFwerCor(data,stat_test='empirical',method='Bonferroni',stepdown=FALSE)
# significant correlations, level alpha:
alpha <- 0.05
whichCor(res<alpha)