A string specifying the likelihood function (distribution) of the response variable. Available options:

• "gaussian"

• "bernoulli_probit": Binary data with Bernoulli likelihood and a probit link function

• "bernoulli_logit": Binary data with Bernoulli likelihood and a logit link function

• "gamma": Pamma distribution with a with log link function

• "poisson": Poisson distribution with a with log link function

• "negative_binomial": negative binomial distribution with a with log link function

• "beta" : Beta likelihood with a logit link function (parametrization of Ferrari and Cribari-Neto, 2004)

• "t": t-distribution (e.g., for robust regression)

• "t_fix_df": t-distribution with the degrees-of-freedom (df) held fixed and not estimated. The df can be set via the likelihood_additional_param parameter

• "gaussian_heteroscedastic": Gaussian likelihood where both the mean and the variance are related to fixed and random effects. This is currently only implemented for GPs with a 'vecchia' approximation

• Note: other likelihoods could be implemented upon request

A vector or matrix whose columns are categorical grouping variables. The elements being group levels defining grouped random effects. The elements of 'group_data' can be integer, double, or character. The number of columns corresponds to the number of grouped (intercept) random effects

A vector or matrix with numeric covariate data for grouped random coefficients

A vector with integer indices that indicate the corresponding categorical grouping variable (=columns) in 'group_data' for every covariate in 'group_rand_coef_data'. Counting starts at 1. The length of this index vector must equal the number of covariates in 'group_rand_coef_data'. For instance, c(1,1,2) means that the first two covariates (=first two columns) in 'group_rand_coef_data' have random coefficients corresponding to the first categorical grouping variable (=first column) in 'group_data', and the third covariate (=third column) in 'group_rand_coef_data' has a random coefficient corresponding to the second grouping variable (=second column) in 'group_data'

A vector of type logical (boolean). Indicates whether intercept random effects are dropped (only for random coefficients). If drop_intercept_group_rand_effect[k] is TRUE, the intercept random effect number k is dropped / not included. Only random effects with random slopes can be dropped.

A matrix with numeric coordinates (= inputs / features) for defining Gaussian processes

A vector or matrix with numeric covariate data for Gaussian process random coefficients

A string specifying the covariance function for the Gaussian process. Available options:

• "matern": Matern covariance function with the smoothness specified by the cov_fct_shape parameter (using the parametrization of Rasmussen and Williams, 2006)

• "matern_estimate_shape": same as "matern" but the smoothness parameter is also estimated

• "matern_space_time": Spatio-temporal Matern covariance function with different range parameters for space and time. Note that the first column in gp_coords must correspond to the time dimension

• "matern_ard": anisotropic Matern covariance function with Automatic Relevance Determination (ARD), i.e., with a different range parameter for every coordinate dimension / column of gp_coords

• "matern_ard_estimate_shape": same as "matern_ard" but the smoothness parameter is also estimated

• "exponential": Exponential covariance function (using the parametrization of Diggle and Ribeiro, 2007)

• "gaussian": Gaussian, aka squared exponential, covariance function (using the parametrization of Diggle and Ribeiro, 2007)

• "gaussian_ard": anisotropic Gaussian, aka squared exponential, covariance function with Automatic Relevance Determination (ARD), i.e., with a different range parameter for every coordinate dimension / column of gp_coords

• "powered_exponential": powered exponential covariance function with the exponent specified by the cov_fct_shape parameter (using the parametrization of Diggle and Ribeiro, 2007)

• "wendland": Compactly supported Wendland covariance function (using the parametrization of Bevilacqua et al., 2019, AOS)

A numeric specifying the shape parameter of the covariance function (e.g., smoothness parameter for Matern and Wendland covariance) This parameter is irrelevant for some covariance functions such as the exponential or Gaussian

A string specifying the large data approximation for Gaussian processes. Available options:

• "none": No approximation

• "vecchia": Vecchia approximation; see Sigrist (2022, JMLR) for more details

• "full_scale_vecchia": Vecchia-inducing points full-scale (VIF) approximation; see Gyger, Furrer, and Sigrist (2025) for more details

• "tapering": The covariance function is multiplied by a compactly supported Wendland correlation function

• "fitc": Fully Independent Training Conditional approximation aka modified predictive process approximation; see Gyger, Furrer, and Sigrist (2024) for more details

• "full_scale_tapering": Full-scale approximation combining an inducing point / predictive process approximation with tapering on the residual process; see Gyger, Furrer, and Sigrist (2024) for more details

• "vecchia_latent": similar as "vecchia" but a Vecchia approximation is applied to the latent Gaussian process for likelihood == "gaussian". For likelihood != "gaussian", "vecchia" and "vecchia_latent" are equivalent

An integer specifying the number of parallel threads for OMP. If num_parallel_threads = NULL, all available threads are used

A string specifying the method used for inverting covariance matrices. Available options:

• "default": iterative methods where possible, otherwise Cholesky factorization

• "cholesky": Cholesky factorization

• "iterative": iterative methods. A combination of the conjugate gradient, the Lanczos algorithm, and other methods.

  This is currently only supported for the following cases:

  • grouped random effects with more than one level

  • likelihood != "gaussian" and gp_approx == "vecchia" (non-Gaussian likelihoods with a Vecchia-Laplace approximation)

  • likelihood != "gaussian" and gp_approx == "full_scale_vecchia" (non-Gaussian likelihoods with a VIF approximation)

  • likelihood == "gaussian" and gp_approx == "full_scale_tapering" (Gaussian likelihood with a full-scale tapering approximation)

A vector with sample weights

A numeric with a learning rate for the likelihood for generalized Bayesian inference (only non-Gaussian likelihoods)

A numeric specifying the range parameter of the Wendland covariance function and Wendland correlation taper function. We follow the notation of Bevilacqua et al. (2019, AOS)

A numeric specifying the shape (=smoothness) parameter of the Wendland covariance function and Wendland correlation taper function. We follow the notation of Bevilacqua et al. (2019, AOS)

An integer specifying the number of neighbors for the Vecchia and VIF approximations. Internal default values if NULL:

• 20 for gp_approx = "vecchia"

• 30 for gp_approx = "full_scale_vecchia"

Note: for prediction, the number of neighbors can be set through the 'num_neighbors_pred' parameter in the 'set_prediction_data' function. By default, num_neighbors_pred = 2 * num_neighbors. Further, the type of Vecchia approximation used for making predictions is set through the 'vecchia_pred_type' parameter in the 'set_prediction_data' function

A string specifying the ordering used in the Vecchia approximation. Available options:

• "none": the default ordering in the data is used

• "random": a random ordering

• "time": ordering accorrding to time (only for space-time models)

• "time_random_space": ordering according to time and randomly for all spatial points with the same time points (only for space-time models)

A string specifying the method for choosing inducing points Available options:

• "kmeans++: the k-means++ algorithm

• "cover_tree": the cover tree algorithm

• "random": random selection from data points

An integer specifying the number of inducing points / knots for FITC, full_scale_tapering, and VIF approximations. Internal default values if NULL:

• 500 for gp_approx = "FITC" and gp_approx = "full_scale_tapering"

• 200 for gp_approx = "full_scale_vecchia"

A numeric specifying the radius (= "spatial resolution") for the cover tree algorithm

An integer specifying the seed used for model creation (e.g., random ordering in Vecchia approximation)

A vector with elements indicating independent realizations of random effects / Gaussian processes (same values = same process realization). The elements of 'cluster_ids' can be integer, double, or character.

A numeric specifying an additional parameter for the likelihood which cannot be estimated for this likelihood (e.g., degrees of freedom for likelihood = "t_fix_df"). This is not to be confused with any auxiliary parameters that can be estimated and accessed through the function get_aux_pars after estimation. Note that this likelihood_additional_param parameter is irrelevant for many likelihoods. If likelihood_additional_param = NULL, the following internal default values are used:

• df = 2 for likelihood = "t_fix_df"

A boolean. If TRUE, the data (groups, coordinates, covariate data for random coefficients) is freed in R after initialization

Discontinued. Use the argument gp_approx instead

A string specifying the type of Vecchia approximation used for making predictions. This is discontinued here. Use the function 'set_prediction_data' to specify this

an integer specifying the number of neighbors for making predictions. This is discontinued here. Use the function 'set_prediction_data' to specify this

A string specifying the likelihood function (distribution) of the response variable. Available options:

• "gaussian"

• "bernoulli_probit": Binary data with Bernoulli likelihood and a probit link function

• "bernoulli_logit": Binary data with Bernoulli likelihood and a logit link function

• "gamma": Pamma distribution with a with log link function

• "poisson": Poisson distribution with a with log link function

• "negative_binomial": negative binomial distribution with a with log link function

• "beta" : Beta likelihood with a logit link function (parametrization of Ferrari and Cribari-Neto, 2004)

• "t": t-distribution (e.g., for robust regression)

• "t_fix_df": t-distribution with the degrees-of-freedom (df) held fixed and not estimated. The df can be set via the likelihood_additional_param parameter

• "gaussian_heteroscedastic": Gaussian likelihood where both the mean and the variance are related to fixed and random effects. This is currently only implemented for GPs with a 'vecchia' approximation

• Note: other likelihoods could be implemented upon request

A numeric specifying an additional parameter for the likelihood which cannot be estimated for this likelihood (e.g., degrees of freedom for likelihood = "t_fix_df"). This is not to be confused with any auxiliary parameters that can be estimated and accessed through the function get_aux_pars after estimation. Note that this likelihood_additional_param parameter is irrelevant for many likelihoods. If likelihood_additional_param = NULL, the following internal default values are used:

• df = 2 for likelihood = "t_fix_df"

A vector or matrix whose columns are categorical grouping variables. The elements being group levels defining grouped random effects. The elements of 'group_data' can be integer, double, or character. The number of columns corresponds to the number of grouped (intercept) random effects

A vector or matrix with numeric covariate data for grouped random coefficients

A vector with integer indices that indicate the corresponding categorical grouping variable (=columns) in 'group_data' for every covariate in 'group_rand_coef_data'. Counting starts at 1. The length of this index vector must equal the number of covariates in 'group_rand_coef_data'. For instance, c(1,1,2) means that the first two covariates (=first two columns) in 'group_rand_coef_data' have random coefficients corresponding to the first categorical grouping variable (=first column) in 'group_data', and the third covariate (=third column) in 'group_rand_coef_data' has a random coefficient corresponding to the second grouping variable (=second column) in 'group_data'

A vector of type logical (boolean). Indicates whether intercept random effects are dropped (only for random coefficients). If drop_intercept_group_rand_effect[k] is TRUE, the intercept random effect number k is dropped / not included. Only random effects with random slopes can be dropped.

A matrix with numeric coordinates (= inputs / features) for defining Gaussian processes

A vector or matrix with numeric covariate data for Gaussian process random coefficients

A string specifying the covariance function for the Gaussian process. Available options:

• "matern": Matern covariance function with the smoothness specified by the cov_fct_shape parameter (using the parametrization of Rasmussen and Williams, 2006)

• "matern_estimate_shape": same as "matern" but the smoothness parameter is also estimated

• "matern_space_time": Spatio-temporal Matern covariance function with different range parameters for space and time. Note that the first column in gp_coords must correspond to the time dimension

• "matern_ard": anisotropic Matern covariance function with Automatic Relevance Determination (ARD), i.e., with a different range parameter for every coordinate dimension / column of gp_coords

• "matern_ard_estimate_shape": same as "matern_ard" but the smoothness parameter is also estimated

• "exponential": Exponential covariance function (using the parametrization of Diggle and Ribeiro, 2007)

• "gaussian": Gaussian, aka squared exponential, covariance function (using the parametrization of Diggle and Ribeiro, 2007)

• "gaussian_ard": anisotropic Gaussian, aka squared exponential, covariance function with Automatic Relevance Determination (ARD), i.e., with a different range parameter for every coordinate dimension / column of gp_coords

• "powered_exponential": powered exponential covariance function with the exponent specified by the cov_fct_shape parameter (using the parametrization of Diggle and Ribeiro, 2007)

• "wendland": Compactly supported Wendland covariance function (using the parametrization of Bevilacqua et al., 2019, AOS)

A numeric specifying the shape parameter of the covariance function (e.g., smoothness parameter for Matern and Wendland covariance) This parameter is irrelevant for some covariance functions such as the exponential or Gaussian

A string specifying the large data approximation for Gaussian processes. Available options:

• "none": No approximation

• "vecchia": Vecchia approximation; see Sigrist (2022, JMLR) for more details

• "full_scale_vecchia": Vecchia-inducing points full-scale (VIF) approximation; see Gyger, Furrer, and Sigrist (2025) for more details

• "tapering": The covariance function is multiplied by a compactly supported Wendland correlation function

• "fitc": Fully Independent Training Conditional approximation aka modified predictive process approximation; see Gyger, Furrer, and Sigrist (2024) for more details

• "full_scale_tapering": Full-scale approximation combining an inducing point / predictive process approximation with tapering on the residual process; see Gyger, Furrer, and Sigrist (2024) for more details

• "vecchia_latent": similar as "vecchia" but a Vecchia approximation is applied to the latent Gaussian process for likelihood == "gaussian". For likelihood != "gaussian", "vecchia" and "vecchia_latent" are equivalent

An integer specifying the number of parallel threads for OMP. If num_parallel_threads = NULL, all available threads are used

A numeric specifying the range parameter of the Wendland covariance function and Wendland correlation taper function. We follow the notation of Bevilacqua et al. (2019, AOS)

A numeric specifying the shape (=smoothness) parameter of the Wendland covariance function and Wendland correlation taper function. We follow the notation of Bevilacqua et al. (2019, AOS)

An integer specifying the number of neighbors for the Vecchia and VIF approximations. Internal default values if NULL:

• 20 for gp_approx = "vecchia"

• 30 for gp_approx = "full_scale_vecchia"

Note: for prediction, the number of neighbors can be set through the 'num_neighbors_pred' parameter in the 'set_prediction_data' function. By default, num_neighbors_pred = 2 * num_neighbors. Further, the type of Vecchia approximation used for making predictions is set through the 'vecchia_pred_type' parameter in the 'set_prediction_data' function

A string specifying the ordering used in the Vecchia approximation. Available options:

• "none": the default ordering in the data is used

• "random": a random ordering

• "time": ordering accorrding to time (only for space-time models)

• "time_random_space": ordering according to time and randomly for all spatial points with the same time points (only for space-time models)

A string specifying the method for choosing inducing points Available options:

• "kmeans++: the k-means++ algorithm

• "cover_tree": the cover tree algorithm

• "random": random selection from data points

An integer specifying the number of inducing points / knots for FITC, full_scale_tapering, and VIF approximations. Internal default values if NULL:

• 500 for gp_approx = "FITC" and gp_approx = "full_scale_tapering"

• 200 for gp_approx = "full_scale_vecchia"

A numeric specifying the radius (= "spatial resolution") for the cover tree algorithm

A string specifying the method used for inverting covariance matrices. Available options:

• "default": iterative methods where possible, otherwise Cholesky factorization

• "cholesky": Cholesky factorization

• "iterative": iterative methods. A combination of the conjugate gradient, the Lanczos algorithm, and other methods.

  This is currently only supported for the following cases:

  • grouped random effects with more than one level

  • likelihood != "gaussian" and gp_approx == "vecchia" (non-Gaussian likelihoods with a Vecchia-Laplace approximation)

  • likelihood != "gaussian" and gp_approx == "full_scale_vecchia" (non-Gaussian likelihoods with a VIF approximation)

  • likelihood == "gaussian" and gp_approx == "full_scale_tapering" (Gaussian likelihood with a full-scale tapering approximation)

An integer specifying the seed used for model creation (e.g., random ordering in Vecchia approximation)

A string specifying the type of Vecchia approximation used for making predictions. Default value if vecchia_pred_type = NULL: "order_obs_first_cond_obs_only". Available options:

• "order_obs_first_cond_obs_only": Vecchia approximation for the observable process and observed training data is ordered first and the neighbors are only observed training data points

• "order_obs_first_cond_all": Vecchia approximation for the observable process and observed training data is ordered first and the neighbors are selected among all points (training + prediction)

• "latent_order_obs_first_cond_obs_only": Vecchia approximation for the latent process and observed data is ordered first and neighbors are only observed points

• "latent_order_obs_first_cond_all": Vecchia approximation for the latent process and observed data is ordered first and neighbors are selected among all points

• "order_pred_first": Vecchia approximation for the observable process and prediction data is ordered first for making predictions. This option is only available for Gaussian likelihoods

an integer specifying the number of neighbors for the Vecchia approximation for making predictions. Default value if NULL: num_neighbors_pred = 2 * num_neighbors

a numeric specifying the tolerance level for L2 norm of residuals for checking convergence in conjugate gradient algorithms when being used for prediction Default value if NULL: 1e-3

an integer specifying the number of samples when simulation is used for calculating predictive variances Internal default values if NULL:

• 500 for grouped random effects

• 1000 for gp_approx = "vecchia" and gp_approx = "full_scale_tapering"

• 100 for gp_approx = "full_scale_vecchia"

an integer specifying the rank of the matrix for approximating predictive covariances obtained using the Lanczos algorithm Default value if NULL: 1000

A vector with elements indicating independent realizations of random effects / Gaussian processes (same values = same process realization). The elements of 'cluster_ids' can be integer, double, or character.

A vector with sample weights

A numeric with a learning rate for the likelihood for generalized Bayesian inference (only non-Gaussian likelihoods)

A boolean. If TRUE, the data (groups, coordinates, covariate data for random coefficients) is freed in R after initialization

A vector with response variable data

A matrix with numeric covariate data for the fixed effects linear regression term (if there is one)

A list with parameters for the estimation / optimization

• trace: boolean (default = FALSE). If TRUE, information on the progress of the parameter optimization is printed

• std_dev: boolean (default = TRUE). If TRUE, approximate standard deviations are calculated for the covariance and linear regression parameters (= square root of diagonal of the inverse Fisher information for Gaussian likelihoods and square root of diagonal of a numerically approximated inverse Hessian for non-Gaussian likelihoods)

• init_cov_pars: vector with numeric elements (default = NULL). Initial values for covariance parameters of Gaussian process and random effects (can be NULL). The order is same as the order of the parameters in the summary function: first is the error variance (only for "gaussian" likelihood), next follow the variances of the grouped random effects (if there are any, in the order provided in 'group_data'), and then follow the marginal variance and the ranges of the Gaussian process. If there are multiple Gaussian processes, then the variances and ranges follow alternatingly. If 'init_cov_pars = NULL', an internal choice is used that depends on the likelihood and the random effects type and covariance function. If you select the option 'trace = TRUE' in the 'params' argument, you will see the first initial covariance parameters in iteration 0.

• init_coef: vector with numeric elements (default = NULL). Initial values for the regression coefficients (if there are any, can be NULL)

• init_aux_pars: vector with numeric elements (default = NULL). Initial values for additional parameters for non-Gaussian likelihoods (e.g., shape parameter of a gamma or negative_binomial likelihood)

• estimate_cov_par_index: vector with integer (default = -1). If estimate_cov_par_index[1] >= 0, some covariance parameters might not be estimated, estimate_cov_par_index[i] is then bool and indicates which ones are estimated. For instance, estimate_cov_par_index = c(1,1,0) means that the first two covariance parameters are estimated and the last one not.

• estimate_aux_pars: boolean (default = TRUE). If TRUE, additional parameters for non-Gaussian likelihoods are also estimated (e.g., shape parameter of a gamma or negative_binomial likelihood)

• optimizer_cov: string (default = "lbfgs"). Optimizer used for estimating covariance parameters. Options: "lbfgs", "gradient_descent", "fisher_scoring", "newton", "nelder_mead". If there are additional auxiliary parameters for non-Gaussian likelihoods, 'optimizer_cov' is also used for those

• optimizer_coef: string (default = "wls" for Gaussian likelihoods and "lbfgs" for other likelihoods). Optimizer used for estimating linear regression coefficients, if there are any (for the GPBoost algorithm there are usually none). Options: "gradient_descent", "lbfgs", "wls", "nelder_mead". Gradient descent steps are done simultaneously with gradient descent steps for the covariance parameters. "wls" refers to doing coordinate descent for the regression coefficients using weighted least squares. If 'optimizer_cov' is set to "nelder_mead" or "lbfgs", 'optimizer_coef' is automatically also set to the same value.

• maxit: integer (default = 1000). Maximal number of iterations for optimization algorithm

• delta_rel_conv: numeric (default = 1E-6 except for "nelder_mead" for which the default is 1E-8). Convergence tolerance. The algorithm stops if the relative change in either the (approximate) log-likelihood or the parameters is below this value. If < 0, internal default values are used

• cg_max_num_it: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithms

• cg_max_num_it_tridiag: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithm when being run as Lanczos algorithm for tridiagonalization

• cg_delta_conv: numeric (default = 1E-2). Tolerance level for L2 norm of residuals for checking convergence in conjugate gradient algorithm when being used for parameter estimation

• num_rand_vec_trace: integer (default = 50). Number of random vectors (e.g., Rademacher) for stochastic approximation of the trace of a matrix

• reuse_rand_vec_trace: boolean (default = TRUE). If true, random vectors (e.g., Rademacher) for stochastic approximations of the trace of a matrix are sampled only once at the beginning of the parameter estimation and reused in later trace approximations. Otherwise they are sampled every time a trace is calculated

• seed_rand_vec_trace: integer (default = 1). Seed number to generate random vectors (e.g., Rademacher)

• cg_preconditioner_type (string): Type of preconditioner used for conjugate gradient algorithms.

  • Options for grouped random effects:

    • "ssor" (= default): SSOR preconditioner

    • "incomplete_cholesky": zero fill-in incomplete Cholesky factorization

  • Options for likelihood != "gaussian" and gp_approx == "vecchia" or likelihood == "gaussian" and gp_approx == "vecchia_latent":

    • "vadu" (= default): (B^T * (D^-1 + W) * B) as preconditioner for inverting (B^T * D^-1 * B + W), where B^T * D^-1 * B approx= Sigma^-1

    • "fitc": FITC / modified predictive process preconditioner for inverting (B^-1 * D * B^-T + W^-1)

    • "pivoted_cholesky": (Lk * Lk^T + W^-1) as preconditioner for inverting (B^-1 * D * B^-T + W^-1), where Lk is a low-rank pivoted Cholesky approximation for Sigma and B^-1 * D * B^-T approx= Sigma

    • "incomplete_cholesky": zero fill-in incomplete (reverse) Cholesky factorization of (B^T * D^-1 * B + W) using the sparsity pattern of B^T * D^-1 * B approx= Sigma^-1

  • Options for likelihood != "gaussian" and gp_approx == "full_scale_vecchia":

    • "fitc" ( = default): FITC / modified predictive process preconditioner

    • "vifdu": VIF with diagonal update preconditioner

  • Options for likelihood == "gaussian" and gp_approx == "full_scale_tapering":

    • "fitc" (= default): modified predictive process preconditioner

    • "none": no preconditioner

• fitc_piv_chol_preconditioner_rank (integer ): Rank of the FITC and pivoted Cholesky decomposition preconditioners for iterative methods for Vecchia and VIF approximations (for full_scale_tapering, the same inducing points as in the approximation as used). Internal default values if NULL or < 0:

  • 200 for the FITC preconditioner

  • 50 for the pivoted Cholesky decomposition preconditioner

• convergence_criterion: string (default = "relative_change_in_log_likelihood", only relevant for "gradient_descent", "fisher_scoring", and "newton"). The convergence criterion used for terminating the optimization algorithm. Options: "relative_change_in_log_likelihood" or "relative_change_in_parameters"

• lr_cov: numeric (default = 0.1 for "gradient_descent" and 1. otherwise, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Initial learning rate for covariance parameters if a gradient-based optimization method is used

  • If lr_cov < 0, internal default values are used (0.1 for "gradient_descent" and 1. otherwise)

  • If there are additional auxiliary parameters for non-Gaussian likelihoods, 'lr_cov' is also used for those

  • For "lbfgs", this is divided by the norm of the gradient in the first iteration

• lr_coef: numeric (default = 0.1, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Learning rate for fixed effect regression coefficients if gradient descent is used

• use_nesterov_acc: boolean (default = TRUE, only relevant for "gradient_descent"). If TRUE Nesterov acceleration is used. This is used only for gradient descent

• acc_rate_coef: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for regression coefficients (if there are any) for Nesterov acceleration

• acc_rate_cov: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for covariance parameters for Nesterov acceleration

• momentum_offset: integer (Default = 2, only relevant for "gradient_descent"). Number of iterations for which no momentum is applied in the beginning.

A numeric vector with additional fixed effects contributions that are added to the linear predictor (= offset). The length of this vector needs to equal the number of training data points.

This is discontinued. Use the renamed equivalent argument offset instead

A vector or matrix with elements being group levels for which predictions are made (if there are grouped random effects in the GPModel)

A vector or matrix with covariate data for grouped random coefficients (if there are some in the GPModel)

A matrix with prediction coordinates (=features) for Gaussian process (if there is a GP in the GPModel)

A vector or matrix with covariate data for Gaussian process random coefficients (if there are some in the GPModel)

A vector with elements indicating the realizations of random effects / Gaussian processes for which predictions are made (set to NULL if you have not specified this when creating the GPModel)

A matrix with prediction covariate data for the fixed effects linear regression term (if there is one in the GPModel)

A boolean. If TRUE, the (posterior) predictive covariance is calculated in addition to the (posterior) predictive mean

A boolean. If TRUE, the (posterior) predictive variances are calculated

Discontinued. Use the argument gp_approx instead

Object of class gpb.Dataset

other parameters

object of class gpb.Dataset

a list of two elements: the first one is ignored and the second one is column names

a GPModel

A vector with response variable data

A matrix with numeric covariate data for the fixed effects linear regression term (if there is one)

A list with parameters for the estimation / optimization

• trace: boolean (default = FALSE). If TRUE, information on the progress of the parameter optimization is printed

• std_dev: boolean (default = TRUE). If TRUE, approximate standard deviations are calculated for the covariance and linear regression parameters (= square root of diagonal of the inverse Fisher information for Gaussian likelihoods and square root of diagonal of a numerically approximated inverse Hessian for non-Gaussian likelihoods)

• init_cov_pars: vector with numeric elements (default = NULL). Initial values for covariance parameters of Gaussian process and random effects (can be NULL). The order is same as the order of the parameters in the summary function: first is the error variance (only for "gaussian" likelihood), next follow the variances of the grouped random effects (if there are any, in the order provided in 'group_data'), and then follow the marginal variance and the ranges of the Gaussian process. If there are multiple Gaussian processes, then the variances and ranges follow alternatingly. If 'init_cov_pars = NULL', an internal choice is used that depends on the likelihood and the random effects type and covariance function. If you select the option 'trace = TRUE' in the 'params' argument, you will see the first initial covariance parameters in iteration 0.

• init_coef: vector with numeric elements (default = NULL). Initial values for the regression coefficients (if there are any, can be NULL)

• init_aux_pars: vector with numeric elements (default = NULL). Initial values for additional parameters for non-Gaussian likelihoods (e.g., shape parameter of a gamma or negative_binomial likelihood)

• estimate_cov_par_index: vector with integer (default = -1). If estimate_cov_par_index[1] >= 0, some covariance parameters might not be estimated, estimate_cov_par_index[i] is then bool and indicates which ones are estimated. For instance, estimate_cov_par_index = c(1,1,0) means that the first two covariance parameters are estimated and the last one not.

• estimate_aux_pars: boolean (default = TRUE). If TRUE, additional parameters for non-Gaussian likelihoods are also estimated (e.g., shape parameter of a gamma or negative_binomial likelihood)

• optimizer_cov: string (default = "lbfgs"). Optimizer used for estimating covariance parameters. Options: "lbfgs", "gradient_descent", "fisher_scoring", "newton", "nelder_mead". If there are additional auxiliary parameters for non-Gaussian likelihoods, 'optimizer_cov' is also used for those

• optimizer_coef: string (default = "wls" for Gaussian likelihoods and "lbfgs" for other likelihoods). Optimizer used for estimating linear regression coefficients, if there are any (for the GPBoost algorithm there are usually none). Options: "gradient_descent", "lbfgs", "wls", "nelder_mead". Gradient descent steps are done simultaneously with gradient descent steps for the covariance parameters. "wls" refers to doing coordinate descent for the regression coefficients using weighted least squares. If 'optimizer_cov' is set to "nelder_mead" or "lbfgs", 'optimizer_coef' is automatically also set to the same value.

• maxit: integer (default = 1000). Maximal number of iterations for optimization algorithm

• delta_rel_conv: numeric (default = 1E-6 except for "nelder_mead" for which the default is 1E-8). Convergence tolerance. The algorithm stops if the relative change in either the (approximate) log-likelihood or the parameters is below this value. If < 0, internal default values are used

• cg_max_num_it: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithms

• cg_max_num_it_tridiag: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithm when being run as Lanczos algorithm for tridiagonalization

• cg_delta_conv: numeric (default = 1E-2). Tolerance level for L2 norm of residuals for checking convergence in conjugate gradient algorithm when being used for parameter estimation

• num_rand_vec_trace: integer (default = 50). Number of random vectors (e.g., Rademacher) for stochastic approximation of the trace of a matrix

• reuse_rand_vec_trace: boolean (default = TRUE). If true, random vectors (e.g., Rademacher) for stochastic approximations of the trace of a matrix are sampled only once at the beginning of the parameter estimation and reused in later trace approximations. Otherwise they are sampled every time a trace is calculated

• seed_rand_vec_trace: integer (default = 1). Seed number to generate random vectors (e.g., Rademacher)

• cg_preconditioner_type (string): Type of preconditioner used for conjugate gradient algorithms.

  • Options for grouped random effects:

    • "ssor" (= default): SSOR preconditioner

    • "incomplete_cholesky": zero fill-in incomplete Cholesky factorization

  • Options for likelihood != "gaussian" and gp_approx == "vecchia" or likelihood == "gaussian" and gp_approx == "vecchia_latent":

    • "vadu" (= default): (B^T * (D^-1 + W) * B) as preconditioner for inverting (B^T * D^-1 * B + W), where B^T * D^-1 * B approx= Sigma^-1

    • "fitc": FITC / modified predictive process preconditioner for inverting (B^-1 * D * B^-T + W^-1)

    • "pivoted_cholesky": (Lk * Lk^T + W^-1) as preconditioner for inverting (B^-1 * D * B^-T + W^-1), where Lk is a low-rank pivoted Cholesky approximation for Sigma and B^-1 * D * B^-T approx= Sigma

    • "incomplete_cholesky": zero fill-in incomplete (reverse) Cholesky factorization of (B^T * D^-1 * B + W) using the sparsity pattern of B^T * D^-1 * B approx= Sigma^-1

  • Options for likelihood != "gaussian" and gp_approx == "full_scale_vecchia":

    • "fitc" ( = default): FITC / modified predictive process preconditioner

    • "vifdu": VIF with diagonal update preconditioner

  • Options for likelihood == "gaussian" and gp_approx == "full_scale_tapering":

    • "fitc" (= default): modified predictive process preconditioner

    • "none": no preconditioner

• fitc_piv_chol_preconditioner_rank (integer ): Rank of the FITC and pivoted Cholesky decomposition preconditioners for iterative methods for Vecchia and VIF approximations (for full_scale_tapering, the same inducing points as in the approximation as used). Internal default values if NULL or < 0:

  • 200 for the FITC preconditioner

  • 50 for the pivoted Cholesky decomposition preconditioner

• convergence_criterion: string (default = "relative_change_in_log_likelihood", only relevant for "gradient_descent", "fisher_scoring", and "newton"). The convergence criterion used for terminating the optimization algorithm. Options: "relative_change_in_log_likelihood" or "relative_change_in_parameters"

• lr_cov: numeric (default = 0.1 for "gradient_descent" and 1. otherwise, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Initial learning rate for covariance parameters if a gradient-based optimization method is used

  • If lr_cov < 0, internal default values are used (0.1 for "gradient_descent" and 1. otherwise)

  • If there are additional auxiliary parameters for non-Gaussian likelihoods, 'lr_cov' is also used for those

  • For "lbfgs", this is divided by the norm of the gradient in the first iteration

• lr_coef: numeric (default = 0.1, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Learning rate for fixed effect regression coefficients if gradient descent is used

• use_nesterov_acc: boolean (default = TRUE, only relevant for "gradient_descent"). If TRUE Nesterov acceleration is used. This is used only for gradient descent

• acc_rate_coef: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for regression coefficients (if there are any) for Nesterov acceleration

• acc_rate_cov: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for covariance parameters for Nesterov acceleration

• momentum_offset: integer (Default = 2, only relevant for "gradient_descent"). Number of iterations for which no momentum is applied in the beginning.

A numeric vector with additional fixed effects contributions that are added to the linear predictor (= offset). The length of this vector needs to equal the number of training data points.

This is discontinued. Use the renamed equivalent argument offset instead

a GPModel

A vector with response variable data

A matrix with numeric covariate data for the fixed effects linear regression term (if there is one)

A list with parameters for the estimation / optimization

• trace: boolean (default = FALSE). If TRUE, information on the progress of the parameter optimization is printed

• std_dev: boolean (default = TRUE). If TRUE, approximate standard deviations are calculated for the covariance and linear regression parameters (= square root of diagonal of the inverse Fisher information for Gaussian likelihoods and square root of diagonal of a numerically approximated inverse Hessian for non-Gaussian likelihoods)

• init_cov_pars: vector with numeric elements (default = NULL). Initial values for covariance parameters of Gaussian process and random effects (can be NULL). The order is same as the order of the parameters in the summary function: first is the error variance (only for "gaussian" likelihood), next follow the variances of the grouped random effects (if there are any, in the order provided in 'group_data'), and then follow the marginal variance and the ranges of the Gaussian process. If there are multiple Gaussian processes, then the variances and ranges follow alternatingly. If 'init_cov_pars = NULL', an internal choice is used that depends on the likelihood and the random effects type and covariance function. If you select the option 'trace = TRUE' in the 'params' argument, you will see the first initial covariance parameters in iteration 0.

• init_coef: vector with numeric elements (default = NULL). Initial values for the regression coefficients (if there are any, can be NULL)

• init_aux_pars: vector with numeric elements (default = NULL). Initial values for additional parameters for non-Gaussian likelihoods (e.g., shape parameter of a gamma or negative_binomial likelihood)

• estimate_cov_par_index: vector with integer (default = -1). If estimate_cov_par_index[1] >= 0, some covariance parameters might not be estimated, estimate_cov_par_index[i] is then bool and indicates which ones are estimated. For instance, estimate_cov_par_index = c(1,1,0) means that the first two covariance parameters are estimated and the last one not.

• estimate_aux_pars: boolean (default = TRUE). If TRUE, additional parameters for non-Gaussian likelihoods are also estimated (e.g., shape parameter of a gamma or negative_binomial likelihood)

• optimizer_cov: string (default = "lbfgs"). Optimizer used for estimating covariance parameters. Options: "lbfgs", "gradient_descent", "fisher_scoring", "newton", "nelder_mead". If there are additional auxiliary parameters for non-Gaussian likelihoods, 'optimizer_cov' is also used for those

• optimizer_coef: string (default = "wls" for Gaussian likelihoods and "lbfgs" for other likelihoods). Optimizer used for estimating linear regression coefficients, if there are any (for the GPBoost algorithm there are usually none). Options: "gradient_descent", "lbfgs", "wls", "nelder_mead". Gradient descent steps are done simultaneously with gradient descent steps for the covariance parameters. "wls" refers to doing coordinate descent for the regression coefficients using weighted least squares. If 'optimizer_cov' is set to "nelder_mead" or "lbfgs", 'optimizer_coef' is automatically also set to the same value.

• maxit: integer (default = 1000). Maximal number of iterations for optimization algorithm

• delta_rel_conv: numeric (default = 1E-6 except for "nelder_mead" for which the default is 1E-8). Convergence tolerance. The algorithm stops if the relative change in either the (approximate) log-likelihood or the parameters is below this value. If < 0, internal default values are used

• cg_max_num_it: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithms

• cg_max_num_it_tridiag: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithm when being run as Lanczos algorithm for tridiagonalization

• cg_delta_conv: numeric (default = 1E-2). Tolerance level for L2 norm of residuals for checking convergence in conjugate gradient algorithm when being used for parameter estimation

• num_rand_vec_trace: integer (default = 50). Number of random vectors (e.g., Rademacher) for stochastic approximation of the trace of a matrix

• reuse_rand_vec_trace: boolean (default = TRUE). If true, random vectors (e.g., Rademacher) for stochastic approximations of the trace of a matrix are sampled only once at the beginning of the parameter estimation and reused in later trace approximations. Otherwise they are sampled every time a trace is calculated

• seed_rand_vec_trace: integer (default = 1). Seed number to generate random vectors (e.g., Rademacher)

• cg_preconditioner_type (string): Type of preconditioner used for conjugate gradient algorithms.

  • Options for grouped random effects:

    • "ssor" (= default): SSOR preconditioner

    • "incomplete_cholesky": zero fill-in incomplete Cholesky factorization

  • Options for likelihood != "gaussian" and gp_approx == "vecchia" or likelihood == "gaussian" and gp_approx == "vecchia_latent":

    • "vadu" (= default): (B^T * (D^-1 + W) * B) as preconditioner for inverting (B^T * D^-1 * B + W), where B^T * D^-1 * B approx= Sigma^-1

    • "fitc": FITC / modified predictive process preconditioner for inverting (B^-1 * D * B^-T + W^-1)

    • "pivoted_cholesky": (Lk * Lk^T + W^-1) as preconditioner for inverting (B^-1 * D * B^-T + W^-1), where Lk is a low-rank pivoted Cholesky approximation for Sigma and B^-1 * D * B^-T approx= Sigma

    • "incomplete_cholesky": zero fill-in incomplete (reverse) Cholesky factorization of (B^T * D^-1 * B + W) using the sparsity pattern of B^T * D^-1 * B approx= Sigma^-1

  • Options for likelihood != "gaussian" and gp_approx == "full_scale_vecchia":

    • "fitc" ( = default): FITC / modified predictive process preconditioner

    • "vifdu": VIF with diagonal update preconditioner

  • Options for likelihood == "gaussian" and gp_approx == "full_scale_tapering":

    • "fitc" (= default): modified predictive process preconditioner

    • "none": no preconditioner

• fitc_piv_chol_preconditioner_rank (integer ): Rank of the FITC and pivoted Cholesky decomposition preconditioners for iterative methods for Vecchia and VIF approximations (for full_scale_tapering, the same inducing points as in the approximation as used). Internal default values if NULL or < 0:

  • 200 for the FITC preconditioner

  • 50 for the pivoted Cholesky decomposition preconditioner

• convergence_criterion: string (default = "relative_change_in_log_likelihood", only relevant for "gradient_descent", "fisher_scoring", and "newton"). The convergence criterion used for terminating the optimization algorithm. Options: "relative_change_in_log_likelihood" or "relative_change_in_parameters"

• lr_cov: numeric (default = 0.1 for "gradient_descent" and 1. otherwise, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Initial learning rate for covariance parameters if a gradient-based optimization method is used

  • If lr_cov < 0, internal default values are used (0.1 for "gradient_descent" and 1. otherwise)

  • If there are additional auxiliary parameters for non-Gaussian likelihoods, 'lr_cov' is also used for those

  • For "lbfgs", this is divided by the norm of the gradient in the first iteration

• lr_coef: numeric (default = 0.1, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Learning rate for fixed effect regression coefficients if gradient descent is used

• use_nesterov_acc: boolean (default = TRUE, only relevant for "gradient_descent"). If TRUE Nesterov acceleration is used. This is used only for gradient descent

• acc_rate_coef: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for regression coefficients (if there are any) for Nesterov acceleration

• acc_rate_cov: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for covariance parameters for Nesterov acceleration

• momentum_offset: integer (Default = 2, only relevant for "gradient_descent"). Number of iterations for which no momentum is applied in the beginning.

A numeric vector with additional fixed effects contributions that are added to the linear predictor (= offset). The length of this vector needs to equal the number of training data points.

This is discontinued. Use the renamed equivalent argument offset instead

A string specifying the likelihood function (distribution) of the response variable. Available options:

• "gaussian"

• "bernoulli_probit": Binary data with Bernoulli likelihood and a probit link function

• "bernoulli_logit": Binary data with Bernoulli likelihood and a logit link function

• "gamma": Pamma distribution with a with log link function

• "poisson": Poisson distribution with a with log link function

• "negative_binomial": negative binomial distribution with a with log link function

• "beta" : Beta likelihood with a logit link function (parametrization of Ferrari and Cribari-Neto, 2004)

• "t": t-distribution (e.g., for robust regression)

• "t_fix_df": t-distribution with the degrees-of-freedom (df) held fixed and not estimated. The df can be set via the likelihood_additional_param parameter

• "gaussian_heteroscedastic": Gaussian likelihood where both the mean and the variance are related to fixed and random effects. This is currently only implemented for GPs with a 'vecchia' approximation

• Note: other likelihoods could be implemented upon request

A vector or matrix whose columns are categorical grouping variables. The elements being group levels defining grouped random effects. The elements of 'group_data' can be integer, double, or character. The number of columns corresponds to the number of grouped (intercept) random effects

A vector or matrix with numeric covariate data for grouped random coefficients

A vector with integer indices that indicate the corresponding categorical grouping variable (=columns) in 'group_data' for every covariate in 'group_rand_coef_data'. Counting starts at 1. The length of this index vector must equal the number of covariates in 'group_rand_coef_data'. For instance, c(1,1,2) means that the first two covariates (=first two columns) in 'group_rand_coef_data' have random coefficients corresponding to the first categorical grouping variable (=first column) in 'group_data', and the third covariate (=third column) in 'group_rand_coef_data' has a random coefficient corresponding to the second grouping variable (=second column) in 'group_data'

A vector of type logical (boolean). Indicates whether intercept random effects are dropped (only for random coefficients). If drop_intercept_group_rand_effect[k] is TRUE, the intercept random effect number k is dropped / not included. Only random effects with random slopes can be dropped.

A matrix with numeric coordinates (= inputs / features) for defining Gaussian processes

A vector or matrix with numeric covariate data for Gaussian process random coefficients

A string specifying the covariance function for the Gaussian process. Available options:

• "matern": Matern covariance function with the smoothness specified by the cov_fct_shape parameter (using the parametrization of Rasmussen and Williams, 2006)

• "matern_estimate_shape": same as "matern" but the smoothness parameter is also estimated

• "matern_space_time": Spatio-temporal Matern covariance function with different range parameters for space and time. Note that the first column in gp_coords must correspond to the time dimension

• "matern_ard": anisotropic Matern covariance function with Automatic Relevance Determination (ARD), i.e., with a different range parameter for every coordinate dimension / column of gp_coords

• "matern_ard_estimate_shape": same as "matern_ard" but the smoothness parameter is also estimated

• "exponential": Exponential covariance function (using the parametrization of Diggle and Ribeiro, 2007)

• "gaussian": Gaussian, aka squared exponential, covariance function (using the parametrization of Diggle and Ribeiro, 2007)

• "gaussian_ard": anisotropic Gaussian, aka squared exponential, covariance function with Automatic Relevance Determination (ARD), i.e., with a different range parameter for every coordinate dimension / column of gp_coords

• "powered_exponential": powered exponential covariance function with the exponent specified by the cov_fct_shape parameter (using the parametrization of Diggle and Ribeiro, 2007)

• "wendland": Compactly supported Wendland covariance function (using the parametrization of Bevilacqua et al., 2019, AOS)

A numeric specifying the shape parameter of the covariance function (e.g., smoothness parameter for Matern and Wendland covariance) This parameter is irrelevant for some covariance functions such as the exponential or Gaussian

A string specifying the large data approximation for Gaussian processes. Available options:

• "none": No approximation

• "vecchia": Vecchia approximation; see Sigrist (2022, JMLR) for more details

• "full_scale_vecchia": Vecchia-inducing points full-scale (VIF) approximation; see Gyger, Furrer, and Sigrist (2025) for more details

• "tapering": The covariance function is multiplied by a compactly supported Wendland correlation function

• "fitc": Fully Independent Training Conditional approximation aka modified predictive process approximation; see Gyger, Furrer, and Sigrist (2024) for more details

• "full_scale_tapering": Full-scale approximation combining an inducing point / predictive process approximation with tapering on the residual process; see Gyger, Furrer, and Sigrist (2024) for more details

• "vecchia_latent": similar as "vecchia" but a Vecchia approximation is applied to the latent Gaussian process for likelihood == "gaussian". For likelihood != "gaussian", "vecchia" and "vecchia_latent" are equivalent

An integer specifying the number of parallel threads for OMP. If num_parallel_threads = NULL, all available threads are used

A string specifying the method used for inverting covariance matrices. Available options:

• "default": iterative methods where possible, otherwise Cholesky factorization

• "cholesky": Cholesky factorization

• "iterative": iterative methods. A combination of the conjugate gradient, the Lanczos algorithm, and other methods.

  This is currently only supported for the following cases:

  • grouped random effects with more than one level

  • likelihood != "gaussian" and gp_approx == "vecchia" (non-Gaussian likelihoods with a Vecchia-Laplace approximation)

  • likelihood != "gaussian" and gp_approx == "full_scale_vecchia" (non-Gaussian likelihoods with a VIF approximation)

  • likelihood == "gaussian" and gp_approx == "full_scale_tapering" (Gaussian likelihood with a full-scale tapering approximation)

A vector with sample weights

A numeric with a learning rate for the likelihood for generalized Bayesian inference (only non-Gaussian likelihoods)

A numeric specifying the range parameter of the Wendland covariance function and Wendland correlation taper function. We follow the notation of Bevilacqua et al. (2019, AOS)

A numeric specifying the shape (=smoothness) parameter of the Wendland covariance function and Wendland correlation taper function. We follow the notation of Bevilacqua et al. (2019, AOS)

An integer specifying the number of neighbors for the Vecchia and VIF approximations. Internal default values if NULL:

• 20 for gp_approx = "vecchia"

• 30 for gp_approx = "full_scale_vecchia"

Note: for prediction, the number of neighbors can be set through the 'num_neighbors_pred' parameter in the 'set_prediction_data' function. By default, num_neighbors_pred = 2 * num_neighbors. Further, the type of Vecchia approximation used for making predictions is set through the 'vecchia_pred_type' parameter in the 'set_prediction_data' function

A string specifying the ordering used in the Vecchia approximation. Available options:

• "none": the default ordering in the data is used

• "random": a random ordering

• "time": ordering accorrding to time (only for space-time models)

• "time_random_space": ordering according to time and randomly for all spatial points with the same time points (only for space-time models)

A string specifying the method for choosing inducing points Available options:

• "kmeans++: the k-means++ algorithm

• "cover_tree": the cover tree algorithm

• "random": random selection from data points

An integer specifying the number of inducing points / knots for FITC, full_scale_tapering, and VIF approximations. Internal default values if NULL:

• 500 for gp_approx = "FITC" and gp_approx = "full_scale_tapering"

• 200 for gp_approx = "full_scale_vecchia"

A numeric specifying the radius (= "spatial resolution") for the cover tree algorithm

An integer specifying the seed used for model creation (e.g., random ordering in Vecchia approximation)

A vector with elements indicating independent realizations of random effects / Gaussian processes (same values = same process realization). The elements of 'cluster_ids' can be integer, double, or character.

A boolean. If TRUE, the data (groups, coordinates, covariate data for random coefficients) is freed in R after initialization

A vector with response variable data

A matrix with numeric covariate data for the fixed effects linear regression term (if there is one)

A list with parameters for the estimation / optimization

• trace: boolean (default = FALSE). If TRUE, information on the progress of the parameter optimization is printed

• std_dev: boolean (default = TRUE). If TRUE, approximate standard deviations are calculated for the covariance and linear regression parameters (= square root of diagonal of the inverse Fisher information for Gaussian likelihoods and square root of diagonal of a numerically approximated inverse Hessian for non-Gaussian likelihoods)

• init_cov_pars: vector with numeric elements (default = NULL). Initial values for covariance parameters of Gaussian process and random effects (can be NULL). The order is same as the order of the parameters in the summary function: first is the error variance (only for "gaussian" likelihood), next follow the variances of the grouped random effects (if there are any, in the order provided in 'group_data'), and then follow the marginal variance and the ranges of the Gaussian process. If there are multiple Gaussian processes, then the variances and ranges follow alternatingly. If 'init_cov_pars = NULL', an internal choice is used that depends on the likelihood and the random effects type and covariance function. If you select the option 'trace = TRUE' in the 'params' argument, you will see the first initial covariance parameters in iteration 0.

• init_coef: vector with numeric elements (default = NULL). Initial values for the regression coefficients (if there are any, can be NULL)

• init_aux_pars: vector with numeric elements (default = NULL). Initial values for additional parameters for non-Gaussian likelihoods (e.g., shape parameter of a gamma or negative_binomial likelihood)

• estimate_cov_par_index: vector with integer (default = -1). If estimate_cov_par_index[1] >= 0, some covariance parameters might not be estimated, estimate_cov_par_index[i] is then bool and indicates which ones are estimated. For instance, estimate_cov_par_index = c(1,1,0) means that the first two covariance parameters are estimated and the last one not.

• estimate_aux_pars: boolean (default = TRUE). If TRUE, additional parameters for non-Gaussian likelihoods are also estimated (e.g., shape parameter of a gamma or negative_binomial likelihood)

• optimizer_cov: string (default = "lbfgs"). Optimizer used for estimating covariance parameters. Options: "lbfgs", "gradient_descent", "fisher_scoring", "newton", "nelder_mead". If there are additional auxiliary parameters for non-Gaussian likelihoods, 'optimizer_cov' is also used for those

• optimizer_coef: string (default = "wls" for Gaussian likelihoods and "lbfgs" for other likelihoods). Optimizer used for estimating linear regression coefficients, if there are any (for the GPBoost algorithm there are usually none). Options: "gradient_descent", "lbfgs", "wls", "nelder_mead". Gradient descent steps are done simultaneously with gradient descent steps for the covariance parameters. "wls" refers to doing coordinate descent for the regression coefficients using weighted least squares. If 'optimizer_cov' is set to "nelder_mead" or "lbfgs", 'optimizer_coef' is automatically also set to the same value.

• maxit: integer (default = 1000). Maximal number of iterations for optimization algorithm

• delta_rel_conv: numeric (default = 1E-6 except for "nelder_mead" for which the default is 1E-8). Convergence tolerance. The algorithm stops if the relative change in either the (approximate) log-likelihood or the parameters is below this value. If < 0, internal default values are used

• cg_max_num_it: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithms

• cg_max_num_it_tridiag: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithm when being run as Lanczos algorithm for tridiagonalization

• cg_delta_conv: numeric (default = 1E-2). Tolerance level for L2 norm of residuals for checking convergence in conjugate gradient algorithm when being used for parameter estimation

• num_rand_vec_trace: integer (default = 50). Number of random vectors (e.g., Rademacher) for stochastic approximation of the trace of a matrix

• reuse_rand_vec_trace: boolean (default = TRUE). If true, random vectors (e.g., Rademacher) for stochastic approximations of the trace of a matrix are sampled only once at the beginning of the parameter estimation and reused in later trace approximations. Otherwise they are sampled every time a trace is calculated

• seed_rand_vec_trace: integer (default = 1). Seed number to generate random vectors (e.g., Rademacher)

• cg_preconditioner_type (string): Type of preconditioner used for conjugate gradient algorithms.

  • Options for grouped random effects:

    • "ssor" (= default): SSOR preconditioner

    • "incomplete_cholesky": zero fill-in incomplete Cholesky factorization

  • Options for likelihood != "gaussian" and gp_approx == "vecchia" or likelihood == "gaussian" and gp_approx == "vecchia_latent":

    • "vadu" (= default): (B^T * (D^-1 + W) * B) as preconditioner for inverting (B^T * D^-1 * B + W), where B^T * D^-1 * B approx= Sigma^-1

    • "fitc": FITC / modified predictive process preconditioner for inverting (B^-1 * D * B^-T + W^-1)

    • "pivoted_cholesky": (Lk * Lk^T + W^-1) as preconditioner for inverting (B^-1 * D * B^-T + W^-1), where Lk is a low-rank pivoted Cholesky approximation for Sigma and B^-1 * D * B^-T approx= Sigma

    • "incomplete_cholesky": zero fill-in incomplete (reverse) Cholesky factorization of (B^T * D^-1 * B + W) using the sparsity pattern of B^T * D^-1 * B approx= Sigma^-1

  • Options for likelihood != "gaussian" and gp_approx == "full_scale_vecchia":

    • "fitc" ( = default): FITC / modified predictive process preconditioner

    • "vifdu": VIF with diagonal update preconditioner

  • Options for likelihood == "gaussian" and gp_approx == "full_scale_tapering":

    • "fitc" (= default): modified predictive process preconditioner

    • "none": no preconditioner

• fitc_piv_chol_preconditioner_rank (integer ): Rank of the FITC and pivoted Cholesky decomposition preconditioners for iterative methods for Vecchia and VIF approximations (for full_scale_tapering, the same inducing points as in the approximation as used). Internal default values if NULL or < 0:

  • 200 for the FITC preconditioner

  • 50 for the pivoted Cholesky decomposition preconditioner

• convergence_criterion: string (default = "relative_change_in_log_likelihood", only relevant for "gradient_descent", "fisher_scoring", and "newton"). The convergence criterion used for terminating the optimization algorithm. Options: "relative_change_in_log_likelihood" or "relative_change_in_parameters"

• lr_cov: numeric (default = 0.1 for "gradient_descent" and 1. otherwise, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Initial learning rate for covariance parameters if a gradient-based optimization method is used

  • If lr_cov < 0, internal default values are used (0.1 for "gradient_descent" and 1. otherwise)

  • If there are additional auxiliary parameters for non-Gaussian likelihoods, 'lr_cov' is also used for those

  • For "lbfgs", this is divided by the norm of the gradient in the first iteration

• lr_coef: numeric (default = 0.1, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Learning rate for fixed effect regression coefficients if gradient descent is used

• use_nesterov_acc: boolean (default = TRUE, only relevant for "gradient_descent"). If TRUE Nesterov acceleration is used. This is used only for gradient descent

• acc_rate_coef: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for regression coefficients (if there are any) for Nesterov acceleration

• acc_rate_cov: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for covariance parameters for Nesterov acceleration

• momentum_offset: integer (Default = 2, only relevant for "gradient_descent"). Number of iterations for which no momentum is applied in the beginning.

Discontinued. Use the argument gp_approx instead

A string specifying the type of Vecchia approximation used for making predictions. This is discontinued here. Use the function 'set_prediction_data' to specify this

an integer specifying the number of neighbors for making predictions. This is discontinued here. Use the function 'set_prediction_data' to specify this

A numeric vector with additional fixed effects contributions that are added to the linear predictor (= offset). The length of this vector needs to equal the number of training data points.

This is discontinued. Use the renamed equivalent argument offset instead

A numeric specifying an additional parameter for the likelihood which cannot be estimated for this likelihood (e.g., degrees of freedom for likelihood = "t_fix_df"). This is not to be confused with any auxiliary parameters that can be estimated and accessed through the function get_aux_pars after estimation. Note that this likelihood_additional_param parameter is irrelevant for many likelihoods. If likelihood_additional_param = NULL, the following internal default values are used:

• df = 2 for likelihood = "t_fix_df"

A GPModel

A GPModel

A GPModel

A GPModel

A GPModel

A GPModel

A vector containing the outer categorical grouping variable within which the inner_var is nested in. Can be of type integer, double, or character.

A vector containing the inner nested categorical grouping variable

Object of class gpb.Dataset

other parameters

the name of the information field to get (see details)

a matrix object, a dgCMatrix object or a character representing a filename

a list of parameters. See the "Dataset Parameters" section of the parameter documentation for a list of parameters and valid values.

reference dataset. When GPBoost creates a Dataset, it does some preprocessing like binning continuous features into histograms. If you want to apply the same bin boundaries from an existing dataset to new data, pass that existing Dataset to this argument.

names of columns

categorical features. This can either be a character vector of feature names or an integer vector with the indices of the features (e.g. c(1L, 10L) to say "the first and tenth columns").

GPBoost constructs its data format, called a "Dataset", from tabular data. By default, this Dataset object on the R side does keep a copy of the raw data. If you set free_raw_data = TRUE, no copy of the raw data is kept (this reduces memory usage)

a list of information of the gpb.Dataset object

other information to pass to info or parameters pass to params

Object of class gpb.Dataset

gpb.Dataset object, training data

a matrix object, a dgCMatrix object or a character representing a filename

a list of information of the gpb.Dataset object

other information to pass to info.

object of class gpb.Dataset

object filename of output file

object of class gpb.Dataset

categorical features. This can either be a character vector of feature names or an integer vector with the indices of the features (e.g. c(1L, 10L) to say "the first and tenth columns").

object of class gpb.Dataset

object of class gpb.Dataset

A data.frame or data.table to prepare.

A set of rules from the data preparator, if already used. This should be an R list, where names are column names in data and values are named character vectors whose names are column values and whose values are new values to replace them with.

list of "tuning" parameters. See the parameter documentation for more information. A few key parameters:

• learning_rate: The learning rate, also called shrinkage or damping parameter (default = 0.1). An important tuning parameter for boosting. Lower values usually lead to higher predictive accuracy but more boosting iterations are needed

• num_leaves: Number of leaves in a tree. Tuning parameter for tree-boosting (default = 31)

• max_depth: Maximal depth of a tree. Tuning parameter for tree-boosting (default = no limit)

• min_data_in_leaf: Minimal number of samples per leaf. Tuning parameter for tree-boosting (default = 20)

• lambda_l2: L2 regularization (default = 0)

• lambda_l1: L1 regularization (default = 0)

• max_bin: Maximal number of bins that feature values will be bucketed in (default = 255)

• line_search_step_length (default = FALSE): If TRUE, a line search is done to find the optimal step length for every boosting update (see, e.g., Friedman 2001). This is then multiplied by the learning rate

• train_gp_model_cov_pars (default = TRUE): If TRUE, the covariance parameters of the Gaussian process are estimated in every boosting iterations, otherwise the gp_model parameters are not estimated. In the latter case, you need to either estimate them beforehand or provide values via the 'init_cov_pars' parameter when creating the gp_model

• use_gp_model_for_validation (default = TRUE): If TRUE, the Gaussian process is also used (in addition to the tree model) for calculating predictions on the validation data

• leaves_newton_update (default = FALSE): Set this to TRUE to do a Newton update step for the tree leaves after the gradient step. Applies only to Gaussian process boosting (GPBoost algorithm)

• num_threads: Number of threads. For the best speed, set this to the number of real CPU cores(parallel::detectCores(logical = FALSE)), not the number of threads (most CPU using hyper-threading to generate 2 threads per CPU core).

a gpb.Dataset object, used for training. Some functions, such as gpb.cv, may allow you to pass other types of data like matrix and then separately supply label as a keyword argument.

A GPModel object that contains the random effects (Gaussian process and / or grouped random effects) model

number of boosting iterations (= number of trees). This is the most important tuning parameter for boosting

int. Activates early stopping. Requires at least one validation data and one metric. When this parameter is non-null, training will stop if the evaluation of any metric on any validation set fails to improve for early_stopping_rounds consecutive boosting rounds. If training stops early, the returned model will have attribute best_iter set to the iteration number of the best iteration.

list provides a possibility to use a list of pre-defined CV folds (each element must be a vector of test fold's indices). When folds are supplied, the nfold and stratified parameters are ignored.

the original dataset is randomly partitioned into nfold equal size subsamples.

Evaluation metric to be monitored when doing CV and parameter tuning. Can be a character string or vector of character strings. If not NULL, the metric in params will be overridden. Non-exhaustive list of supported metrics: "test_neg_log_likelihood", "mse", "rmse", "mae", "crps_gaussian", "auc", "average_precision", "binary_logloss", "binary_error". See the "metric" section of the parameter documentation for a complete list of valid metrics.

verbosity for output, if <= 0, also will disable the print of evaluation during training

Boolean. If TRUE, the gp_model (Gaussian process and/or random effects) is also used (in addition to the tree model) for calculating predictions on the validation data. If FALSE, the gp_model (random effects part) is ignored for making predictions and only the tree ensemble is used for making predictions for calculating the validation / test error.

Boolean (default = FALSE). If TRUE, the covariance parameters of the gp_model model are estimated using the out-of-sample (OOS) predictions on the validation data using the optimal number of iterations (after performing the CV). This corresponds to the GPBoostOOS algorithm.

Boolean. If TRUE, the covariance parameters of the gp_model (Gaussian process and/or random effects) are estimated in every boosting iterations, otherwise the gp_model parameters are not estimated. In the latter case, you need to either estimate them beforehand or provide the values via the init_cov_pars parameter when creating the gp_model

Vector of labels, used if data is not an gpb.Dataset

vector of response values. If not NULL, will set to dataset

(character) The distribution of the response variable (=label) conditional on fixed and random effects. This only needs to be set when doing independent boosting without random effects / Gaussian processes.

Evaluation metric to be monitored when doing CV and parameter tuning. This can be a string, function, or list with a mixture of strings and functions.

• a. character vector: Non-exhaustive list of supported metrics: "test_neg_log_likelihood", "mse", "rmse", "mae", "auc", "average_precision", "binary_logloss", "binary_error" See the "metric" section of the parameter documentation for a complete list of valid metrics.

• b. function: You can provide a custom evaluation function. This should accept the keyword arguments preds and dtrain and should return a named list with three elements:

  • name: A string with the name of the metric, used for printing and storing results.

  • value: A single number indicating the value of the metric for the given predictions and true values

  • higher_better: A boolean indicating whether higher values indicate a better fit. For example, this would be FALSE for metrics like MAE or RMSE.

• c. list: If a list is given, it should only contain character vectors and functions. These should follow the requirements from the descriptions above.

Boolean, TRUE will record iteration message to booster$record_evals

evaluation output frequency, only effect when verbose > 0

boolean, whether to show standard deviation of cross validation. This parameter defaults to TRUE.

a boolean indicating whether sampling of folds should be stratified by the values of outcome labels.

path of model file of gpb.Booster object, will continue training from this model

feature names, if not null, will use this to overwrite the names in dataset

categorical features. This can either be a character vector of feature names or an integer vector with the indices of the features (e.g. c(1L, 10L) to say "the first and tenth columns").

List of callback functions that are applied at each iteration.

Boolean, setting it to TRUE (not the default value) will transform the booster model into a predictor model which frees up memory and the original datasets

Boolean, setting it to TRUE (not the default value) will delete the boosters of the individual folds

other parameters, see Parameters.rst for more information.

Object of class gpb.Booster

number of iteration want to predict with, NULL or <= 0 means use best iteration

Object of class gpb.Booster

Name of the dataset to return evaluation results for.

Name of the evaluation metric to return results for.

An integer vector of iterations you want to get evaluation results for. If NULL (the default), evaluation results for all iterations will be returned.

TRUE will return evaluation error instead

list with candidate parameters defining the grid over which a search is done

integer with number of random trial on parameter grid. If NULL, a deterministic search is done

a gpb.Dataset object, used for training. Some functions, such as gpb.cv, may allow you to pass other types of data like matrix and then separately supply label as a keyword argument.

A GPModel object that contains the random effects (Gaussian process and / or grouped random effects) model

list with other parameters not included in param_grid

number of boosting iterations (= number of trees). This is the most important tuning parameter for boosting

int. Activates early stopping. Requires at least one validation data and one metric. When this parameter is non-null, training will stop if the evaluation of any metric on any validation set fails to improve for early_stopping_rounds consecutive boosting rounds. If training stops early, the returned model will have attribute best_iter set to the iteration number of the best iteration.

list provides a possibility to use a list of pre-defined CV folds (each element must be a vector of test fold's indices). When folds are supplied, the nfold and stratified parameters are ignored.

the original dataset is randomly partitioned into nfold equal size subsamples.

Evaluation metric to be monitored when doing CV and parameter tuning. Can be a character string or vector of character strings. If not NULL, the metric in params will be overridden. Non-exhaustive list of supported metrics: "test_neg_log_likelihood", "mse", "rmse", "mae", "crps_gaussian", "auc", "average_precision", "binary_logloss", "binary_error". See the "metric" section of the parameter documentation for a complete list of valid metrics.

integer. Whether to display information on the progress of tuning parameter choice. If None or 0, verbose is of. If = 1, summary progress information is displayed for every parameter combination. If >= 2, detailed progress is displayed at every boosting stage for every parameter combination.

Seed for generating folds when doing nfold CV

Boolean. If TRUE, the gp_model (Gaussian process and/or random effects) is also used (in addition to the tree model) for calculating predictions on the validation data. If FALSE, the gp_model (random effects part) is ignored for making predictions and only the tree ensemble is used for making predictions for calculating the validation / test error.

Boolean. If TRUE, the covariance parameters of the gp_model (Gaussian process and/or random effects) are estimated in every boosting iterations, otherwise the gp_model parameters are not estimated. In the latter case, you need to either estimate them beforehand or provide the values via the init_cov_pars parameter when creating the gp_model

Vector of labels, used if data is not an gpb.Dataset

vector of response values. If not NULL, will set to dataset

(character) The distribution of the response variable (=label) conditional on fixed and random effects. This only needs to be set when doing independent boosting without random effects / Gaussian processes.

Evaluation metric to be monitored when doing CV and parameter tuning. This can be a string, function, or list with a mixture of strings and functions.

• a. character vector: Non-exhaustive list of supported metrics: "test_neg_log_likelihood", "mse", "rmse", "mae", "auc", "average_precision", "binary_logloss", "binary_error" See the "metric" section of the parameter documentation for a complete list of valid metrics.

• b. function: You can provide a custom evaluation function. This should accept the keyword arguments preds and dtrain and should return a named list with three elements:

  • name: A string with the name of the metric, used for printing and storing results.

  • value: A single number indicating the value of the metric for the given predictions and true values

  • higher_better: A boolean indicating whether higher values indicate a better fit. For example, this would be FALSE for metrics like MAE or RMSE.

• c. list: If a list is given, it should only contain character vectors and functions. These should follow the requirements from the descriptions above.

a boolean indicating whether sampling of folds should be stratified by the values of outcome labels.

path of model file of gpb.Booster object, will continue training from this model

feature names, if not null, will use this to overwrite the names in dataset

categorical features. This can either be a character vector of feature names or an integer vector with the indices of the features (e.g. c(1L, 10L) to say "the first and tenth columns").

List of callback functions that are applied at each iteration.

a boolean indicating whether all tried parameter combinations are returned

other parameters, see Parameters.rst for more information.

object of class gpb.Booster.

whether to show importance in relative percentage.

object of class gpb.Booster.

a matrix object or a dgCMatrix object.

an integer vector of indices of rows needed.

number of iteration want to predict with, NULL or <= 0 means use best iteration.

path of model file

a str containing the model

object of class gpb.Booster

number of iterations you want to predict with. NULL or <= 0 means use best iteration

a data.table returned by gpb.importance.

maximal number of top features to include into the plot.

the name of importance measure to plot, can be "Gain", "Cover" or "Frequency".

(base R barplot) allows to adjust the left margin size to fit feature names.

(base R barplot) passed as cex.names parameter to barplot. Set a number smaller than 1.0 to make the bar labels smaller than R's default and values greater than 1.0 to make them larger.

other parameters passed to graphics::barplot

a data.table returned by gpb.interprete.

maximal number of top features to include into the plot.

the column numbers of layout, will be used only for multiclass classification feature contribution.

(base R barplot) allows to adjust the left margin size to fit feature names.

(base R barplot) passed as cex.names parameter to barplot.

A gpb.Booster model object

A matrix with data for creating partial dependence plots

A vector of length two of type string with names of the columns or integer with indices of the columns in data for which an interaction dependence plot is created

Number of grid points per variable (used only if a variable is not discrete) For continuous variables, the two-dimensional grid for the interaction plot has dimension c(n.pt.per.var, n.pt.per.var)

Fraction of random samples in data to be used for calculating the partial dependence plot

A vector of length two of type boolean. If an entry is TRUE, the evaluation grid of the corresponding variable is set to the unique values of the variable

An integer indicating the class in multi-class classification (value from 0 to num_class - 1)

A character string indicating the type of the plot. Supported values: "filled.contour" and "contour"

Parameter passed to the filled.contour or contour function

Parameter passed to the filled.contour or contour function

Parameter passed to the filled.contour or contour function

Parameter passed to the filled.contour or contour function

Parameter passed to the filled.contour or contour function

A boolean. If TRUE, the data for creating the partial dependence plot is returned

Additional parameters passed to the filled.contour or contour function

A gpb.Booster model object

A matrix with data for creating partial dependence plots

A string with a name of the column or an integer with an index of the column in data for which a dependence plot is created

Evaluation grid size (used only if x is not discrete)

Fraction of random samples in data to be used for calculating the partial dependence plot

A boolean. If TRUE, the evaluation grid is set to the unique values of x

An integer indicating the class in multi-class classification (value from 0 to num_class - 1)

Parameter passed to plot

Parameter passed to plot

Parameter passed to plot

Parameter passed to plot

A boolean. If TRUE, the data for creating the partial dependence plot is returned

Additional parameters passed to plot

Object of class gpb.Booster

saved filename

int or NULL, optional (default=NULL) Start index of the iteration to predict. If NULL or <= 0, starts from the first iteration.

int or NULL, optional (default=NULL) Limit number of iterations in the prediction. If NULL, if the best iteration exists and start_iteration is NULL or <= 0, the best iteration is used; otherwise, all iterations from start_iteration are used. If <= 0, all iterations from start_iteration are used (no limits).

If TRUE, the raw data (predictor / covariate data) for the Booster is also saved. Enable this option if you want to change start_iteration or num_iteration at prediction time after loading.

Additional named arguments passed to the predict() method of the gpb.Booster object passed to object. This is only used when there is a gp_model and when save_raw_data=FALSE

list of "tuning" parameters. See the parameter documentation for more information. A few key parameters:

• learning_rate: The learning rate, also called shrinkage or damping parameter (default = 0.1). An important tuning parameter for boosting. Lower values usually lead to higher predictive accuracy but more boosting iterations are needed

• num_leaves: Number of leaves in a tree. Tuning parameter for tree-boosting (default = 31)

• max_depth: Maximal depth of a tree. Tuning parameter for tree-boosting (default = no limit)

• min_data_in_leaf: Minimal number of samples per leaf. Tuning parameter for tree-boosting (default = 20)

• lambda_l2: L2 regularization (default = 0)

• lambda_l1: L1 regularization (default = 0)

• max_bin: Maximal number of bins that feature values will be bucketed in (default = 255)

• line_search_step_length (default = FALSE): If TRUE, a line search is done to find the optimal step length for every boosting update (see, e.g., Friedman 2001). This is then multiplied by the learning rate

• train_gp_model_cov_pars (default = TRUE): If TRUE, the covariance parameters of the Gaussian process are estimated in every boosting iterations, otherwise the gp_model parameters are not estimated. In the latter case, you need to either estimate them beforehand or provide values via the 'init_cov_pars' parameter when creating the gp_model

• use_gp_model_for_validation (default = TRUE): If TRUE, the Gaussian process is also used (in addition to the tree model) for calculating predictions on the validation data

• leaves_newton_update (default = FALSE): Set this to TRUE to do a Newton update step for the tree leaves after the gradient step. Applies only to Gaussian process boosting (GPBoost algorithm)

• num_threads: Number of threads. For the best speed, set this to the number of real CPU cores(parallel::detectCores(logical = FALSE)), not the number of threads (most CPU using hyper-threading to generate 2 threads per CPU core).

a gpb.Dataset object, used for training. Some functions, such as gpb.cv, may allow you to pass other types of data like matrix and then separately supply label as a keyword argument.

number of boosting iterations (= number of trees). This is the most important tuning parameter for boosting

A GPModel object that contains the random effects (Gaussian process and / or grouped random effects) model

Boolean. If TRUE, the gp_model (Gaussian process and/or random effects) is also used (in addition to the tree model) for calculating predictions on the validation data. If FALSE, the gp_model (random effects part) is ignored for making predictions and only the tree ensemble is used for making predictions for calculating the validation / test error.

Boolean. If TRUE, the covariance parameters of the gp_model (Gaussian process and/or random effects) are estimated in every boosting iterations, otherwise the gp_model parameters are not estimated. In the latter case, you need to either estimate them beforehand or provide the values via the init_cov_pars parameter when creating the gp_model

a list of gpb.Dataset objects, used for validation

(character) The distribution of the response variable (=label) conditional on fixed and random effects. This only needs to be set when doing independent boosting without random effects / Gaussian processes.

Evaluation metric to be monitored when doing CV and parameter tuning. This can be a string, function, or list with a mixture of strings and functions.

• a. character vector: Non-exhaustive list of supported metrics: "test_neg_log_likelihood", "mse", "rmse", "mae", "auc", "average_precision", "binary_logloss", "binary_error" See the "metric" section of the parameter documentation for a complete list of valid metrics.

• b. function: You can provide a custom evaluation function. This should accept the keyword arguments preds and dtrain and should return a named list with three elements:

  • name: A string with the name of the metric, used for printing and storing results.

  • value: A single number indicating the value of the metric for the given predictions and true values

  • higher_better: A boolean indicating whether higher values indicate a better fit. For example, this would be FALSE for metrics like MAE or RMSE.

• c. list: If a list is given, it should only contain character vectors and functions. These should follow the requirements from the descriptions above.

verbosity for output, if <= 0, also will disable the print of evaluation during training

Boolean, TRUE will record iteration message to booster$record_evals

evaluation output frequency, only effect when verbose > 0

path of model file of gpb.Booster object, will continue training from this model

feature names, if not null, will use this to overwrite the names in dataset

categorical features. This can either be a character vector of feature names or an integer vector with the indices of the features (e.g. c(1L, 10L) to say "the first and tenth columns").

int. Activates early stopping. Requires at least one validation data and one metric. When this parameter is non-null, training will stop if the evaluation of any metric on any validation set fails to improve for early_stopping_rounds consecutive boosting rounds. If training stops early, the returned model will have attribute best_iter set to the iteration number of the best iteration.

List of callback functions that are applied at each iteration.

Boolean, setting it to TRUE (not the default value) will transform the booster model into a predictor model which frees up memory and the original datasets

other parameters, see the parameter documentation for more information.

List of callback functions that are applied at each iteration.

a gpb.Dataset object, used for training. Some functions, such as gpb.cv, may allow you to pass other types of data like matrix and then separately supply label as a keyword argument.

list provides a possibility to use a list of pre-defined CV folds (each element must be a vector of test fold's indices). When folds are supplied, the nfold and stratified parameters are ignored.

the original dataset is randomly partitioned into nfold equal size subsamples.

Seed for generating folds when doing nfold CV

int. Activates early stopping. Requires at least one validation data and one metric. When this parameter is non-null, training will stop if the evaluation of any metric on any validation set fails to improve for early_stopping_rounds consecutive boosting rounds. If training stops early, the returned model will have attribute best_iter set to the iteration number of the best iteration.

Evaluation metric to be monitored when doing CV and parameter tuning. Can be a character string or vector of character strings. If not NULL, the metric in params will be overridden. Non-exhaustive list of supported metrics: "test_neg_log_likelihood", "mse", "rmse", "mae", "crps_gaussian", "auc", "average_precision", "binary_logloss", "binary_error". See the "metric" section of the parameter documentation for a complete list of valid metrics.

integer. Whether to display information on the progress of tuning parameter choice. If None or 0, verbose is of. If = 1, summary progress information is displayed for every parameter combination. If >= 2, detailed progress is displayed at every boosting stage for every parameter combination.

Evaluation metric to be monitored when doing CV and parameter tuning. This can be a string, function, or list with a mixture of strings and functions.

• a. character vector: Non-exhaustive list of supported metrics: "test_neg_log_likelihood", "mse", "rmse", "mae", "auc", "average_precision", "binary_logloss", "binary_error" See the "metric" section of the parameter documentation for a complete list of valid metrics.

• b. function: You can provide a custom evaluation function. This should accept the keyword arguments preds and dtrain and should return a named list with three elements:

  • name: A string with the name of the metric, used for printing and storing results.

  • value: A single number indicating the value of the metric for the given predictions and true values

  • higher_better: A boolean indicating whether higher values indicate a better fit. For example, this would be FALSE for metrics like MAE or RMSE.

• c. list: If a list is given, it should only contain character vectors and functions. These should follow the requirements from the descriptions above.

evaluation output frequency, only effect when verbose > 0

a list of gpb.Dataset objects, used for validation

Boolean, TRUE will record iteration message to booster$record_evals

feature names, if not null, will use this to overwrite the names in dataset

categorical features. This can either be a character vector of feature names or an integer vector with the indices of the features (e.g. c(1L, 10L) to say "the first and tenth columns").

path of model file of gpb.Booster object, will continue training from this model

number of boosting iterations (= number of trees). This is the most important tuning parameter for boosting

(character) The distribution of the response variable (=label) conditional on fixed and random effects. This only needs to be set when doing independent boosting without random effects / Gaussian processes.

list of "tuning" parameters. See the parameter documentation for more information. A few key parameters:

• learning_rate: The learning rate, also called shrinkage or damping parameter (default = 0.1). An important tuning parameter for boosting. Lower values usually lead to higher predictive accuracy but more boosting iterations are needed

• num_leaves: Number of leaves in a tree. Tuning parameter for tree-boosting (default = 31)

• max_depth: Maximal depth of a tree. Tuning parameter for tree-boosting (default = no limit)

• min_data_in_leaf: Minimal number of samples per leaf. Tuning parameter for tree-boosting (default = 20)

• lambda_l2: L2 regularization (default = 0)

• lambda_l1: L1 regularization (default = 0)

• max_bin: Maximal number of bins that feature values will be bucketed in (default = 255)

• line_search_step_length (default = FALSE): If TRUE, a line search is done to find the optimal step length for every boosting update (see, e.g., Friedman 2001). This is then multiplied by the learning rate

• train_gp_model_cov_pars (default = TRUE): If TRUE, the covariance parameters of the Gaussian process are estimated in every boosting iterations, otherwise the gp_model parameters are not estimated. In the latter case, you need to either estimate them beforehand or provide values via the 'init_cov_pars' parameter when creating the gp_model

• use_gp_model_for_validation (default = TRUE): If TRUE, the Gaussian process is also used (in addition to the tree model) for calculating predictions on the validation data

• leaves_newton_update (default = FALSE): Set this to TRUE to do a Newton update step for the tree leaves after the gradient step. Applies only to Gaussian process boosting (GPBoost algorithm)

• num_threads: Number of threads. For the best speed, set this to the number of real CPU cores(parallel::detectCores(logical = FALSE)), not the number of threads (most CPU using hyper-threading to generate 2 threads per CPU core).

verbosity for output, if <= 0, also will disable the print of evaluation during training

A GPModel object that contains the random effects (Gaussian process and / or grouped random effects) model

Boolean. If TRUE, a line search is done to find the optimal step length for every boosting update (see, e.g., Friedman 2001). This is then multiplied by the learning_rate. Applies only to the GPBoost algorithm

Boolean. If TRUE, the gp_model (Gaussian process and/or random effects) is also used (in addition to the tree model) for calculating predictions on the validation data. If FALSE, the gp_model (random effects part) is ignored for making predictions and only the tree ensemble is used for making predictions for calculating the validation / test error.

Boolean. If TRUE, the covariance parameters of the gp_model (Gaussian process and/or random effects) are estimated in every boosting iterations, otherwise the gp_model parameters are not estimated. In the latter case, you need to either estimate them beforehand or provide the values via the init_cov_pars parameter when creating the gp_model

a gpb.Dataset object, used for training. Some functions, such as gpb.cv, may allow you to pass other types of data like matrix and then separately supply label as a keyword argument.

Vector of response values / labels, used if data is not an gpb.Dataset

Vector of weights. The GPBoost algorithm currently does not support weights

list of "tuning" parameters. See the parameter documentation for more information. A few key parameters:

• learning_rate: The learning rate, also called shrinkage or damping parameter (default = 0.1). An important tuning parameter for boosting. Lower values usually lead to higher predictive accuracy but more boosting iterations are needed

• num_leaves: Number of leaves in a tree. Tuning parameter for tree-boosting (default = 31)

• max_depth: Maximal depth of a tree. Tuning parameter for tree-boosting (default = no limit)

• min_data_in_leaf: Minimal number of samples per leaf. Tuning parameter for tree-boosting (default = 20)

• lambda_l2: L2 regularization (default = 0)

• lambda_l1: L1 regularization (default = 0)

• max_bin: Maximal number of bins that feature values will be bucketed in (default = 255)

• line_search_step_length (default = FALSE): If TRUE, a line search is done to find the optimal step length for every boosting update (see, e.g., Friedman 2001). This is then multiplied by the learning rate

• train_gp_model_cov_pars (default = TRUE): If TRUE, the covariance parameters of the Gaussian process are estimated in every boosting iterations, otherwise the gp_model parameters are not estimated. In the latter case, you need to either estimate them beforehand or provide values via the 'init_cov_pars' parameter when creating the gp_model

• use_gp_model_for_validation (default = TRUE): If TRUE, the Gaussian process is also used (in addition to the tree model) for calculating predictions on the validation data

• leaves_newton_update (default = FALSE): Set this to TRUE to do a Newton update step for the tree leaves after the gradient step. Applies only to Gaussian process boosting (GPBoost algorithm)

• num_threads: Number of threads. For the best speed, set this to the number of real CPU cores(parallel::detectCores(logical = FALSE)), not the number of threads (most CPU using hyper-threading to generate 2 threads per CPU core).

number of boosting iterations (= number of trees). This is the most important tuning parameter for boosting

A GPModel object that contains the random effects (Gaussian process and / or grouped random effects) model

Boolean. If TRUE, the gp_model (Gaussian process and/or random effects) is also used (in addition to the tree model) for calculating predictions on the validation data. If FALSE, the gp_model (random effects part) is ignored for making predictions and only the tree ensemble is used for making predictions for calculating the validation / test error.

Boolean. If TRUE, the covariance parameters of the gp_model (Gaussian process and/or random effects) are estimated in every boosting iterations, otherwise the gp_model parameters are not estimated. In the latter case, you need to either estimate them beforehand or provide the values via the init_cov_pars parameter when creating the gp_model

a list of gpb.Dataset objects, used for validation

(character) The distribution of the response variable (=label) conditional on fixed and random effects. This only needs to be set when doing independent boosting without random effects / Gaussian processes.

Evaluation metric to be monitored when doing CV and parameter tuning. This can be a string, function, or list with a mixture of strings and functions.

• a. character vector: Non-exhaustive list of supported metrics: "test_neg_log_likelihood", "mse", "rmse", "mae", "auc", "average_precision", "binary_logloss", "binary_error" See the "metric" section of the parameter documentation for a complete list of valid metrics.

• b. function: You can provide a custom evaluation function. This should accept the keyword arguments preds and dtrain and should return a named list with three elements:

  • name: A string with the name of the metric, used for printing and storing results.

  • value: A single number indicating the value of the metric for the given predictions and true values

  • higher_better: A boolean indicating whether higher values indicate a better fit. For example, this would be FALSE for metrics like MAE or RMSE.

• c. list: If a list is given, it should only contain character vectors and functions. These should follow the requirements from the descriptions above.

verbosity for output, if <= 0, also will disable the print of evaluation during training

Boolean, TRUE will record iteration message to booster$record_evals

evaluation output frequency, only effect when verbose > 0

int. Activates early stopping. Requires at least one validation data and one metric. When this parameter is non-null, training will stop if the evaluation of any metric on any validation set fails to improve for early_stopping_rounds consecutive boosting rounds. If training stops early, the returned model will have attribute best_iter set to the iteration number of the best iteration.

path of model file of gpb.Booster object, will continue training from this model

feature names, if not null, will use this to overwrite the names in dataset

categorical features. This can either be a character vector of feature names or an integer vector with the indices of the features (e.g. c(1L, 10L) to say "the first and tenth columns").

List of callback functions that are applied at each iteration.

Additional arguments passed to gpb.train. For example

• valids: a list of gpb.Dataset objects, used for validation

• eval: evaluation function, can be (a list of) character or custom eval function

• record: Boolean, TRUE will record iteration message to booster$record_evals

• colnames: feature names, if not null, will use this to overwrite the names in dataset

• categorical_feature: categorical features. This can either be a character vector of feature names or an integer vector with the indices of the features (e.g. c(1L, 10L) to say "the first and tenth columns").

• reset_data: Boolean, setting it to TRUE (not the default value) will transform the booster model into a predictor model which frees up memory and the original datasets

filename for loading

A GPModel

A vector with numeric elements. Covariance parameters of Gaussian process and random effects

A vector with response variable data

A numeric vector with fixed effects, e.g., containing a linear predictor. The length of this vector needs to equal the number of training data points.

A vector with numeric elements. Additional parameters for non-Gaussian likelihoods (e.g., shape parameter of a gamma or negative_binomial likelihood)

A GPModel

A vector with numeric elements. Covariance parameters of Gaussian process and random effects

A vector with response variable data

A numeric vector with fixed effects, e.g., containing a linear predictor. The length of this vector needs to equal the number of training data points.

A vector with numeric elements. Additional parameters for non-Gaussian likelihoods (e.g., shape parameter of a gamma or negative_binomial likelihood)

a GPModel

A boolean. If TRUE, the response variable (label) is predicted, otherwise the latent random effects

A boolean. If TRUE, the (posterior) predictive variances are calculated

A boolean. If TRUE, the (posterior) predictive covariance is calculated in addition to the (posterior) predictive mean

Observed data (can be NULL, e.g. when the model has been estimated already and the same data is used for making predictions)

A vector containing covariance parameters which are used if the GPModel has not been trained or if predictions should be made for other parameters than the trained ones

A vector or matrix with elements being group levels for which predictions are made (if there are grouped random effects in the GPModel)

A vector or matrix with covariate data for grouped random coefficients (if there are some in the GPModel)

A matrix with prediction coordinates (=features) for Gaussian process (if there is a GP in the GPModel)

A vector or matrix with covariate data for Gaussian process random coefficients (if there are some in the GPModel)

A vector with elements indicating the realizations of random effects / Gaussian processes for which predictions are made (set to NULL if you have not specified this when creating the GPModel)

A matrix with prediction covariate data for the fixed effects linear regression term (if there is one in the GPModel)

A boolean. If TRUE, predictions are done using a priory set data via the function '$set_prediction_data' (this option is not used by users directly)

A numeric vector with additional fixed effects contributions that are added to the linear predictor (= offset). The length of this vector needs to equal the number of training data points.

A numeric vector with additional fixed effects contributions that are added to the linear predictor for the prediction points (= offset). The length of this vector needs to equal the number of prediction points.

This is discontinued. Use the renamed equivalent argument offset instead

This is discontinued. Use the renamed equivalent argument offset_pred instead

A string specifying the type of Vecchia approximation used for making predictions. This is discontinued here. Use the function 'set_prediction_data' to specify this

an integer specifying the number of neighbors for making predictions. This is discontinued here. Use the function 'set_prediction_data' to specify this

(not used, ignore this, simply here that there is no CRAN warning)

Object of class gpb.Booster

a matrix object, a dgCMatrix object or a character representing a filename

int or NULL, optional (default=NULL) Start index of the iteration to predict. If NULL or <= 0, starts from the first iteration.

int or NULL, optional (default=NULL) Limit number of iterations in the prediction. If NULL, if the best iteration exists and start_iteration is NULL or <= 0, the best iteration is used; otherwise, all iterations from start_iteration are used. If <= 0, all iterations from start_iteration are used (no limits).

If TRUE latent variables, both fixed effects (tree-ensemble) and random effects (gp_model) are predicted. Otherwise, the response variable (label) is predicted. Depending on how the argument 'pred_latent' is set, different values are returned from this function; see the 'Value' section for more details. If there is no gp_model, this argument corresponds to 'raw_score' in LightGBM.

whether predict leaf index instead.

return per-feature contributions for each record.

only used for prediction for text file. True if text file has header

whether to reshape the vector of predictions to a matrix form when there are several prediction outputs per case.

A vector or matrix with elements being group levels for which predictions are made (if there are grouped random effects in the GPModel)

A vector or matrix with covariate data for grouped random coefficients (if there are some in the GPModel)

A matrix with prediction coordinates (=features) for Gaussian process (if there is a GP in the GPModel)

A vector or matrix with covariate data for Gaussian process random coefficients (if there are some in the GPModel)

A vector with elements indicating the realizations of random effects / Gaussian processes for which predictions are made (set to NULL if you have not specified this when creating the GPModel)

A boolean. If TRUE, the (posterior) predictive covariance is calculated in addition to the (posterior) predictive mean

A boolean. If TRUE, the (posterior) predictive variances are calculated

A vector containing covariance parameters which are used if the gp_model has not been trained or if predictions should be made for other parameters than the trained ones

A boolean. If TRUE, predictions are only made for the tree ensemble part and the gp_model is ignored

This is discontinued. Use the renamed equivalent argument pred_latent instead

A string specifying the type of Vecchia approximation used for making predictions. This is discontinued here. Use the function 'set_prediction_data' to specify this

an integer specifying the number of neighbors for making predictions. This is discontinued here. Use the function 'set_prediction_data' to specify this

Additional named arguments passed to the predict() method of the gpb.Booster object passed to object.

A GPModel

A boolean. If TRUE, the (posterior) predictive variances are calculated

A GPModel

A boolean. If TRUE, the (posterior) predictive variances are calculated

a connection or the name of the file where the R object is saved to or read from.

a hook function for handling reference objects.

a GPModel

filename for saving

R object to serialize.

a connection or the name of the file where the R object is saved to or read from.

a logical. If TRUE or NA, an ASCII representation is written; otherwise (default), a binary one is used. See the comments in the help for save.

the workspace format version to use. NULL specifies the current default version (2). Versions prior to 2 are not supported, so this will only be relevant when there are later versions.

a logical specifying whether saving to a named file is to use "gzip" compression, or one of "gzip", "bzip2" or "xz" to indicate the type of compression to be used. Ignored if file is a connection.

a hook function for handling reference objects.

whether to save the model in a raw variable or not, recommended to leave it to TRUE.

A GPModel

A list with parameters for the estimation / optimization

• trace: boolean (default = FALSE). If TRUE, information on the progress of the parameter optimization is printed

• std_dev: boolean (default = TRUE). If TRUE, approximate standard deviations are calculated for the covariance and linear regression parameters (= square root of diagonal of the inverse Fisher information for Gaussian likelihoods and square root of diagonal of a numerically approximated inverse Hessian for non-Gaussian likelihoods)

• init_cov_pars: vector with numeric elements (default = NULL). Initial values for covariance parameters of Gaussian process and random effects (can be NULL). The order is same as the order of the parameters in the summary function: first is the error variance (only for "gaussian" likelihood), next follow the variances of the grouped random effects (if there are any, in the order provided in 'group_data'), and then follow the marginal variance and the ranges of the Gaussian process. If there are multiple Gaussian processes, then the variances and ranges follow alternatingly. If 'init_cov_pars = NULL', an internal choice is used that depends on the likelihood and the random effects type and covariance function. If you select the option 'trace = TRUE' in the 'params' argument, you will see the first initial covariance parameters in iteration 0.

• init_coef: vector with numeric elements (default = NULL). Initial values for the regression coefficients (if there are any, can be NULL)

• init_aux_pars: vector with numeric elements (default = NULL). Initial values for additional parameters for non-Gaussian likelihoods (e.g., shape parameter of a gamma or negative_binomial likelihood)

• estimate_cov_par_index: vector with integer (default = -1). If estimate_cov_par_index[1] >= 0, some covariance parameters might not be estimated, estimate_cov_par_index[i] is then bool and indicates which ones are estimated. For instance, estimate_cov_par_index = c(1,1,0) means that the first two covariance parameters are estimated and the last one not.

• estimate_aux_pars: boolean (default = TRUE). If TRUE, additional parameters for non-Gaussian likelihoods are also estimated (e.g., shape parameter of a gamma or negative_binomial likelihood)

• optimizer_cov: string (default = "lbfgs"). Optimizer used for estimating covariance parameters. Options: "lbfgs", "gradient_descent", "fisher_scoring", "newton", "nelder_mead". If there are additional auxiliary parameters for non-Gaussian likelihoods, 'optimizer_cov' is also used for those

• optimizer_coef: string (default = "wls" for Gaussian likelihoods and "lbfgs" for other likelihoods). Optimizer used for estimating linear regression coefficients, if there are any (for the GPBoost algorithm there are usually none). Options: "gradient_descent", "lbfgs", "wls", "nelder_mead". Gradient descent steps are done simultaneously with gradient descent steps for the covariance parameters. "wls" refers to doing coordinate descent for the regression coefficients using weighted least squares. If 'optimizer_cov' is set to "nelder_mead" or "lbfgs", 'optimizer_coef' is automatically also set to the same value.

• maxit: integer (default = 1000). Maximal number of iterations for optimization algorithm

• delta_rel_conv: numeric (default = 1E-6 except for "nelder_mead" for which the default is 1E-8). Convergence tolerance. The algorithm stops if the relative change in either the (approximate) log-likelihood or the parameters is below this value. If < 0, internal default values are used

• cg_max_num_it: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithms

• cg_max_num_it_tridiag: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithm when being run as Lanczos algorithm for tridiagonalization

• cg_delta_conv: numeric (default = 1E-2). Tolerance level for L2 norm of residuals for checking convergence in conjugate gradient algorithm when being used for parameter estimation

• num_rand_vec_trace: integer (default = 50). Number of random vectors (e.g., Rademacher) for stochastic approximation of the trace of a matrix

• reuse_rand_vec_trace: boolean (default = TRUE). If true, random vectors (e.g., Rademacher) for stochastic approximations of the trace of a matrix are sampled only once at the beginning of the parameter estimation and reused in later trace approximations. Otherwise they are sampled every time a trace is calculated

• seed_rand_vec_trace: integer (default = 1). Seed number to generate random vectors (e.g., Rademacher)

• cg_preconditioner_type (string): Type of preconditioner used for conjugate gradient algorithms.

  • Options for grouped random effects:

    • "ssor" (= default): SSOR preconditioner

    • "incomplete_cholesky": zero fill-in incomplete Cholesky factorization

  • Options for likelihood != "gaussian" and gp_approx == "vecchia" or likelihood == "gaussian" and gp_approx == "vecchia_latent":

    • "vadu" (= default): (B^T * (D^-1 + W) * B) as preconditioner for inverting (B^T * D^-1 * B + W), where B^T * D^-1 * B approx= Sigma^-1

    • "fitc": FITC / modified predictive process preconditioner for inverting (B^-1 * D * B^-T + W^-1)

    • "pivoted_cholesky": (Lk * Lk^T + W^-1) as preconditioner for inverting (B^-1 * D * B^-T + W^-1), where Lk is a low-rank pivoted Cholesky approximation for Sigma and B^-1 * D * B^-T approx= Sigma

    • "incomplete_cholesky": zero fill-in incomplete (reverse) Cholesky factorization of (B^T * D^-1 * B + W) using the sparsity pattern of B^T * D^-1 * B approx= Sigma^-1

  • Options for likelihood != "gaussian" and gp_approx == "full_scale_vecchia":

    • "fitc" ( = default): FITC / modified predictive process preconditioner

    • "vifdu": VIF with diagonal update preconditioner

  • Options for likelihood == "gaussian" and gp_approx == "full_scale_tapering":

    • "fitc" (= default): modified predictive process preconditioner

    • "none": no preconditioner

• fitc_piv_chol_preconditioner_rank (integer ): Rank of the FITC and pivoted Cholesky decomposition preconditioners for iterative methods for Vecchia and VIF approximations (for full_scale_tapering, the same inducing points as in the approximation as used). Internal default values if NULL or < 0:

  • 200 for the FITC preconditioner

  • 50 for the pivoted Cholesky decomposition preconditioner

• convergence_criterion: string (default = "relative_change_in_log_likelihood", only relevant for "gradient_descent", "fisher_scoring", and "newton"). The convergence criterion used for terminating the optimization algorithm. Options: "relative_change_in_log_likelihood" or "relative_change_in_parameters"

• lr_cov: numeric (default = 0.1 for "gradient_descent" and 1. otherwise, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Initial learning rate for covariance parameters if a gradient-based optimization method is used

  • If lr_cov < 0, internal default values are used (0.1 for "gradient_descent" and 1. otherwise)

  • If there are additional auxiliary parameters for non-Gaussian likelihoods, 'lr_cov' is also used for those

  • For "lbfgs", this is divided by the norm of the gradient in the first iteration

• lr_coef: numeric (default = 0.1, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Learning rate for fixed effect regression coefficients if gradient descent is used

• use_nesterov_acc: boolean (default = TRUE, only relevant for "gradient_descent"). If TRUE Nesterov acceleration is used. This is used only for gradient descent

• acc_rate_coef: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for regression coefficients (if there are any) for Nesterov acceleration

• acc_rate_cov: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for covariance parameters for Nesterov acceleration

• momentum_offset: integer (Default = 2, only relevant for "gradient_descent"). Number of iterations for which no momentum is applied in the beginning.

A GPModel

A list with parameters for the estimation / optimization

• trace: boolean (default = FALSE). If TRUE, information on the progress of the parameter optimization is printed

• std_dev: boolean (default = TRUE). If TRUE, approximate standard deviations are calculated for the covariance and linear regression parameters (= square root of diagonal of the inverse Fisher information for Gaussian likelihoods and square root of diagonal of a numerically approximated inverse Hessian for non-Gaussian likelihoods)

• init_cov_pars: vector with numeric elements (default = NULL). Initial values for covariance parameters of Gaussian process and random effects (can be NULL). The order is same as the order of the parameters in the summary function: first is the error variance (only for "gaussian" likelihood), next follow the variances of the grouped random effects (if there are any, in the order provided in 'group_data'), and then follow the marginal variance and the ranges of the Gaussian process. If there are multiple Gaussian processes, then the variances and ranges follow alternatingly. If 'init_cov_pars = NULL', an internal choice is used that depends on the likelihood and the random effects type and covariance function. If you select the option 'trace = TRUE' in the 'params' argument, you will see the first initial covariance parameters in iteration 0.

• init_coef: vector with numeric elements (default = NULL). Initial values for the regression coefficients (if there are any, can be NULL)

• init_aux_pars: vector with numeric elements (default = NULL). Initial values for additional parameters for non-Gaussian likelihoods (e.g., shape parameter of a gamma or negative_binomial likelihood)

• estimate_cov_par_index: vector with integer (default = -1). If estimate_cov_par_index[1] >= 0, some covariance parameters might not be estimated, estimate_cov_par_index[i] is then bool and indicates which ones are estimated. For instance, estimate_cov_par_index = c(1,1,0) means that the first two covariance parameters are estimated and the last one not.

• estimate_aux_pars: boolean (default = TRUE). If TRUE, additional parameters for non-Gaussian likelihoods are also estimated (e.g., shape parameter of a gamma or negative_binomial likelihood)

• optimizer_cov: string (default = "lbfgs"). Optimizer used for estimating covariance parameters. Options: "lbfgs", "gradient_descent", "fisher_scoring", "newton", "nelder_mead". If there are additional auxiliary parameters for non-Gaussian likelihoods, 'optimizer_cov' is also used for those

• optimizer_coef: string (default = "wls" for Gaussian likelihoods and "lbfgs" for other likelihoods). Optimizer used for estimating linear regression coefficients, if there are any (for the GPBoost algorithm there are usually none). Options: "gradient_descent", "lbfgs", "wls", "nelder_mead". Gradient descent steps are done simultaneously with gradient descent steps for the covariance parameters. "wls" refers to doing coordinate descent for the regression coefficients using weighted least squares. If 'optimizer_cov' is set to "nelder_mead" or "lbfgs", 'optimizer_coef' is automatically also set to the same value.

• maxit: integer (default = 1000). Maximal number of iterations for optimization algorithm

• delta_rel_conv: numeric (default = 1E-6 except for "nelder_mead" for which the default is 1E-8). Convergence tolerance. The algorithm stops if the relative change in either the (approximate) log-likelihood or the parameters is below this value. If < 0, internal default values are used

• cg_max_num_it: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithms

• cg_max_num_it_tridiag: integer (default = 1000). Maximal number of iterations for conjugate gradient algorithm when being run as Lanczos algorithm for tridiagonalization

• cg_delta_conv: numeric (default = 1E-2). Tolerance level for L2 norm of residuals for checking convergence in conjugate gradient algorithm when being used for parameter estimation

• num_rand_vec_trace: integer (default = 50). Number of random vectors (e.g., Rademacher) for stochastic approximation of the trace of a matrix

• reuse_rand_vec_trace: boolean (default = TRUE). If true, random vectors (e.g., Rademacher) for stochastic approximations of the trace of a matrix are sampled only once at the beginning of the parameter estimation and reused in later trace approximations. Otherwise they are sampled every time a trace is calculated

• seed_rand_vec_trace: integer (default = 1). Seed number to generate random vectors (e.g., Rademacher)

• cg_preconditioner_type (string): Type of preconditioner used for conjugate gradient algorithms.

  • Options for grouped random effects:

    • "ssor" (= default): SSOR preconditioner

    • "incomplete_cholesky": zero fill-in incomplete Cholesky factorization

  • Options for likelihood != "gaussian" and gp_approx == "vecchia" or likelihood == "gaussian" and gp_approx == "vecchia_latent":

    • "vadu" (= default): (B^T * (D^-1 + W) * B) as preconditioner for inverting (B^T * D^-1 * B + W), where B^T * D^-1 * B approx= Sigma^-1

    • "fitc": FITC / modified predictive process preconditioner for inverting (B^-1 * D * B^-T + W^-1)

    • "pivoted_cholesky": (Lk * Lk^T + W^-1) as preconditioner for inverting (B^-1 * D * B^-T + W^-1), where Lk is a low-rank pivoted Cholesky approximation for Sigma and B^-1 * D * B^-T approx= Sigma

    • "incomplete_cholesky": zero fill-in incomplete (reverse) Cholesky factorization of (B^T * D^-1 * B + W) using the sparsity pattern of B^T * D^-1 * B approx= Sigma^-1

  • Options for likelihood != "gaussian" and gp_approx == "full_scale_vecchia":

    • "fitc" ( = default): FITC / modified predictive process preconditioner

    • "vifdu": VIF with diagonal update preconditioner

  • Options for likelihood == "gaussian" and gp_approx == "full_scale_tapering":

    • "fitc" (= default): modified predictive process preconditioner

    • "none": no preconditioner

• fitc_piv_chol_preconditioner_rank (integer ): Rank of the FITC and pivoted Cholesky decomposition preconditioners for iterative methods for Vecchia and VIF approximations (for full_scale_tapering, the same inducing points as in the approximation as used). Internal default values if NULL or < 0:

  • 200 for the FITC preconditioner

  • 50 for the pivoted Cholesky decomposition preconditioner

• convergence_criterion: string (default = "relative_change_in_log_likelihood", only relevant for "gradient_descent", "fisher_scoring", and "newton"). The convergence criterion used for terminating the optimization algorithm. Options: "relative_change_in_log_likelihood" or "relative_change_in_parameters"

• lr_cov: numeric (default = 0.1 for "gradient_descent" and 1. otherwise, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Initial learning rate for covariance parameters if a gradient-based optimization method is used

  • If lr_cov < 0, internal default values are used (0.1 for "gradient_descent" and 1. otherwise)

  • If there are additional auxiliary parameters for non-Gaussian likelihoods, 'lr_cov' is also used for those

  • For "lbfgs", this is divided by the norm of the gradient in the first iteration

• lr_coef: numeric (default = 0.1, only relevant for "gradient_descent", "fisher_scoring", and "newton"). Learning rate for fixed effect regression coefficients if gradient descent is used

• use_nesterov_acc: boolean (default = TRUE, only relevant for "gradient_descent"). If TRUE Nesterov acceleration is used. This is used only for gradient descent

• acc_rate_coef: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for regression coefficients (if there are any) for Nesterov acceleration

• acc_rate_cov: numeric (default = 0.5, only relevant for "gradient_descent"). Acceleration rate for covariance parameters for Nesterov acceleration

• momentum_offset: integer (Default = 2, only relevant for "gradient_descent"). Number of iterations for which no momentum is applied in the beginning.

A GPModel

A string specifying the type of Vecchia approximation used for making predictions. Default value if vecchia_pred_type = NULL: "order_obs_first_cond_obs_only". Available options:

• "order_obs_first_cond_obs_only": Vecchia approximation for the observable process and observed training data is ordered first and the neighbors are only observed training data points

• "order_obs_first_cond_all": Vecchia approximation for the observable process and observed training data is ordered first and the neighbors are selected among all points (training + prediction)

• "latent_order_obs_first_cond_obs_only": Vecchia approximation for the latent process and observed data is ordered first and neighbors are only observed points

• "latent_order_obs_first_cond_all": Vecchia approximation for the latent process and observed data is ordered first and neighbors are selected among all points

• "order_pred_first": Vecchia approximation for the observable process and prediction data is ordered first for making predictions. This option is only available for Gaussian likelihoods

an integer specifying the number of neighbors for the Vecchia approximation for making predictions. Default value if NULL: num_neighbors_pred = 2 * num_neighbors

a numeric specifying the tolerance level for L2 norm of residuals for checking convergence in conjugate gradient algorithms when being used for prediction Default value if NULL: 1e-3

an integer specifying the number of samples when simulation is used for calculating predictive variances Internal default values if NULL:

• 500 for grouped random effects

• 1000 for gp_approx = "vecchia" and gp_approx = "full_scale_tapering"

• 100 for gp_approx = "full_scale_vecchia"

an integer specifying the rank of the matrix for approximating predictive covariances obtained using the Lanczos algorithm Default value if NULL: 1000

A vector or matrix with elements being group levels for which predictions are made (if there are grouped random effects in the GPModel)

A vector or matrix with covariate data for grouped random coefficients (if there are some in the GPModel)

A matrix with prediction coordinates (=features) for Gaussian process (if there is a GP in the GPModel)

A vector or matrix with covariate data for Gaussian process random coefficients (if there are some in the GPModel)

A vector with elements indicating the realizations of random effects / Gaussian processes for which predictions are made (set to NULL if you have not specified this when creating the GPModel)

A matrix with prediction covariate data for the fixed effects linear regression term (if there is one in the GPModel)

A GPModel

A string specifying the type of Vecchia approximation used for making predictions. Default value if vecchia_pred_type = NULL: "order_obs_first_cond_obs_only". Available options:

• "order_obs_first_cond_obs_only": Vecchia approximation for the observable process and observed training data is ordered first and the neighbors are only observed training data points

• "order_obs_first_cond_all": Vecchia approximation for the observable process and observed training data is ordered first and the neighbors are selected among all points (training + prediction)

• "latent_order_obs_first_cond_obs_only": Vecchia approximation for the latent process and observed data is ordered first and neighbors are only observed points

• "latent_order_obs_first_cond_all": Vecchia approximation for the latent process and observed data is ordered first and neighbors are selected among all points

• "order_pred_first": Vecchia approximation for the observable process and prediction data is ordered first for making predictions. This option is only available for Gaussian likelihoods

an integer specifying the number of neighbors for the Vecchia approximation for making predictions. Default value if NULL: num_neighbors_pred = 2 * num_neighbors

a numeric specifying the tolerance level for L2 norm of residuals for checking convergence in conjugate gradient algorithms when being used for prediction Default value if NULL: 1e-3

an integer specifying the number of samples when simulation is used for calculating predictive variances Internal default values if NULL:

• 500 for grouped random effects

• 1000 for gp_approx = "vecchia" and gp_approx = "full_scale_tapering"

• 100 for gp_approx = "full_scale_vecchia"

an integer specifying the rank of the matrix for approximating predictive covariances obtained using the Lanczos algorithm Default value if NULL: 1000

A vector or matrix with elements being group levels for which predictions are made (if there are grouped random effects in the GPModel)

A vector or matrix with covariate data for grouped random coefficients (if there are some in the GPModel)

A matrix with prediction coordinates (=features) for Gaussian process (if there is a GP in the GPModel)

A vector or matrix with covariate data for Gaussian process random coefficients (if there are some in the GPModel)

A vector with elements indicating the realizations of random effects / Gaussian processes for which predictions are made (set to NULL if you have not specified this when creating the GPModel)

A matrix with prediction covariate data for the fixed effects linear regression term (if there is one in the GPModel)

Object of class gpb.Dataset

other parameters

the name of the field to get

the specific field of information to set

Object of class gpb.Dataset

other parameters (currently not used)

an integer vector of indices of rows needed

a GPModel

(not used, ignore this, simply here that there is no CRAN warning)