Type: Package
Title: Tidy Multiverse Analysis Made Simple
Version: 0.2.2
Maintainer: Michael Jongho Moon <michael.moon@utoronto.ca>
Description: Extends 'multiverse' package (Sarma A., Kale A., Moon M., Taback N., Chevalier F., Hullman J., Kay M., 2021) <doi:10.31219/osf.io/yfbwm>, which allows users perform to create explorable multiverse analysis in R. This extension provides an additional level of abstraction to the 'multiverse' package with the aim of creating user friendly syntax to researchers, educators, and students in statistics. The 'mverse' syntax is designed to allow piping and takes hints from the 'tidyverse' grammar. The package allows users to define and inspect multiverse analysis using familiar syntax in R.
License: GPL (≥ 3)
Encoding: UTF-8
LazyData: true
Depends: R (≥ 3.6), multiverse (≥ 0.6.2)
Imports: Rdpack, magrittr (≥ 1.5), rlang, dplyr (≥ 1.1), tidyr, tidyselect, stringr, stats, utils, broom, igraph, ggraph, ggplot2 (≥ 3.5.2), ggupset (≥ 0.4.1)
Suggests: methods, tibble, purrr, scales, MASS, testthat (≥ 3.0.0), pkgdown (≥ 1.5.1), covr, knitr, rmarkdown, kableExtra, spelling
RoxygenNote: 7.3.2
RdMacros: Rdpack
VignetteBuilder: knitr
URL: https://github.com/mverseanalysis/mverse/, https://mverseanalysis.github.io/mverse/
Config/testthat/edition: 3
Language: en-US
NeedsCompilation: no
Packaged: 2025-06-20 16:29:40 UTC; moon
Author: Michael Jongho Moon [aut, cre], Haoda Li [aut], Mingwei Xu [aut], Nathan Taback [aut], Fanny Chevalier [aut], Alison Gibbs [ctb]
Repository: CRAN
Date/Publication: 2025-06-21 10:40:06 UTC

Display the AIC and BIC score of the fitted models across the multiverse

Description

Display the AIC and BIC score of glm regression results across the multiverse.

Usage

## S3 method for class 'mverse'
AIC(object, ..., k = 2)

## S3 method for class 'mverse'
BIC(object, ...)

AIC(object, ..., k = 2)

BIC(object, ...)

Arguments

object

a glm_mverse object.

...

ignored. for compatibility only.

k

ignored. for compatibility only.

Value

a multiverse table as a tibble


Add branch conditions to a mverse object.

Description

This method adds one or more branch conditions to an existing mverse object. Branch conditions are used to specify an option in one branch dependent on an option in another branch.

Usage

add_branch_condition(.mverse, ...)

## S3 method for class 'mverse'
add_branch_condition(.mverse, ...)

Arguments

.mverse

a mverse object.

...

branch conditions.

Value

a mverse object.

See Also

Other branch condition functions: branch_condition()

Examples

# Define branches and add them to an \code{mverse} object.
y <- mutate_branch(alldeaths, log(alldeaths + 1))
distribution <- family_branch(poisson, gaussian)
# You can match branching options by providing the options
# the way provide them when defining branches.
match_poisson <- branch_condition(alldeaths, poisson)
mv <- mverse(hurricane) %>%
  add_mutate_branch(y) %>%
  add_family_branch(distribution) %>%
  add_branch_condition(match_poisson)
summary(mv)
# You can also condition to reject a pair of options by
# setting reject = TRUE.
match_log_lin <- branch_condition(log(alldeaths + 1), poisson, reject = TRUE)
mv <- add_branch_condition(mv, match_log_lin)
summary(mv)

Add family branches to a mverse object.

Description

This method adds a family branch to an existing mverse object. A family branch is used to define options for the analysis distributions when using glm_mverse().

Usage

add_family_branch(.mverse, br)

Arguments

.mverse

a mverse object.

br

a family_branch object.

Value

The resulting mverse object.

See Also

Other family branch functions: family_branch

Examples

# Define a family branch.
model_distributions <- family_branch(
  gaussian, poisson(link = "log")
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
  add_family_branch(model_distributions)

Add filter branches to a mverse object.

Description

This method adds one or more filter branches to an existing mverse object. Filter branches are used to define options for conditions for selecting subsets of data rows.

Usage

add_filter_branch(.mverse, ...)

Arguments

.mverse

a mverse object.

...

filter_branch objects.

Value

The resulting mverse object.

See Also

Other filter branch functions: filter_branch

Examples

# Define a filter branch.
hurricane_outliers <- filter_branch(
  !Name %in% c("Katrina", "Audrey", "Andrew"),
  !Name %in% c("Katrina"),
  !Name %in% c("Katrina"),
  TRUE # include all
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
  add_filter_branch(hurricane_outliers)

Add formula branches to a mverse object.

Description

This method adds a formula branch to an existing mverse object. A formula branch is used to specify model structure options for the analysis.

Usage

add_formula_branch(.mverse, br)

Arguments

.mverse

a mverse object.

br

a formula_branch object.

Value

The resulting mverse object.

See Also

Other formula branch functions: formula_branch

Examples

# Define a formula branch.
model_specifications <- formula_branch(
  y ~ MasFem,
  y ~ MasFem + hurricane_strength,
  y ~ MasFem * hurricane_strength
)
# Create a mverse, add the branch.
mv <- create_multiverse(hurricane) %>%
  add_formula_branch(model_specifications)

Add mutate branches to a mverse object.

Description

This method adds one or more mutate branches to an existing mverse object. Mutate branches are used to define options for adding a new column to the analysis dataset.

Usage

add_mutate_branch(.mverse, ...)

Arguments

.mverse

a mverse object.

...

mutate_branch objects.

Value

The resulting mverse object.

See Also

Other mutate branch functions: mutate_branch

Examples

# Define mutate branches.
hurricane_strength <- mutate_branch(
  # damage vs. wind speed vs.pressure
  NDAM,
  HighestWindSpeed,
  Minpressure_Updated_2014,
  # Standardized versions
  scale(NDAM),
  scale(HighestWindSpeed),
  -scale(Minpressure_Updated_2014),
)
y <- mutate_branch(
  alldeaths, log(alldeaths + 1)
)
# Create a mverse and add the branches.
mv <- create_multiverse(hurricane) %>%
  add_mutate_branch(hurricane_strength) %>%
  add_mutate_branch(y)
# You can also add multiple branches with a single call.
mv <- create_multiverse(hurricane) %>%
  add_mutate_branch(hurricane_strength, y)

Create a new branch condition.

Description

A branch condition conditions option x to depend on option y. When the branch condition is added to a mverse object, option x is executed only when y is. Use reject = TRUE, to negate the condition.

Usage

branch_condition(x, y, reject = FALSE)

Arguments

x

option 1

y

option 2

reject

if TRUE, the condition rejects universes with option 1 and option 2

Value

A branch_condition object.

See Also

Other branch condition functions: add_branch_condition()

Examples

# Example branches.
y <- mutate_branch(alldeaths, log(alldeaths + 1))
model <- formula_branch(y ~ MasFem * strength, y ~ MasFem + strength)
# Define a new branch condition.
match_poisson <- branch_condition(alldeaths, poisson)
# Define a branch condition that reject an option dependent on another.
match_log_lin <- branch_condition(log(alldeaths + 1), poisson, reject = TRUE)


Execute the entire multiverse.

Description

This method executes the analysis steps defined in the mverse objected across the entire multiverse.

Usage

execute_multiverse(.mverse, parallel = FALSE, progress = FALSE)

## S3 method for class 'mverse'
execute_multiverse(.mverse, parallel = FALSE, progress = FALSE)

Arguments

.mverse

a mverse object.

parallel

passed to multiverse::execute_multiverse() to indicate whether to execute the multiverse analysis in parallel. Defaults to FALSE.

progress

passed to multiverse::execute_multiverse() to indicate whether to include a progress bar for each step of the execution. Defaults to FALSE.

Value

The resulting mverse object.

Examples

# Define a mutate branch.
hurricane_strength <- mutate_branch(
  # damage vs. wind speed vs.pressure
  NDAM,
  HighestWindSpeed,
  Minpressure_Updated_2014,
  # Standardized versions
  scale(NDAM),
  scale(HighestWindSpeed),
  -scale(Minpressure_Updated_2014),
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
  add_mutate_branch(hurricane_strength)
# The branched variables are not populated across the multiverse yet.
# Execute the multiverse; the variables are populated after the execution.
execute_multiverse(mv)

Extract branched values.

Description

extract returns a tibble of selected values across the multiverse in a long format.

Usage

extract(...)

## S3 method for class 'mverse'
extract(
  .mverse,
  columns = NULL,
  nuni = NULL,
  frow = NULL,
  include_branch_options = TRUE,
  ...
)

Arguments

...

Ignored.

.mverse

a mverse object.

columns

a character vector of column names to extract.

nuni

a positive integer for the number of universes to extract.

frow

proportion of rows to extract from each universe.

include_branch_options

when TRUE (default), include the mutate statements used to specified the options for each branched columns

Details

This method extracts data values across the multiverse. You can specify a subset of data to extract using columns, universe, nuni, and frow.

You can specify the columns to extract from each universe by passing the column names as a character vector to columns. The default values is NULL extracting all columns with branches.

Use universe to specify a set of universes by their integer ids. Use nuni to specify the number of universes to extract data from. The method then selects the subset randomly. Specifying universe manually will override nuni value. By default, they are both set to NULL and the method returns data from all universes.

Use frow to randomly extract a fraction of data from each universe. The default value is NULL and all rows are returned as they are. Note if you select 1 the method will return shuffle rows in each universe before returning them. If frow is greater than 1, the method randomly samples rows with replacement.

Value

a tibble containing the selected columns across the multiverse.

Examples

# Define mutate branches.
hurricane_strength <- mutate_branch(
  # damage vs. wind speed vs.pressure
  NDAM,
  HighestWindSpeed,
  Minpressure_Updated_2014,
  # Standardized versions
  scale(NDAM),
  scale(HighestWindSpeed),
  -scale(Minpressure_Updated_2014),
)
y <- mutate_branch(
  alldeaths, log(alldeaths + 1)
)
# Create a mverse and add the branches.
mv <- create_multiverse(hurricane) %>%
  add_mutate_branch(hurricane_strength, y)
execute_multiverse(mv)
# Extract all branched columns from all universes
extract(mv)
# Specify the columns to extract from each universe using columns
# You can select both branched and non-branched columns
extract(mv, columns = c("hurricane_strength", "NDAM"))
# Specify the universe to extract from using universe
extract(mv, universe = 1)
# Specify the number of universes to extract from using nuni
# The universes are randomly selected
extract(mv, nuni = 3)
# Specify the proportion of data to extract from each universe using
# frow. The rows are randomly selected
extract(mv, frow = 0.7)

Create a new family branch.

Description

Create a new family branch.

Usage

family_branch(..., name = NULL)

Arguments

...

branch definition expressions.

name

(optional) Name for the new family.

Value

a family_branch object.

See Also

Other family branch functions: add_family_branch()

Examples

# Define a family branch.
model_distributions <- family_branch(
  gaussian, poisson(link = "log")
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
  add_family_branch(model_distributions)

Create a new filter branch.

Description

Create a new filter branch.

Usage

filter_branch(..., name = NULL)

Arguments

...

branch definition expressions.

name

(optional) Name for the new filter.

Value

a filter_branch object.

See Also

Other filter branch functions: add_filter_branch()

Examples

# Define a filter branch.
hurricane_outliers <- filter_branch(
  !Name %in% c("Katrina", "Audrey", "Andrew"),
  !Name %in% c("Katrina"),
  !Name %in% c("Katrina"),
  TRUE # include all
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
  add_filter_branch(hurricane_outliers)

Create a new formula branch.

Description

The function specifies the model formula for fitting 'lm_mverse()' and 'glm_mverse()'. You can list the model specification formulae individually or use covariates option paired with one or more formulae.

Usage

formula_branch(..., covariates = NULL, name = NULL)

Arguments

...

branch definition expressions.

covariates

(optional) A character vector of optional covariates. Each unique combination of the supplied covariates is translated into a unique branch option. See Details.

name

(optional) Name for the new formula.

Details

The optional argument covariates is allows you to specify a set of optional covariates in addition to other independent variable such as treatment variables and blocking variables which are specified using formula. For each covariate provided, a branch is added to the multiverse with the option to include or exclude the covariate in the model.

For example, formula_branch(y ~ x, covariates = c("c1", "c2")) creates the following 4 model specifications:

y ~ x

y ~ x + c1

y ~ x + c2

y ~ x + c1 + c2

Here, y is the outcome variable and x may be a treatment variable in an experiment setting. c1 and c2 may be additional covariates about the experiment units that may or may not be relevant.

Value

a formula_branch object.

See Also

Other formula branch functions: add_formula_branch()

Examples

# Define a formula branch.
model_specifications <- formula_branch(
  y ~ MasFem,
  y ~ MasFem + hurricane_strength,
  y ~ MasFem * hurricane_strength
)
# Create a mverse, add the branch.
mv <- create_multiverse(hurricane) %>%
  add_formula_branch(model_specifications)
# Specify the covariates separately.
model_specifications <- formula_branch(
  y ~ MasFem,
  covariates = c("hurricane_strength", "Year", "Category", "NDAM")
)
model_specifications

Fit negative binomial regression models across the multiverse

Description

glm.nb_mverse fits MASS::glm.nb across the multiverse according to model specifications provided by formula_branch. At least one formula_branch must have been added.

Usage

glm.nb_mverse(.mverse, parallel = FALSE, progress = FALSE)

Arguments

.mverse

a mverse object.

parallel

passed to multiverse::execute_multiverse() to indicate whether to execute the multiverse analysis in parallel. Defaults to FALSE.

progress

passed to multiverse::execute_multiverse() to indicate whether to include a progress bar for each step of the execution. Defaults to FALSE.

Value

A mverse object with glm.nb fitted.

See Also

Other model fitting functions: glm_mverse(), lm_mverse()

Examples


# Displaying the multiverse table with \code{glm.nb} models fitted.
hurricane_outliers <- filter_branch(
  !Name %in% c("Katrina", "Audrey", "Andrew"),
  TRUE # include all
)
model_specifications <- formula_branch(alldeaths ~ MasFem)
mv <- create_multiverse(hurricane) %>%
  add_filter_branch(hurricane_outliers) %>%
  add_formula_branch(model_specifications) %>%
  glm.nb_mverse()
summary(mv)


Fit generalized linear regression models across the multiverse.

Description

glm_mverse fits glm across the multiverse according to model specifications provided by formula_branch. At least one formula_branch must have been added. You can also specify the underlying error distribution and the link function by adding a family_branch. If no family_branch has been provided, it follows the default behaviour of glm using the Gaussian distribution with an identity link.

Usage

glm_mverse(.mverse, parallel = FALSE, progress = FALSE)

Arguments

.mverse

a mverse object.

parallel

passed to multiverse::execute_multiverse() to indicate whether to execute the multiverse analysis in parallel. Defaults to FALSE.

progress

passed to multiverse::execute_multiverse() to indicate whether to include a progress bar for each step of the execution. Defaults to FALSE.

Value

A mverse object with glm fitted.

See Also

Other model fitting functions: glm.nb_mverse(), lm_mverse()

Examples



# Fitting \code{glm} models across a multiverse.
hurricane_strength <- mutate_branch(
  NDAM,
  HighestWindSpeed,
  Minpressure_Updated_2014
)
hurricane_outliers <- filter_branch(
  !Name %in% c("Katrina", "Audrey", "Andrew"),
  TRUE # include all
)
model_specifications <- formula_branch(
  alldeaths ~ MasFem,
  alldeaths ~ MasFem + hurricane_strength
)
model_distributions <- family_branch(poisson)
mv <- create_multiverse(hurricane) %>%
  add_filter_branch(hurricane_outliers) %>%
  add_mutate_branch(hurricane_strength) %>%
  add_formula_branch(model_specifications) %>%
  add_family_branch(model_distributions) %>%
  glm_mverse()


Data on Atlantic hurricanes in the U.S. between 1950 and 2012.

Description

A dataset for the study conducted by Jung et al. (2014) in Female hurricanes are deadlier than male hurricanes.

Usage

hurricane

Format

A data frame with 94 rows and 12 variables:

Year

Year in which the hurricane landed on U.S.

Name

Name of the hurricane.

MasFem

Femininity index of the hurricane name collected by Jung et al. (1 - very masculine; 11 - very feminine).

MinPressure_before

Minimum pressure of the hurricane at the time of landfall in the U.S. (original).

Minpressure_Updated_2014

Minimum pressure of the hurricane at the time of landfall in the U.S. (updated).

Gender_MF

Gender indicator for the hurricane name based on MasFem index (1 - MasFem > 6; 0 otherwise).

Category

Hurricane category on a scale of 1 to 5, with 5 being the most severe.

alldeaths

Number of fatalities.

NDAM

Normalized damage in 2013 U.S. million dollars.

Elapsed.Yrs

Time since hurricane.

Source

Source from where the data was gathered.

HighestWindSpeed

Maximum wind speed.

MasFem_MTUrk

Femininity index of the hurricane name collected by Simonsohn et al.

NDAM15

Normalized damage in 2015 U.S. million dollars.

Details

The dataset was collected by Jung et al. in their study Female hurricanes are deadlier than male hurricanes. Their study didn't include hurricanes Katrina and Audrey which were deemed as outliers. Simonsohn et al. collected the extra data for Specification curve analysis including an additional femininity index based on an MTUrk survey and updated normalized damage amount in 2015 U.S. dollars.

This dataset includes data prepared by Jung et al. (2014) as well as those prepared by Simonsohn et al. (2020). Specifically, all data on Katrina and Audrey are from Simonsohn et al. (2020) except minimum pressure updated in 2014. They were retrieved from Continental United States Hurricane Impacts/Landfalls 1851-2021 table maintained by U.S. National Oceanic and Atmospheric Administration. Maximum wind speed, femininity index from MTUrk survey, and 2015 damage amounts are also from Simonsohn et al. (2020).

Source

Kiju Jung, Sharon Shavitt, Madhu Viswanathan, and Joseph M. Hilbe. (2014). "Female hurricanes are deadlier than male hurricanes." Proceedings of the National Academy of Sciences, 111(24), 8782-8787. doi:10.1073/pnas.1402786111

Uri Simonsohn, Joseph P. Simmons, and Leif D. Nelson. (2020). “Specification curve analysis” Nature Human Behaviour, 4, 1208–14. doi:10.1038/s41562-020-0912-z


Fit linear regression models across the multiverse.

Description

lm_mverse fits lm across the multiverse according to model specifications provided by formula_branch. At least one formula_branch must have been added.

Usage

lm_mverse(.mverse, parallel = FALSE, progress = FALSE)

Arguments

.mverse

a mverse object.

parallel

passed to multiverse::execute_multiverse() to indicate whether to execute the multiverse analysis in parallel. Defaults to FALSE.

progress

passed to multiverse::execute_multiverse() to indicate whether to include a progress bar for each step of the execution. Defaults to FALSE.

Value

A mverse object with lm fitted.

See Also

Other model fitting functions: glm.nb_mverse(), glm_mverse()

Examples



# Fitting \code{lm} models fitted across a multiverse.
hurricane_strength <- mutate_branch(
  NDAM,
  HighestWindSpeed,
  Minpressure_Updated_2014
)
y <- mutate_branch(
  alldeaths, log(alldeaths + 1)
)
hurricane_outliers <- filter_branch(
  !Name %in% c("Katrina", "Audrey", "Andrew"),
  TRUE # include all
)
model_specifications <- formula_branch(
  y ~ MasFem,
  y ~ MasFem + hurricane_strength
)
mv <- create_multiverse(hurricane) %>%
  add_filter_branch(hurricane_outliers) %>%
  add_mutate_branch(hurricane_strength, y) %>%
  add_formula_branch(model_specifications) %>%
  lm_mverse()


Plot a multiverse tree diagram.

Description

A multiverse tree diagram displays the branching combination of all the branches added to the given mverse object taking any branch conditions defined. The method also allows zooming into a subset of branches using branches parameter.

Usage

multiverse_tree(
  .mverse,
  label = "none",
  branches = NULL,
  label_size = NULL,
  label_angle = 0,
  label_hjust = 0,
  label_vjust = 0
)

Arguments

.mverse

A mverse object.

label

Display the branch option name when "name" or the definition when "code". No label is displayed when "none" (default).

branches

A character vector. Display a subset of branches when specified. Display all when NULL.

label_size

A numeric. Set size of option labels.

label_angle

A numeric. Rotate option labels.

label_hjust

A numeric. Set the horizontal justification of the node labels.

label_vjust

A numeric. Set the vertical justification of the node labels.

Value

A ggplot object displaying the multiverse tree.

Examples

{
# Display a multiverse tree with multiple branches.
outliers <- filter_branch(!Name %in% c("Katrina", "Audrey"), TRUE)
femininity <- mutate_branch(MasFem, Gender_MF)
strength <- mutate_branch(
  NDAM, HighestWindSpeed, Minpressure_Updated_2014, log(NDAM)
)
y <- mutate_branch(alldeaths, log(alldeaths + 1))
model <- formula_branch(y ~ femininity * strength, y ~ femininity + strength)
distribution <- family_branch(poisson, gaussian)
mv <- mverse(hurricane) %>%
  add_filter_branch(outliers) %>%
  add_mutate_branch(femininity, strength, y) %>%
  add_formula_branch(model) %>%
  add_family_branch(distribution)
multiverse_tree(mv)
# Display a multiverse tree with branch conditions.
match_poisson <- branch_condition(alldeaths, poisson)
match_log_lin <- branch_condition(log(alldeaths + 1), gaussian)
add_branch_condition(mv, match_poisson)
add_branch_condition(mv, match_log_lin)
multiverse_tree(mv)
# You can adjust colour scale of the edges
# using a ggraph::scale_edge_colour*() function.
multiverse_tree(mv) + ggraph::scale_edge_colour_viridis(
  discrete = TRUE,
  labels = c("Distribution", "Model", "Strength",
             "Femininity", "Outliers", "y")
)
# Display a multiverse tree for a subset of branches
# with name label for each option.
multiverse_tree(mv, branches = c("y", "distribution"), label = "name")
# with code label for each option.
multiverse_tree(mv, branches = c("y", "distribution"), label = "code")
# adjusting size and orientation of the labels
multiverse_tree(mv, branches = c("y", "distribution"),
  label = "name", label_size = 4, label_angle = 45)
}

Create a new mutate branch.

Description

Create a new mutate branch.

Usage

mutate_branch(..., name = NULL)

Arguments

...

branch definition expressions.

name

(optional) Name for the new variable.

Value

a mutate_branch object.

See Also

Other mutate branch functions: add_mutate_branch()

Examples

# Define mutate branches.
hurricane_strength <- mutate_branch(
  # damage vs. wind speed vs.pressure
  NDAM,
  HighestWindSpeed,
  Minpressure_Updated_2014,
  # Standardized versions
  scale(NDAM),
  scale(HighestWindSpeed),
  -scale(Minpressure_Updated_2014),
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
  add_mutate_branch(hurricane_strength)

Create a new mverse object

Description

Constructs a new mverse object which extends multiverse::multiverse object.

Usage

mverse(data)

create_multiverse(data)

Arguments

data

source dataframe.

Value

A mverse object with the source dataframe attached.

Examples

# Create a mverse object.
mv <- mverse(hurricane)
# create_multiverse() is an alias of mverse().
mv <- create_multiverse(hurricane)

Print method for *_branch objects.

Description

Print method for *_branch objects.

Usage

## S3 method for class 'branch'
print(x, ...)

Arguments

x

a branch object.

...

ignored. for compatibility only.

Value

No return value, called for printing only.


Objects exported from other packages

Description

These objects are imported from other packages. Follow the links below to see their documentation.

magrittr

%>%


Number of cards given for each referee-player pair in soccer.

Description

A dataset containing card counts between 2,053 soccer players playing in the first male divisions of England, Germany, France, and Spain in the 2012-2013 season and 3,147 referees that these players played under in professional matches. The dataset contains other covariates including 2 independent skin tone ratings per player. Each line represents a player-referee pair.

Usage

soccer

Format

A data frame with 146,028 rows and 26 variables:

playerShort

short player ID

player

player name

club

player club

leagueCountry

country of player club ( England, Germany, France, and Spain)

birthday

player birthday

height

player height (in cm)

weight

player weight (in kg)

position

detailed player position

games

number of games in the player-referee dyad

victories

victories in the player-referee dyad

ties

ties in the player-referee dyad

defeats

losses in the player-referee dyad

goals

goals scored by a player in the player-referee dyad

yellowCards

number of yellow cards player received from referee

yellowReds

number of yellow-red cards player received from referee

redCards

number of red cards player received from referee

rater1

skin rating of photo by rater 1 (5-point scale ranging from “very light skin” to “very dark skin”)

rater2

skin rating of photo by rater 2 (5-point scale ranging from “very light skin” to “very dark skin”)

refNum

unique referee ID number (referee name removed for anonymizing purposes)

refCountry

unique referee country ID number (country name removed for anonymizing purposes)

meanIAT

mean implicit bias score (using the race IAT) for referee country, higher values correspond to faster white | good, black | bad associations

nIAT

sample size for race IAT in that particular country

seIAT

standard error for mean estimate of race IAT

meanExp

mean explicit bias score (using a racial thermometer task) for referee country, higher values correspond to greater feelings of warmth toward whites versus blacks

nExp

sample size for explicit bias in that particular country

seExp

standard error for mean estimate of explicit bias measure

Details

The skin colour of each player was rated by two independent raters, rater1 and rater2, and the 5-point scale values were scaled to 0 to 1 - i.e., 0, 0.25, 0.5, 0.75, 1.

Source

Silberzahn R, Uhlmann EL, Martin DP, et al. Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science. 2018;1(3):337-356. doi:10.1177/2515245917747646


Display a specification curve across the multiverse.

Description

Returns a ggplot object that displays the specification curve as proposed by (Simonsohn et al. 2020). Note that the order of universes may not correspond to the order in the summary table.

Usage

spec_curve(
  .spec_summary,
  label = "name",
  order_by = c("estimate", "is_significant"),
  colour_by = "is_significant",
  palette_common = NULL,
  pointsize = 2,
  linewidth = 0.5,
  spec_matrix_spacing = 10,
  theme_common = ggplot2::theme_minimal(),
  sep = "---"
)

Arguments

.spec_summary

A specification table created using spec_summary().

label

If "name", uses the branch option names. If "code", display the codes used to define the branch options.

order_by

A character vector by which the curve is sorted.

colour_by

The name of the variable to colour the curve.

palette_common

A character vector of colours to match the values of the variable colour_by in the specification curve and the specification matrix. The palette must contain more colours than the number of unique values of colour_by variable.

pointsize

Size of the points in the specification curve and the specification matrix.

linewidth

Width of confidence interval lines.

spec_matrix_spacing

A numeric for adjusting the specification matrix spacing passed to combmatrix.label.extra_spacing in ggupset::theme_combmatrix().

theme_common

A ggplot theme to be used for both the specification curve and the specification matrix.

sep

A string used internally to create the specification matrix. The string must be distinct from all branch names, option names, and option codes. Use a different value if any of them contains the default value.

Value

a ggplot object with the specification curve plot for the estimates passed in the spec_summary().

References

Simonsohn U, Simmons JP, Nelson LD (2020). “Specification curve analysis.” Nature Human Behaviour, 4(11), 1208–1214. doi:10.1038/s41562-020-0912-z.

See Also

Other specification curve analysis: spec_summary()

Examples

femininity <- mutate_branch(
  1 * (MasFem > 6), 1 * (MasFem > mean(MasFem))
)
y <- mutate_branch(log(alldeaths + 1), alldeaths)
intensity <- mutate_branch(
  Minpressure_Updated_2014,
  Category,
  NDAM,
  HighestWindSpeed
)
model <- formula_branch(
  y ~ femininity,
  y ~ femininity * intensity
)
family <- family_branch(
  gaussian, poisson
)
match_poisson <- branch_condition(alldeaths, poisson)
match_gaussian <- branch_condition(log(alldeaths + 1), gaussian)
stable <- mverse(hurricane) %>%
  add_mutate_branch(y, femininity, intensity) %>%
  add_formula_branch(model) %>%
  add_family_branch(family) %>%
  add_branch_condition(match_poisson, match_gaussian) %>%
  glm_mverse() %>%
  spec_summary("femininity")
# default behaviour
spec_curve(stable)
# coloring and sorting based on other variable
stable %>%
  dplyr::mutate(colour_by = y_branch) %>%
  spec_curve(order_by = c("estimate", "colour_by"), colour_by = "colour_by")
# Because the output is a \code{ggplot} object, you can
# further modify the asethetics of the specification curve
# using \code{ggplot2::theme()} and the specication matrix
# using \code{ggupset::theme_combmatrix()}
spec_curve(stable) +
  ggplot2::labs(y = "Estimates", colour = "Significant at 0.05 level",
                title = "Specification curve of femininity") +
  ggplot2::theme(legend.position = "bottom") +
  ggupset::theme_combmatrix(
    combmatrix.label.width = ggplot2::unit(c(25, 100, 0, 0), "pt")
  )

Create a specification table for a selected variable.

Description

Returns estimates for a selected variable across the multiverse along with the universe specification information in a table. The resulting table can be used for spe_curve().

Usage

spec_summary(.mverse, var, conf.int = TRUE, conf.level = 0.95)

Arguments

.mverse

A mverse object.

var

A character specifying the variable of interest.

conf.int

Whether the table should include confidence intervals.

conf.level

The confidence level for the confidence level and is_significant.

Value

A spec_summary object that includes estimates and specification across the multiverse for the selected term(s). A boolean column is_significant indicates whether p.value for the universe is less than the specified significance level (1 - conf.level).

See Also

Other specification curve analysis: spec_curve()

Examples

femininity <- mutate_branch(
  1 * (MasFem > 6), 1 * (MasFem > mean(MasFem))
)
intensity <- mutate_branch(
  Minpressure_Updated_2014,
  Category,
  NDAM,
  HighestWindSpeed
)
model <- formula_branch(
  log(alldeaths + 1) ~ femininity,
  log(alldeaths + 1) ~ femininity * intensity
)
mv <- mverse(hurricane) %>%
  add_mutate_branch(femininity) %>%
  add_mutate_branch(intensity) %>%
  add_formula_branch(model) %>%
  lm_mverse()
spec_summary(mv, "femininity")

Display the multiverse table with results.

Description

This method returns the multiverse table displaying all universes defined by the multiverse. Each row corresponds to a universe and the columns include universe number, branch option name, and branch option definition.

Usage

## S3 method for class 'mverse'
summary(object, ...)

## S3 method for class 'lm_mverse'
summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...)

## S3 method for class 'glm_mverse'
summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...)

## S3 method for class 'glm.nb_mverse'
summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...)

Arguments

object

a glm.nb_mverse object.

...

Ignored.

conf.int

When TRUE (default), the estimate output includes the confidence intervals.

conf.level

The confidence level of the confidence interval returned using conf.int = TRUE. Default value is 0.95.

output

The output of interest. The possible values are "estimates" ("e"), "df", "deviance" ("de"), and "aic" ("bic"). Alternatively, the first letters may be used. Default value is "estimates".

Details

When you pass a mverse objected fitted with model, the summary table includes results of the fitted models across the multiverse.

Value

a multiverse table as a tibble.

Examples



# Displaying the multiverse table without any fitted values.
hurricane_strength <- mutate_branch(
  NDAM,
  HighestWindSpeed,
  Minpressure_Updated_2014
)
mv <- create_multiverse(hurricane) %>%
  add_mutate_branch(hurricane_strength)
summary(mv)
## Displaying after adding a a filter branch.
hurricane_outliers <- filter_branch(
  !Name %in% c("Katrina", "Audrey", "Andrew"),
  !Name %in% c("Katrina"),
  TRUE # include all
)
mv <- add_filter_branch(mv, hurricane_outliers)
summary(mv)



# Displaying the multiverse table with \code{lm} models fitted.
hurricane_strength <- mutate_branch(
  NDAM,
  HighestWindSpeed,
  Minpressure_Updated_2014
)
y <- mutate_branch(
  alldeaths, log(alldeaths + 1)
)
hurricane_outliers <- filter_branch(
  !Name %in% c("Katrina", "Audrey", "Andrew"),
  TRUE # include all
)
model_specifications <- formula_branch(
  y ~ MasFem,
  y ~ MasFem + hurricane_strength
)
mv <- create_multiverse(hurricane) %>%
  add_filter_branch(hurricane_outliers) %>%
  add_mutate_branch(hurricane_strength, y) %>%
  add_formula_branch(model_specifications) %>%
  lm_mverse()
summary(mv)



# Displaying the multiverse table with \code{glm} models fitted.
hurricane_strength <- mutate_branch(
  NDAM,
  HighestWindSpeed,
  Minpressure_Updated_2014
)
hurricane_outliers <- filter_branch(
  !Name %in% c("Katrina", "Audrey", "Andrew"),
  TRUE # include all
)
model_specifications <- formula_branch(
  alldeaths ~ MasFem,
  alldeaths ~ MasFem + hurricane_strength
)
model_distributions <- family_branch(poisson)
mv <- create_multiverse(hurricane) %>%
  add_filter_branch(hurricane_outliers) %>%
  add_mutate_branch(hurricane_strength) %>%
  add_formula_branch(model_specifications) %>%
  add_family_branch(model_distributions) %>%
  glm_mverse()
summary(mv)


# Displaying the multiverse table with \code{glm.nb} models fitted.
hurricane_outliers <- filter_branch(
  !Name %in% c("Katrina", "Audrey", "Andrew"),
  TRUE # include all
)
model_specifications <- formula_branch(alldeaths ~ MasFem)
mv <- create_multiverse(hurricane) %>%
  add_filter_branch(hurricane_outliers) %>%
  add_formula_branch(model_specifications) %>%
  glm.nb_mverse()
summary(mv)


Performs one or two sample t-tests on data columns.

Description

t_test_mverse performs t-tests across the multiverse. If x or y is specified, then performs one and two sample t-tests on specified columns of the data. If both x and y are NULL, then performs t.test based on the formula branches.

Usage

t_test_mverse(
  .mverse,
  x = NULL,
  y = NULL,
  alternative = "two.sided",
  mu = 0,
  paired = FALSE,
  var.equal = FALSE,
  conf.level = 0.95,
  parallel = FALSE,
  progress = FALSE
)

Arguments

.mverse

a mverse object.

x

(optional) column name of data within mverse object

y

(optional) column name of data within mverse object

alternative

a character string specifying the alternative hypothesis, must be one of "two.sided" (default), "greater" or "less". You can specify just the initial letter.

mu

a number indicating the true value of the mean (or difference in means if you are performing a two sample test).

paired

a logical indicating whether you want a paired t-test.

var.equal

a logical variable indicating whether to treat the two variances as being equal.

conf.level

confidence level of the interval.

parallel

passed to multiverse::execute_multiverse() to indicate whether to execute the multiverse analysis in parallel. Defaults to FALSE.

progress

passed to multiverse::execute_multiverse() to indicate whether to include a progress bar for each step of the execution. Defaults to FALSE.

Value

a multiverse table displaying the t-test results as a tibble.

Examples

# Performing a unpaired two sample t-test.
library(dplyr)
mv <- soccer %>%
  filter(!is.na(rater1), !is.na(rater2)) %>%
  mverse()
x <- mutate_branch(
  ((rater1 + rater2) / 2) > mean((rater1 + rater2) / 2),
  ifelse(rater1 > rater2, rater1 > 0.5, rater2 > 0.5)
)
y <- mutate_branch(
  redCards, yellowCards, yellowReds
)
two_sample_form <- formula_branch(y ~ x)
mv <- mv %>%
  add_mutate_branch(x, y) %>%
  add_formula_branch(two_sample_form)
t_test_mverse(mv)