| Type: | Package |
| Title: | Power Analysis for AB Testing |
| Version: | 0.1.0 |
| Maintainer: | William Cha <william.minseuk.cha@gmail.com> |
| Description: | Power analysis for AB testing. The calculations are based on the Welch's unequal variances t-test, which is generally preferred over the Student's t-test when sample sizes and variances of the two groups are unequal, which is frequently the case in AB testing. In such situations, the Student's t-test will give biased results due to using the pooled standard deviation, unlike the Welch's t-test. |
| License: | GPL (≥ 3) |
| Encoding: | UTF-8 |
| LazyData: | true |
| Imports: | stats |
| URL: | http://github.com/williamcha/pwrAB |
| BugReports: | http://github.com/williamcha/pwrAB/issues |
| Depends: | R (≥ 3.3.1) |
| RoxygenNote: | 6.0.1 |
| Suggests: | testthat |
| NeedsCompilation: | no |
| Packaged: | 2017-06-06 06:57:46 UTC; william |
| Author: | William Cha [aut, cre] |
| Repository: | CRAN |
| Date/Publication: | 2017-06-06 10:19:03 UTC |
Two-Sample t-Test Power Analysis
Description
AB_t2n performs the power analysis for AB testing. It uses the Welch's t-test,
which allows for the standard deviation to vary across groups.
Usage
AB_t2n(N = NULL, percent_B = NULL, mean_diff = NULL, sd_A, sd_B,
sig_level = NULL, power = NULL, alternative = c("two_sided", "less",
"greater"), max_sample = 1e+07)
Arguments
N |
Total number of observations (sum of observations for groups A and B) |
percent_B |
Percentage of total observations allocated to group B (between 0 and 1 - e.g. input .5 for 50%) |
mean_diff |
Difference in means of the two groups, with mean_B - mean_A |
sd_A |
Standard deviation of group A |
sd_B |
Standard deviation of group B |
sig_level |
Significance level (Type I error probability) |
power |
Power of test (1 minus Type II error probability) |
alternative |
Character string specifying the alternative hypothesis, must be one of "two_sided" (default), "greater" or "less" |
max_sample |
Maximum sample size that is searched for |
Details
Exactly one of the parameters 'N', 'percent_B', 'mean_diff', 'sig_level', and 'power' must be passed as NULL, and the omitted parameter is determined from the others. sd_A and sd_B must be specified. When 'percent_B' is the parameter omitted, two solutions may exist, in which case the smaller value will be returned
Value
Object of class "power.htest", a list of the arguments (including the computed one).
Examples
# Search for power given other parameters
AB_t2n(N = 3000, percent_B = .3, mean_diff = .15, sd_A = 1,
sd_B = 2, sig_level = .05, alternative = 'two_sided')
# Search for sample size required to satisfy other parameters
AB_t2n(percent_B = .3, mean_diff = .15, sd_A = 1,
sd_B = 2, sig_level = .05, power = .8, alternative = 'two_sided')
Two-Sample t-Test Power Analysis for Proportions
Description
AB_t2n_prop performs the power analysis for AB testing, and when
dependent variables are proportions (between 0 and 1). It uses the Welch's t-test,
which allows for the standard deviation to vary across groups.
Usage
AB_t2n_prop(prop_A = NULL, prop_B = NULL, N = NULL, percent_B = NULL,
sig_level = NULL, power = NULL, alternative = c("two_sided", "less",
"greater"), max_sample = 1e+07)
Arguments
prop_A |
Proportion of successes in group A (between 0 and 1) |
prop_B |
Proportion of successes in group B (between 0 and 1) |
N |
Total number of observations (sum of observations for groups A and B) |
percent_B |
Percentage of total observations allocated to group B (between 0 and 1 - e.g. input .5 for 50%) |
sig_level |
Significance level (Type I error probability) |
power |
Power of test (1 minus Type II error probability) |
alternative |
Character string specifying the alternative hypothesis, must be one of "two_sided" (default), "greater" or "less" |
max_sample |
Maximum sample size that is searched for |
Details
Exactly one of the parameters 'prop_A', 'prop_B', 'N', 'percent_B', 'sig_level', and 'power' must be passed as NULL, and the omitted parameter is determined from the others. The standard deviations for each group are calculated using the formula sqrt(prop * (1 - prop)). When 'percent_B' is the parameter omitted, two solutions may exist, in which case the smaller value will be returned. For two_sided tests, when 'prop_A' or 'prop_B' is omitted, two solutions may exist, in which case both will be reported
Value
Object of class "power.htest", a list of the arguments (including the computed one).
Examples
# Search for power given other parameters
AB_t2n_prop(prop_A = .2, prop_B = .25,
N = 3000, percent_B = .3,
sig_level = .05, alternative = 'two_sided')
# Search for proportion in group B required to satisfy other parameters
AB_t2n_prop(prop_A = .2, N = 3000, percent_B = .3,
power = .8, sig_level = .05,
alternative = 'two_sided')