P-value using the Monte Carlo procedure (2024)

pvalue.random {SCRT}R Documentation

Description

The P-value corresponding to the observed value of the test statistic is obtained by locating this value in the randomization distribution generated by a random sample of all assignment possibilities (the nonexhaustive randomization distribution).

Usage

pvalue.random(design, statistic, save = "no", number, limit, data = read.table(file.choose(new = FALSE)), starts = file.choose(new = FALSE), assignments = file.choose(new = FALSE))

Arguments

design

Type of single-case design: "AB", "ABA", "ABAB", "CRD" (completely randomized design), "RBD" (randomized block design), "ATD" (alternating treatments design), "MBD" (multiple-baseline AB design) or "Custom" (user specified design).

statistic

Test statistic. For alternation designs, multiple-baseline designs and AB phase designs, there are 3 built-in possibilities: "A-B", "B-A", and "|A-B|", which stand for the (absolute value of the) difference between condition means. For phase designs with more than 2 phases, 3 more built-in options are available: "PA-PB", "PB-PA", and "|PA-PB|" refer to the (absolute value of the) difference between the means of phase means.Additionally, it is possible to specify a custom test statistic using the variable identifiers "A" and "B" (or in the case of phase deisgns with more than 2 phases, "A1", "B1", "A2", "B2", "A" and "B") and any of the basic R functions. For example, "abs(mean(A) - mean(B))" can be used as a test statistic and it will be the same as using "|A-B|".

save

Save the randomization distribution to a file (save="yes") or just see it as output in the R console (default: save="no").

number

Number of randomizations required. Please note that the observed test statistic is always included in the randomization distribution.

limit

For phase designs: minimum number of observations per phase. For alternating treatments designs: maximum number of consecutive administrations of the same condition.

data

File in which the data can be found. Default: a window pops up in which the file can be selected.

starts

Only for multiple baseline designs: location of the file where the possible start points can be found. Default: a window pops up in which the file can be selected.

assignments

Only for user specified designs: location of the file where all the possible assignments can be found. Default: a window pops up in which the file can be selected.

Details

When using the default data argument, a window will pop up to ask in what file the data can be found. This text file containing the data should consist of two columns for single-case phase and alternation designs: the first with the condition labels and the second with the obtained scores.For multiple-baseline designs it should consist of these two columns for EACH unit. This way, each row represents one measurement occasion. It is important not to label the rows or columns.

For multiple baseline designs, when using the default starts argument, second a window pops up in which is asked in what file the possible start points can be found. In this startpoint file, each row should contain all possibilities for one unit, separated by a tab. The rows and columns should not be labeled.

For user specified designs, when using the default assignments argument, second a window pops up in which is asked in what file all the possible assignments can be found. In this file, each row should contain the sequence of conditions in one possible assignment, separated by a tab. There should be one row for every possible assignment. The rows and columns should not be labeled.

Missing data should be indicated as NA. When there is missing data, randomization distribution is generated as usual, but instead of randomly reshuffling numerical scores only, the missing data markers (NA) are also included in the reshuffling. For test statistic calculations, missing data are omitted. If test statistic cannot be calculated for a particular randomization due to insufficient data for a treatment condition, the test statistic from this randomization is conservatively considered more extreme than the observed test statistic.

When choosing to save the randomization distribution to a file, next a window will pop up (for multiple baseline designs or user specified designs this is the third pop-up window, for all other designs it is the second window) to ask where to save it. This location can be an existing file, as well as a new file that can be created by giving a file name and the extension .txt. In this latter case a confirmation is required ("The file does not exist yet. Create the file?").

References

Bulte, I., & Onghena, P. (2008). An R package for single-case randomization tests. Behavior Research Methods, 40, 467-478.

Bulte, I., & Onghena, P. (2009). Randomization tests for multiple baseline designs: An extension of the SCRT-R package. Behavior Research Methods, 41, 477-485.

Edgington, E.S., & Onghena, P. (2007). Randomization Tests (4th ed.). Boca Raton, FL: Chapman & Hall/CRC.

Hope, A.C.A. (1968). A simplified Monte Carlo significance test procedure. Journal of the Royal Statistical Society, Series B 30, 582-598.

Onghena, P. & May, R.B. (1995). Pitfalls in computing and interpreting randomization test p values: A commentary on Chen and Dunlap. Behavior Research Methods, Instruments, & Computers, 27, 408-411.

http://ppw.kuleuven.be/home/english/research/mesrg

See Also

distribution.random to generate the corresponding nonexhaustive randomization distribution.

observed to calculate the observed test statistic.

distribution.systematic to generate the exhaustive randomization distribution and pvalue.systematic to obtain the corresponding p-value.

Examples

data(ABAB)pvalue.random(design = "ABAB", statistic = "PA-PB", save = "no", number = 100, limit = 4, data = ABAB)

[Package SCRT version 1.3.1 Index]

P-value using the Monte Carlo procedure (2024)

FAQs

P-value using the Monte Carlo procedure? ›

The results of the Monte Carlo simulation are teh results that we would expect if the null hypothesis were true. So, to compute a p -value, you count the number of results that are at least as extreme as the observed result, and divide this by the total number of results. This value is then reported as a decimal value.

What is Monte Carlo's p-value? ›

The P-value is the relative ranking of the test statistic among the sample values from the Monte Carlo randomization. You can calculate P-values to see whether observed values are unusually large or small for the null distribution.

How to find p-value using empirical rule? ›

However, Davison and Hinkley (1997) give the correct formula for obtaining an empirical P value as (r+1)/(n+1). The reasoning is roughly as follows: if the null hypothesis is true, then the test statistics of the n replicates and the test statistic of the actual data are all realizations of the same random variable.

How to interpret Monte Carlo simulation results? ›

Interpreting the results of a Monte Carlo simulation

When a Monte Carlo simulation is run, it generates many possible outcomes, usually represented as a histogram or a graph. The histogram's x-axis represents the different outcomes, and the y-axis represents the number of times that outcome was generated.

What is the Monte Carlo significance test? ›

MONTE CARLO significance test procedures consist of the comparison of the observed data with random samples generated in accordance with the hypothesis being tested. A test criterion is chosen to facilitate this comparison.

What is the formula for the Monte Carlo method? ›

This integral is then calculated with the Monte Carlo method. P{X ∈ O} = ∫ IO(x)f(x)dx where IO(x) = { 1 if x ∈ O, 0 if x /∈ O. of a Rd-valued random variable, the components of which are random numbers.

What is probability using Monte Carlo? ›

A Monte Carlo simulation is a model used to predict the probability of a variety of outcomes when the potential for random variables is present. Monte Carlo simulations help to explain the impact of risk and uncertainty in prediction and forecasting models.

What is the formula for the Monte Carlo estimator? ›

∫ f(x)p(x)dx = E(f(X)) = I. f(Xi). This method, the method of evaluating the integration via simulating random points, is called the integration by Monte Carlo Simulation. An appealing feature of the Monte Carlo Simulation is that the statistical theory is rooted in the theory of sample average.

What is the formula for the p-value mean? ›

P-value defines the probability of getting a result that is either the same or more extreme than the other actual observations. The P-value represents the probability of occurrence of the given event. The formula to calculate the p-value is: Z=^p−p0√p0(1−p0)n Z = p ^ − p 0 p 0 ( 1 − p 0 ) n.

What is the p-value for dummies? ›

'P-value' is the probability of observing a value for getting three heads out of 3 tosses if our null hypothesis is true. We write P-value in short form as P-Value= P(Experiment results | H0 is true) or probability of getting a result of three heads out of three coin tosses if our null hypothesis is true.

What is an example of a p-value? ›

P-values are expressed as decimals and can be converted into percentage. For example, a p-value of 0.0237 is 2.37%, which means there's a 2.37% chance of your results being random or having happened by chance. The smaller the P-value, the more significant your results are.

Can we calculate p-value from standard deviation? ›

It is used to determine the statistical significance of a hypothesis . To calculate the p value from a given standard deviation , one would need to know the sample size , the test statistic , and the degrees of freedom .

How to calculate p-value from test statistic in Excel? ›

In the fx tab above the cells, enter the TTEST's formula =T. TEST(array1, array2, tails, type), replacing array1, array2, tails, and type with values or cell numbers such as B3, B6, B9, and so on, then press the Enter key to calculate the P-Value.

What is the p-value method for proportions? ›

The P-value is the probability of seeing a sample proportion at least as extreme as the one observed from the data if the null hypothesis is true. In the previous example, only sample proportions higher than the null proportion were evidence in favor of the alternative hypothesis.

What is the p-value of the empirical distribution? ›

Empirical p-value is calculated as SN (or S+1N+1) where S is the number of permutation tests with a “more extreme” (larger or smaller, depending on the null hypothesis) observed test statistic Ti than T0.

What is the Monte Carlo probability distribution? ›

Monte Carlo methods, or MC for short, are a class of techniques for randomly sampling a probability distribution. There are three main reasons to use Monte Carlo methods to randomly sample a probability distribution; they are: Estimate density, gather samples to approximate the distribution of a target function.

What is value at risk in Monte Carlo simulation? ›

Using Monte Carlo to Calculate Value At Risk (VaR) VaR is a measurement of the downside risk of a position based on the current value of a portfolio or security, the expected volatility and a time frame. It is most commonly used to determine both the probability and the extent of potential losses.

What is the empirical p-value of a test? ›

It tests the dataset to find out if it is unlikely to get the observed value if the null hypothesis is true. A high P-value favors null hypothesis. Normally, significance level, i.e., Type I error is used as the cutoff point.

References

Top Articles
Latest Posts
Article information

Author: Nathanial Hackett

Last Updated:

Views: 6231

Rating: 4.1 / 5 (52 voted)

Reviews: 83% of readers found this page helpful

Author information

Name: Nathanial Hackett

Birthday: 1997-10-09

Address: Apt. 935 264 Abshire Canyon, South Nerissachester, NM 01800

Phone: +9752624861224

Job: Forward Technology Assistant

Hobby: Listening to music, Shopping, Vacation, Baton twirling, Flower arranging, Blacksmithing, Do it yourself

Introduction: My name is Nathanial Hackett, I am a lovely, curious, smiling, lively, thoughtful, courageous, lively person who loves writing and wants to share my knowledge and understanding with you.