Title: | Test Suite for 'Future API' Backends |
---|---|
Description: | Backends implementing the 'Future' API, as defined by the 'future' package, should use the tests provided by this package to validate that they meet the minimal requirements of the 'Future' API. The tests can be performed easily from within R or from outside of R from the command line making it straightforward to include them in package tests and in Continuous Integration (CI) pipelines. |
Authors: | Henrik Bengtsson [aut, cre, cph], The R Consortium [fnd] (Project was awared an Infrastructure Steering Committee (ISC) grant in 2017) |
Maintainer: | Henrik Bengtsson <[email protected]> |
License: | LGPL (>= 2.1) |
Version: | 0.7.0 |
Built: | 2023-09-19 06:56:03 UTC |
Source: | https://github.com/HenrikBengtsson/future.tests |
Add a Future Plan to Test Against
add_test_plan(expr, substitute = TRUE)
expr |
... |
substitute |
... |
(invisibly) returns current list of test plans.
Evaluate an Expression Across A Set of Future Plans
along_test_plans(
expr,
substitute = TRUE,
envir = parent.frame(),
local = TRUE,
plans = test_plans()
)
expr |
An R expression. |
substitute |
... |
envir |
The environment where tests are run. |
local |
Should tests be evaluated in a local environment or not. |
plans |
A list of future plans. |
A list of results, one for each future plan tested against.
Run All or a Subset of the Tests Across Future Plans
check(
plan = NULL,
tags = character(),
timeout = NULL,
settings = TRUE,
session_info = FALSE,
envir = parent.frame(),
local = TRUE,
debug = FALSE,
exit_value = !interactive(),
.args = commandArgs()
)
plan |
(character vector) One or more future strategy plans to be validated. |
tags |
(character vector; optional) Filter test by tags. If NULL, all tests are performed. |
timeout |
(numeric; optional) Maximum time (in seconds) each test may run before a timeout error is produced. |
settings |
(logical) If TRUE, details on the settings are outputted before the tests start. |
session_info |
(logical) If TRUE, session information is outputted after the tests complete. |
envir |
The environment where tests are run. |
local |
Should tests be evaluated in a local environment or not. |
debug |
(logical) If TRUE, the raw test results are printed. |
exit_value |
(logical) If TRUE, and in a non-interactive session,
then use |
.args |
(character vector; optional) Command-line arguments. |
(list; invisible) A list of test results.
This function can be called from the shell. To specify an argument, use the
format --test-<arg_name>=<value>
. For example, --test-timeout=600
will
set argument timeout=600
, and --tags=lazy,rng
, or equivalently,
--tags=lazy --tags=rng
will set argument tags=c("lazy", "rng")
.
Here are some examples on how to call this function from the command line:
Rscript -e future.tests::check --args --test-plan=sequential Rscript -e future.tests::check --args --test-plan=multicore,workers=2 Rscript -e future.tests::check --args --test-plan=sequential --test-plan=multicore,workers=2 Rscript -e future.tests::check --args --test-plan=future.callr::callr Rscript -e future.tests::check --args --test-plan=future.batchtools::batchtools_local
The exit code will be 0 if all tests passed, otherwise 1. You
can use for instance exit_code=$?
to retrieve the exit code of the
most recent call.
## Not run:
results <- future.tests::check(plan = "sequential", tags = c("rng"))
exit_code <- attr(results, "exit_code")
if (exit_code != 0) stop("One or more tests failed")
## End(Not run)
Run All Tests
check_plan(
tests = test_db(),
defaults = list(),
timeout = getOption("future.tests.timeout", 30),
envir = parent.frame(),
local = TRUE
)
tests |
A list of tests to subset. |
defaults |
(optional) Named list with default argument values. |
timeout |
Maximum time allowed for evaluation before a timeout error is produced. |
envir |
The environment where tests are run. |
local |
Should tests be evaluated in a local environment or not. |
Nothing.
Evaluate an R Expression
evaluate_expr(
expr,
envir = parent.frame(),
local = TRUE,
output = c("stdout+stderr", "stdout", "none"),
timeout = +Inf
)
expr |
An expression |
envir |
The environment where tests are run. |
local |
Should tests be evaluated in a local environment or not. |
output |
Specifies whether standard output, standard error, or both should be captured or not. |
timeout |
Maximum time allowed for evaluation before a timeout error is produced. |
Value of test expression and benchmark information.
Loads Future Tests
load_tests(
path = ".",
recursive = TRUE,
pattern = "[.]R$",
root = getOption("future.tests.root", Sys.getenv("R_FUTURE_TESTS_ROOT",
system.file("test-db", package = "future.tests", mustWork = TRUE)))
)
path |
A character string specifying a test script folder or file. |
recursive |
If TRUE, test-definition scripts are search recursively. |
pattern |
Regular expression matching filenames to include. |
root |
(internal) An alternative file directory from where future.tests tests are sourced. |
(invisible) the value of test_db()
.
Make a Test
make_test(
expr,
title = NA_character_,
args = list(),
tags = NULL,
substitute = TRUE,
reset_workers = FALSE,
register = TRUE
)
expr , substitute
|
The expression to be tested and whether it is passes as an expression already or not. |
title |
(character) The title of the test. |
args |
(optional) Named arguments. |
tags |
(optional) Character vector of tags. |
reset_workers |
(optional) Specifies whether background workers should futures. |
register |
If TRUE, the test is registered in the test database, otherwise not. |
(invisibly) A Test.
Register a Test
register_test(test)
test |
A Test. |
(invisibly) The Test registered.
Run a Test
run_test(
test,
envir = parent.frame(),
local = TRUE,
args = list(),
defaults = list(),
output = "stdout+stderr",
timeout = getOption("future.tests.timeout", 30)
)
test |
A Test. |
envir |
The environment where tests are run. |
local |
Should tests be evaluated in a local environment or not. |
args |
Arguments used in this test. |
defaults |
(optional) Named list with default argument values. |
output |
If TRUE, standard output is captured, otherwise not. |
timeout |
Maximum time allowed for evaluation before a timeout error is produced. |
Value of test expression and benchmark information.
Run All Tests
run_tests(
tests = test_db(),
envir = parent.frame(),
local = TRUE,
defaults = list(),
output = "stdout+stderr"
)
tests |
A list of tests to subset. |
envir |
The environment where tests are run. |
local |
Should tests be evaluated in a local environment or not. |
defaults |
(optional) Named list with default argument values. |
output |
If TRUE, standard output is captured, otherwise not. |
List of test results.
Signals a TestSkipped
condition.
skip_test(..., domain = NULL)
... |
zero or more objects which can be coerced to character
(and which are pasted together with no separator) or (for
|
domain |
see |
(invisible) A base::condition of class TestSkipped
.
Identify Subset of Tests with Specified Tags and that Support Specified Argument Settings
subset_tests(tests = test_db(), tags = NULL, args = NULL, defaults = list())
tests |
A list of tests to subset. |
tags |
(optional) A character vector of tags that tests must have. |
args |
Named arguments with sets of values to test against. |
defaults |
(optional) Named list with default argument values. |
A list of tests that support specified arguments.