Certain types of objects are tied to a given R session. Such objects cannot be saved to file by one R process and then later be reloaded in another R process and expected to work correctly. If attempted, we will often get an informative error but not always. For the same reason, these type of objects cannot be exported to another R processes(*) for parallel processing regardless of which parallelization framework we use. We refer to these type of objects as “non-exportable objects”.
(*) The exception might be when forked processes
are used, i.e. plan(multicore). However, such attempts to
work around the underlying problem, which is non-exportable objects,
should be avoided and considered non-stable. Moreover, such code will
fail to parallelize when using other future backends.
An example of a non-exportable object is a connection, e.g. a file connection. For instance, if you create a file connection,
con <- file("output.log", open = "wb")
cat("hello ", file = con)
flush(con)
readLines("output.log", warn = FALSE)
## [1] "hello "it will not work when used in another R process. If we try, the result is “unknown”, e.g.
library(future)
plan(multisession)
f <- future({ cat("world!", file = con); flush(con) })
value(f)
## NULL
close(con)
readLines("output.log", warn = FALSE)
## [1] "hello "In other words, the output "world!" written by the R
worker is completely lost.
The culprit here is that the connection uses a so called external pointer:
which is bound to the main R process and makes no sense to the worker. Ideally, the R process of the worker would detect this and produce an informative error message, but as seen here, that does not always occur.
To help avoiding exporting non-exportable objects by mistakes, which typically happens because a global variable is non-exportable, the future framework provides a mechanism for automatically detecting such objects. To enable it, do:
options(future.globals.onReference = "error")
f <- future({ cat("world!", file = con); flush(con) })
## Error: Detected a non-exportable reference ('externalptr') in one of the globals
## ('con' of class 'file') used in the future expressionComment: The future.globals.onReference options
is set to "ignore" by default due to the extra overhead
"error" introduces, which can be significant for very large
nested objects. Furthermore, some subclasses of external pointers can be
exported without causing problems.
The below table and sections provide a few examples of non-exportable R objects that you may run into when trying to parallelize your code, or simply from trying reload objects saved in a previous R session. If you identify other cases, please consider reporting them so they can be documented here and possibly even be fixed.
| Package | Examples of non-exportable types or classes |
|---|---|
| arrow | Table (externalptr) |
| base | connection (externalptr) |
| bigmemory | big.matrix (externalptr) |
| cpp11 | E.g. functions created by
cpp_source() |
| DBI | DBIConnection (externalptr) |
| inline | CFunc (externalptr of class
DLLHandle) |
| keras | keras.engine.sequential.Sequential
(externalptr), keras.engine.base_layer.Layer
(externalptr) |
| magick | magick-image (externalptr) |
| ncdf4 | ncdf4 (custom reference; non-detectable) |
| parallel | cluster and cluster nodes
(connection) |
| polars | RPolarsDataFrame (externalptr) |
| raster | RasterLayer (externalptr; not
all) |
| Rcpp | NativeSymbol (externalptr) |
| reticulate | python.builtin.function (externalptr),
python.builtin.module (externalptr) |
| rJava | jclassName (externalptr) |
| ShortRead | FastqFile, FastqStreamer, FastqStreamerList
(connection) |
| sparklyr | tbl_spark (externalptr) |
| terra | SpatRaster, SpatVector (externalptr) |
| udpipe | udpipe_model (externalptr) |
| xgboost | xgb.DMatrix (externalptr) |
| XML | XMLInternalDocument, XMLInternalElementNode
(externalptr) |
| xml2 | xml_document (externalptr) |
These are illustrated in sections ‘Packages that rely on external pointers’ and ‘Packages with other types of non-external objects’ below.
Importantly, there are objects with external pointer than can indeed be exported. Here are some example,
| Package | Examples of exportable types or classes |
|---|---|
| data.table | data.table (externalptr) |
| rstan | stanmodel (externalptr) |
These are discussed in sections ‘False positives - packages with exportable external pointers’ at the very end of this vignette.
library(future)
plan(multisession, workers = 2)
cl <- parallel::makeCluster(2L)
y <- parSapply(cl, X = 2:3, FUN = sqrt)
y
## [1] 1.414214 1.732051
y %<-% parSapply(cl, X = 2:3, FUN = sqrt)
y
## Error in summary.connection(connection) : invalid connectionIf we turn on
options(future.globals.onReference = "error"), we will
catch this already when we create the future:
If an object carries an external pointer, it is likely that
it can only be used in the R session where it was created. If it is
exported to and used in a parallel process, it will likely cause an
error there. As shown above, and in below examples, setting option
future.globals.onReference to "error" will
make future to scan for external pointer:s
before launching the future on a parallel worker, and throw an error if
one is detected.
However, there are objects with external pointer:s that can
be exported, e.g. data.table objects of the data.table
package is one such example. In other words, the existence of a
external pointer is just a suggestion for an object being
non-exportable - it is not a sufficient condition.
Below are some examples of packages who produce non-exportable objects with external pointer:s.
The arrow package provides efficient in-memory storage of arrays and tables. However, these objects cannot be transferred as-is to a parallel worker.
library(arrow)
library(future)
plan(multisession)
data <- as_arrow_table(iris)
f <- future(dim(data))
v <- value(f)
#> Error: Invalid <Table>, external pointer to nullThis error takes place on the parallel worker. We could set
options(future.globals.onReference = "error") to have
future detect the problem before it sends the object
over to the parallel worker.
That said, the arrow package provides low-level
functions write_to_raw() and read_ipc_stream()
that can used to marshal and unmarshal arrow objects.
For example,
The bigmemory package provides mechanisms for working with very large matrices that can be updated in-place, which helps save memory. For example,
library(bigmemory)
g <- function(x) {
x[1,1] <- 42L
x
}
x <- big.matrix(nrow = 3, ncol = 2, type = "integer")
print(x[1,1])
#> [1] NA
void <- g(x)
print(x[1,1])
#> [1] 42Note how x was updated in-place. This is achieved by
big.matrix objects holds an external pointer to where the
matrix data is stored;
If we would try to use x in a parallel worker, then the
parallel worker crashes due to a bug in bigmemory,
e.g.
library(bigmemory)
library(future)
plan(multisession, workers = 2)
x <- big.matrix(nrow = 3, ncol = 2, type = "integer")
f <- future(dim(x), packages = "bigmemory")
value(f)
#> Error in unserialize(node$con) :
#> MultisessionFuture (<none>) failed to receive message results from
#> cluster RichSOCKnode #1 (PID 1746676 on localhost 'localhost'). The
#> reason reported was 'error reading from connection'. Post-mortem
#> diagnostic: No process exists with this PID, i.e. the localhost worker
#> is no longer alive. Detected a non-exportable reference
#> ('externalptr') in one of the globals ('x' of class 'big.matrix') used
#> in the future expression. The total size of the 1 globals exported is
#> 696 bytes. There is one global: 'x' (696 bytes of class 'S4')We can protected against this setting:
which gives:
Another example is cpp11, which allows us to easily create R functions that are implemented in C++, e.g.
cpp11::cpp_source(code = '
#include "cpp11/doubles.hpp"
using namespace cpp11;
[[cpp11::register]]
int my_length(doubles x) {
return x.size();
}
')so that:
x <- rnorm(10)
my_length(x)
## [1] 10However, this function cannot be exported to another R process:
DBI provides a unified database interface for communication between R and various database engines. Analogously to regular connections in R, DBIConnection objects can not safely be exported to another R process, e.g.
library(future)
options(future.globals.onReference = "error")
plan(multisession)
library(DBI)
con <- dbConnect(RSQLite::SQLite(), ":memory:")
dummy %<-% print(con)
## Error: Detected a non-exportable reference ('externalptr') in one of the globals
## ('con' of class 'SQLiteConnection') used in the future expressionAnother example is inline, which allows us to easily create R functions that are implemented in C and C++, e.g.
library(inline)
code <- "
int i;
for (i = 0; i < *n; i++) x[0] = x[0] + (i+1);
"
sum_1_to_n <- cfunction(signature(n="integer", x="numeric"), code, language = "C", convention = ".C")
y <- sum_1_to_n(10, 0)$x
print(y)
## 55However, if we would attempt to call sum_1_to_n() in a
future, we get an error:
library(future)
plan(cluster, workers = 1L)
f <- future(sum_1_to_n(10, 0))
v <- value(f)
## Error in .Primitive(".C")(<pointer: (nil)>, n = as.integer(n), x = as.double(x)) :
## NULL value passed as symbol addressThis is because:
The keras package provides an R interface to Keras, which “is a high-level neural networks API developed with a focus on enabling fast experimentation”. The R implementation accesses the Keras Python API via reticulate. However, Keras model instances in R make use of R connections and external pointers, which prevents them from being exported to external R processes. For example, if the attempt to use a Keras model in a multisession workers, the worker will produce a run-time error:
library(keras)
library(future)
plan(multisession)
## Adopted from the 'keras' vignettes
inputs <- layer_input(shape = shape(32))
outputs <- layer_dense(inputs, units = 1L)
model <- keras_model(inputs, outputs)
model <- compile(model, optimizer = "adam", loss = "mean_squared_error")
test_input <- array(runif(128 * 32), dim = c(128, 32))
test_target <- array(runif(128), dim = c(128, 1))
fit(model, test_input, test_target)
f <- future({
stats::predict(model, test_input)
}, seed = TRUE)
pred <- value(f)
## Error in do.call(object$predict, args) :
## 'what' must be a function or character stringThis is error message is not very helpful. But, if we turn on
options(future.globals.onReference = "error"), we get more
clues;
Error: Detected a non-exportable reference ('externalptr') in one of the
globals ('model' of class 'keras.engine.functional.Functional') used in
the future expressionFunctions serialize_model() and
unserialize_model() of the keras package
can be used as workaround to marshal and unmarshal non-exportable
keras objects, e.g.
The magick package provides an R-level API for ImageMagick to work with images. When working with this API, the images are represented internally as external pointers of class ‘magick_image’ that cannot be be exported to another R process, e.g.
library(future)
plan(multisession)
library(magick)
frink <- magick::image_read("https://jeroen.github.io/images/frink.png")
f <- future(image_fill(frink, "orange", "+100+200", 20))
v <- value(f)
## Error: Image pointer is dead. You cannot save or cache image objects
## between R sessions.If we set:
we’ll see that this is caught even before attempting to run this in parallel;
The polars package provides objects for performant processing on tabular data. However, these objects are tied to the R process that created them. If we attempt to use them in a parallel worker, we end up crashing the parallel worker:
library(future)
plan(multisession)
library(polars)
data <- as_polars_df(data.frame(x = 1:3))
f <- future(dim(data), packages = "polars")
v <- value(f)
#> Error: Execution halted with the following contexts
#> 0: In R: in `$.RPolarsDataFrame`
#> 0: During function call [workRSOCK()]
#> 1: This Polars object is not valid. Execute `rm(<object>)` to remove
#> the object or restart the R session.This is because the external pointer in the
RPolarsDataFrame object is erased when transferred to
another process, which polars (>= 0.15.0) detects
and gives an informative error message about.
The raster package provides methods for working with spatial data, which are held in ‘RasterLayer’ objects. Not all but some of these objects use an external pointer. For example,
library(future)
plan(multisession)
options(future.globals.onReference = "error")
library(raster)
r <- raster(system.file("external/test.grd", package = "raster"))
tf <- tempfile(fileext = ".grd")
s <- writeStart(r, filename = tf, overwrite = TRUE)
f <- future({
print(dim(r))
print(dim(s))
})
Error: Detected a non-exportable reference ('externalptr') in one of the
globals ('s' of class 'RasterLayer') used in the future expressionNote that it is only the RasterLayer object s that
carries an external pointer. If we dig deeper, we find that this is
because attr(s@file, "con") is file connection opened for
writing. This is why s cannot be passed on to an external
worker. In contrast, RasterLayer object r does not have
this problem and would be fine to pass on to a worker.
Similarly to cpp11, Rcpp can be use to create R functions that are implemented in C++, e.g.
Rcpp::sourceCpp(code = '
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
int my_length(NumericVector x) {
return x.size();
}
')so that:
x <- 1:10
my_length(x)
## [1] 10However, since this function uses an external pointer internally, we cannot pass it to another R process:
library(future)
plan(multisession)
x <- rnorm(10)
n %<-% my_length(x)
n
## Error in .Call(<pointer: (nil)>, x) : NULL value passed as symbol addressWe can detect and protect against this using:
The reticulate package provides methods for creating and calling Python code from within R. If one attempt to use Python-binding objects from this package, we get errors like:
library(future)
plan(multisession)
library(reticulate)
os <- import("os")
pwd %<-% os$getcwd()
pwd
## Error in eval(quote(os$getcwd()), new.env()) :
## attempt to apply non-functionand by telling the future package to validate globals further, we get:
options(future.globals.onReference = "error")
pwd %<-% os$getcwd()
## Error: Detected a non-exportable reference ('externalptr') in one of the
## globals ('os' of class 'python.builtin.module') used in the future expressionAnother reticulate example is when we try to use a Python function that we create ourselves as in:
cat("def twice(x):\n return 2*x\n", file = "twice.py")
source_python("twice.py")
twice(1.2)
## [1] 2.4
y %<-% twice(1.2)
y
## Error in unserialize(node$con) :
## Failed to retrieve the value of MultisessionFuture from cluster node #1
## (on 'localhost'). The reason reported was 'error reading from connection'which, again, is because:
Here is an example that shows how rJava objects cannot be exported to external R processes.
library(future)
plan(multisession)
library(rJava)
.jinit() ## Initialize Java VM on master
Double <- J("java.lang.Double")
d0 <- new(Double, "3.14")
d0
## [1] "Java-Object{3.14}"
f <- future({
.jinit() ## Initialize Java VM on worker
new(Double, "3.14")
})
d1 <- value(f)
d1
## [1] "Java-Object<null>"Although no error is produced, we see that the value d1
is a Java NULL Object. As before, we can catch this by using:
The ShortRead package from Bioconductor implements efficient methods for sampling, iterating, and reading FASTQ files. Some of the helper objects used cannot be saved to file or exported to a parallel worker, because they comprise of connections and other non-exportable objects.
Here is an example that illustrates how an attempt to use a ‘FastqStreamer’ object created in the main R session fails when used in a parallel worker:
library(future)
plan(multisession)
# Adopted from example("FastqStreamer", package = "ShortRead")
library(ShortRead)
sp <- SolexaPath(system.file("extdata", package = "ShortRead"))
fl <- file.path(analysisPath(sp), "s_1_sequence.txt")
fs <- FastqStreamer(fl, 50)
reads %<-% yield(fs)
reads
## Error in status(update = TRUE) : invalid FastqStreamerTo catch this earlier, and to get a more informative error message, we do as before;
library(future)
plan(multisession)
library(sparklyr)
sc <- spark_connect(master = "local")
file <- system.file("misc", "exDIF.csv", package = "utils")
data <- spark_read_csv(sc, "exDIF", file)
d %<-% dim(data)
d
## Error in unserialize(node$con) :
## Failed to retrieve the value of MultisessionFuture (<none>) from cluster
## SOCKnode #1 (PID 29864 on localhost 'localhost'). The reason reported was
## 'unknown input format'. Post-mortem diagnostic: A process with this PID
## exists, which suggests that the localhost worker is still alive.To catch this as soon as possible,
library(future)
plan(multisession)
library(terra)
file <- system.file("ex/lux.shp", package = "terra")
v <- vect(file)
dv %<-% dim(v)
dv
Error in x@ptr$nrow() : external pointer is not valid
file <- system.file("ex/elev.tif", package = "terra")
r <- rast(file)
dr %<-% dim(r)
dr
## Error in .External(list(name = "CppMethod__invoke_notvoid", address = <pointer: (nil)>, :
## NULL value passed as symbol addressTo catch this as soon as possible,
options(future.globals.onReference = "error")
dv %<-% dim(v)
## Error: Detected a non-exportable reference ('externalptr' of class
## 'RegisteredNativeSymbol') in one of the globals ('v' of class
## 'SpatVector') used in the future expression
dr %<-% dim(data)
## Error: Detected a non-exportable reference ('externalptr' of class
## 'RegisteredNativeSymbol') in one of the globals ('r' of class
## 'SpatRaster') used in the future expressionFunctions wrap() and unwrap() of the
terra package can be used as workaround to marshal and
unmarshal non-exportable terra objects, e.g.
library(future)
plan(multisession)
library(terra)
file <- system.file("ex/lux.shp", package = "terra")
v <- vect(file)
.v <- wrap(v) ## marshal
dv %<-% {
v <- unwrap(.v) ## unmarshal
dim(v)
}
rm(.v) ## not needed anymore
dv
[1] 12 6and
file <- system.file("ex/elev.tif", package = "terra")
r <- rast(file)
.r <- wrap(r) ## marshal
dr %<-% {
r <- unwrap(.r) ## unmarshal
dim(r)
}
rm(.r) ## not needed anymore
dr
[1] 90 95 1For more details, see
help("wrap", package = "terra").
library(future)
plan(multisession)
library(udpipe)
udmodel <- udpipe_download_model(language = "dutch")
udmodel <- udpipe_load_model(file = udmodel$file_model)
x %<-% udpipe_annotate(udmodel, x = "Ik ging op reis en ik nam mee.")
x
## Error in udp_tokenise_tag_parse(object$model, x, doc_id, tokenizer, tagger, :
## external pointer is not validTo catch this as soon as possible,
options(future.globals.onReference = "error")
x %<-% udpipe_annotate(udmodel, x = "Ik ging op reis en ik nam mee.")
## Error: Detected a non-exportable reference ('externalptr') in one of the
## globals ('udmodel' of class 'udpipe_model') used in the future expressionNow, it is indeed possible to parallelize udpipe calls. For details on how to do this, see the ‘UDPipe Natural Language Processing - Parallel’ vignette that comes with the udpipe package.
The xgboost package provides fast gradient-boosting methods. Some of its data structures use external pointers. For example,
library(future)
plan(multisession)
library(xgboost)
data(agaricus.train, package = "xgboost")
train <- xgb.DMatrix(agaricus.train$data, label = agaricus.train$label)
class(train)
## [1] "xgb.DMatrix"
d <- dim(train)
d
## [1] 6513 126works just fine but if we attempt to pass on the ‘xgb.DMatrix’ object
train to an external worker, we silently get a incorrect
value:
This is unfortunate, but we can at least detect this by:
options(future.globals.onReference = "error")
f <- future(dim(dtrain))
## Error: Detected a non-exportable reference ('externalptr' of class 'xgb.DMatrix')
## in one of the globals ('dtrain' of class 'xgb.DMatrix') used in the future expressionThis is because train itself is an external pointer,
i.e. mode(train) == "externalptr".
Another example is XML objects of the XML package, which may produce evaluation error, or even cause R to abort if used in another R process, e.g.
library(future)
plan(multisession)
library(XML)
doc <- xmlParse(system.file("exampleData", "tagnames.xml", package = "XML"))
a <- getNodeSet(doc, "/doc//a[@status]")[[1]]
f <- future(xmlGetAttr(a, "status"))
value(f)
## Error in unserialize(node$con) :
## MultisessionFuture (<none>) failed to receive results from cluster
## RichSOCKnode #1 (PID 31541 on localhost 'localhost'). The reason
## reported was 'error reading from connection'. Post-mortem diagnostic:
## No process exists with this PID, i.e. the localhost worker is no
## longer alive. Detected a non-exportable reference ('externalptr' of
## class 'XMLInternalElementNode') in one of the globals ('a' of class
## 'XMLInternalElementNode') used in the future expression. The total
## size of the 1 globals exported is 520 bytes. There is one global: 'a'
## (520 bytes of class 'externalptr')This is an example, where we end up exporting an
XMLInternalElementNode object to another R process, where
it is no longer valid. When we try to use it there by calling
xmlGetAttr() on it, XML causes R to crash
and abort. To illustrate what’s going on the parallel workers, if we
save the object to file using saveRDS(a, "a.rds"), and try
to use it in another R session, the following happens:
$ R --quiet --vanilla
> a <- readRDS("a.rds")
> XML::xmlGetAttr(a, "status")
*** caught segfault ***
address 0x40, cause 'memory not mapped'
Traceback:
1: xmlAttrs.XMLInternalNode(node, addNamespace)
2: xmlAttrs(node, addNamespace)
3: XML::xmlGetAttr(a, "status")
Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection:This is a very harsh way of telling us that we cannot export all types of objects produced by XML. Ideally, XML would detect this and give am informative error message and not crash R like this.
A workaround for working around this is to marshal the problematic objects before exporting them to a parallel R process, and unmarshal them before working with them there. For example,
library(future)
plan(multisession)
library(XML)
doc <- xmlParse(system.file("exampleData", "tagnames.xml", package = "XML"))
a <- getNodeSet(doc, "/doc//a[@status]")[[1]]
## Marshall the non-exportable XMLInternalElementNode object
.a <- xmlSerializeHook(a) ## marshal
f <- future({
a <- xmlDeserializeHook(.a) ## unmarshal
xmlGetAttr(a, "status")
})
value(f)
## [1] "xyz"An alternative, more generic workaround, is to always create the
doc element, an XMLInternalDocument object, on
the parallel workers, i.e.
Yet another example is XML objects of the xml2 package, which may produce evaluation errors (or just invalid results depending on how they are used), e.g.
library(future)
plan(multisession)
library(xml2)
doc <- read_xml("<body></body>")
f <- future(xml_children(doc))
value(f)
## Error: external pointer is not validThe future framework can help detect this before sending off the future to the worker;
options(future.globals.onReference = "error")
f <- future(xml_children(xml))
## Error: Detected a non-exportable reference ('externalptr') in one of the
## globals ('xml' of class 'xml_document') used in the future expressionOne workaround when dealing with non-exportable objects is to look
for ways to encode the object such that it can be exported, and the
decoded on the receiving end. With xml2, we can use
xml2::xml_serialize() and
xml2::xml_unserialize() to do this. Here is how we can
rewrite the above example such that we can pass xml2
object back and forth between the main R session and R workers:
## Encode the 'xml_document' object 'doc' as a 'raw' object
.doc <- xml_serialize(doc, connection = NULL) ## marshal
f <- future({
## In the future, reconstruct the 'xml_document' object
## from the 'raw' object
doc <- xml_unserialize(.doc) ## unmarshal
## Continue as usual
children <- xml_children(doc)
## Send back a 'raw' representation of the 'xml_nodeset'
## object 'children'
xml_serialize(children, connection = NULL)
})
## Reconstruct the 'xml_nodeset' object in the main R session
children <- xml_unserialize(value(f))Package ncdf4 provides an R API to work with data that live in netCDF files. For example, we can create a simple netCDF file that holds a variable ‘x’:
library(ncdf4)
x <- ncvar_def("x", units = "count", dim = list())
file <- nc_create("example.nc", x)
ncvar_put(file, x, 42)
nc_close(file)We can now use this netCDF file next time we start R, e.g.
However, it would fail if we attempt to use file, which
is an object of class ‘ncdf4’, in a parallel worker, we will get an
error:
library(future)
plan(multisession)
library(ncdf4)
file <- nc_open("example.nc")
f <- future(ncvar_get(file))
y <- value(f)
## Error in R_nc4_inq_varndims: NetCDF: Not a valid ID
## Error in ncvar_ndims(ncid, varid) : error returned from C callThis is because ncdf4 objects make use of internal references that
are unique to the R session where they were created. However, these are
not formal external pointer:s, meaning the future
framework cannot detect them. That is, using
options(future.globals.onReference = "error") is of no help
here.
A workaround is to open the netCDF in each worker, e.g.
The data.table package creates objects comprising external pointers. Contrary to above non-exportable examples, such objects can be saved to file and used in another R session, or exported to a parallel worker. This is because data.table is capable of restoring these objects to a valid state. Consider the following example:
library(data.table)
DT <- data.table(a = 1:3, b = letters[1:3])
## Extract second row
row <- DT[2]
print(row)
#> a b
#> 1: 2 bIf we would try the last step with a future with strict checking for references enabled, we would get an error:
library(future)
plan(multisession)
options(future.globals.onReference = "error")
row %<-% DT[2]
Error: Detected a non-exportable reference ('externalptr') in one of
the globals ('DT' of class 'data.table') used in the future expressionThis is a false positive. If we relax the checks, it does indeed work:
The rstan
package creates objects comprising external pointers. Contrary to above
non-exportable examples, such objects can be saved to file and used in
another R session, or exported to a parallel worker. This is because
rstan is capable of restoring these objects to a valid
state. Consider the following example from
example("rstan", package = "rstan"):
library(rstan)
code <- "
data {
int<lower=0> N;
real y[N];
}
parameters {
real mu;
}
model {
target += normal_lpdf(mu | 0, 10);
target += normal_lpdf(y | mu, 1);
}
"
y <- rnorm(20)
data <- list(N = 20, y = y)
fit <- stan(model_code = code, model_name = "example",
data = data, iter = 2012L, chains = 3L,
sample_file = file.path(tempdir(), "norm.csv"))
e <- extract(fit, permuted = FALSE)If we would try the last step with a future with strict checking for references enabled, we would get an error:
library(future)
plan(multisession)
options(future.globals.onReference = "error")
e %<-% extract(fit, permuted = FALSE)
Error: Detected a non-exportable reference ('externalptr' of class 'DLLHandle')
in one of the globals ('fit' of class 'stanfit') used in the future expressionHowever, this is a false positive. The fit object, which
is of class ‘stanfit’, can indeed be exported to be used in an external
R process, e.g.
options(future.globals.onReference = NULL)
e %<-% extract(fit, permuted = FALSE)
str(e)
#> num [1:1006, 1:3, 1:2] -0.3028 -0.4017 -0.3379 -0.2358 0.0443 ...
#> - attr(*, "dimnames")=List of 3
#> ..$ iterations: NULL
#> ..$ chains : chr [1:3] "chain:1" "chain:2" "chain:3"
#> ..$ parameters: chr [1:2] "mu" "lp__"