readMat() now preserves the dimension of cell arrays and empty
matrices produces by MATLAB. See below Bug Fixes for details.readMat() for the upcoming Matrix (>= 1.4.2).readMat() read empty MATLAB matrices as NULL. Now it preserves
the original data type and dimension, e.g. matrix(integer(), nrow = 0, ncol = 5). Thanks to Gordon Turner for the troubleshooting,
the bug fix, and the package test solving this.
readMat() lost the dimension for MATLAB cell arrays. Thanks to
Brian James for the troubleshooting, the bug fix, and the package
test solving this.
Matlab$getOption() produced "Error in base::getOption(...) : 'x'
must be a character string" in R-devel (to become R 3.6.0). Thank
you Kurt Hornik for the troubleshooting and the fix.readMat() detects when MAT v7.3 files are trying to be read
and gives an informative error explaining those are not supported
(Issue #23). Thanks to Thomas Beutlich (author of C library matio)
for (non-official) pointers on how to distinguish MAT v5 from MAT
v7.3 files.writeMat() gained argument fixNames = TRUE for automatically
replacing periods (.) in variable and field names with
underscores (_). Thanks to Eugenio Piasini at University College
London for suggesting this feature.
The MatlabServer.m script now outputs verbose/debug messages to
standard error (or more precisely MATLAB file #2) (Issue #27).
Internal writeCommand() now gives an informative error if an
unknown command is used. Previously it silently sent a 0L
command to MATLAB.
writeMat() could not write logical elements. They are now
converted to integers as MATLAB does not have a native logical /
boolean data type. Previously, writeMat() silently ignored
writing the data section of these logical elements resulting in a
MAT file that could not be parsed by MATLAB (although readMat()
could). Thanks to Eugenio Piasini for reporting on this problem.utils::setInternet2() in
help("Matlab").readMat() parses (effectively skips) also objects of array
class mxOPAQUE_CLASS (=17). The official 'MAT-File Format
R2015b' (Sept 2015) documentation does not describe this type. The
parsing/skipping was done by reverse engineering and clever
guesses. Thanks to GitHub user 'yakzan' for reporting on this type
of MAT files.Now readMat() parses also functional objects of array type
(class) 16. Note that the official 'MAT-File Format R2015b' (Sept
2015) documentation does not describe this type. I had to
implement the parser based on best guess on how the MAT file format
works in general, so it might not work in all cases. Thanks to
GitHub user 'anilatx' for reporting on this type of MAT files.
ROBUSTNESS: Now writeMat(pathname, ...) writes the MAT file
atomically by first writing to a temporary file which is renamed
only if successful.
ROBUSTNESS: Now writeMat() asserts that a maximum of 2^31-1 bytes
are written per variable, otherwise an informative error is thrown.
This byte limit is set by the MAT file format definition. Thanks
to GitHub user 'intelinsight' for reporting on this.
evaluate() gained argument capture for capturing MATLAB output
while evaluating MATLAB expressions. Thanks to Rohan Shah (GitHub
user 'rohan-shah') for contributing code for this.
ROBUSTNESS: Now the temporary filenames generated by MATLAB also includes the port number in addition to a semi-random sequence string. This should lower the risk for two MATLAB server using the same temporary filenames.
readMat() gave an error when reading empty sparse matrices.
Thanks to Gregory Jefferis at University of Cambridge for the bug
report and bug fix.Matlab$startServer() always overwrites any existing
MatlabServer.m and InputStreamByteWrapper.class to make sure
the most recent versions are always used. Previously, existing
files would be kept, requiring a manual removal in order to use
more resent versions of these files.readMat(..., sparseMatrixClass = "matrix") did not return the
correct results in all cases. Added more package tests.
readMat(..., sparseMatrixClass = "SparseM") did not handle
all-zero sparse matrices. Added package test for this.
readMat(..., sparseMatrixClass = "Matrix") would give "Error in
validObject(.Object) : invalid class "dgCMatrix" object: lengths of
slots 'i' and 'x' must match", whenever trying to read an sparse
matrix with all zeros. Added package test for this case. Thanks to
Abhinav Jauhri at Carnegie Mellon University for reporting on this
and donating a test MAT file.
Trying to retrieve a non-existing MATLAB variable using
getVariable() would crash the R-to-MATLAB communication if
remote = FALSE. Similarly, Trying to evaluate() a MATLAB
expression that is invalid or by other means fails would also break
the R-to-MATLAB bridge. The only solution is both cases was to
restart R and MATLAB. Thanks to GitHub user 'badbye' (yalei) for
the report.
InputStreamByteWrapper.java is no longer installed with this
package. It is only the precompiled InputStreamByteWrapper.class
file that is installed. Updated documentation accordingly.readMat() parsed an mxCELL_CLASS structure incorrectly if one
of its names was empty. Thanks Dieter Menne at Menne Biomed
Consulting in Germany for reporting on this.help("Matlab").Added argument workdir to Matlab$startServer() for setting the
working directory of MATLAB. Thanks to Steven Jaffe at Morgan
Stanley for the suggestion.
ROBUSTNESS: Under certain circumstances the MATLAB server script
(MatlabServer.m) would try to access variables that were not yet
defined. Thanks to Steven Jaffe at Morgan Stanley for spotting
this.
uncompressZlib() of readMat() opened several raw
connections without closing them. Added system package test.
Thanks Thilo Klein at University of Cambridge, UK, for the report.readMat() can now read all (zlib) compressed MAT files where it
previously gave an error message on: "INTERNAL ERROR: Failed to
decompress data [...] internal error -3 in memDecompress(2)".miUINT32), readMat()
would drop any characters that were not in the ASCII set
(0,32-127). This was overly conservative as it should at least have
kept 7-bit ASCII characters (in 0-127), e.g. newline characters
were dropped. This bug was introduced in R.matlab v1.2.0
(2008-02-12) when adding support for UTF-* strings via iconv().
Now we keep all 8-bit ASCII characters, which is also what the
MATLAB file format documentation suggests are used in
non-UTF-encoded text matrices. Thanks to Steven Pav (San Francisco,
CA) for reporting on this.mxCHAR_CLASS structures are now read slighly faster.finalize() is called on any deleted Matlab objects.setOption() for Matlab gave Error in UseMethod("setOption") :
no applicable method for 'setOption' applied to an object of class
"c('Options', 'Object')". Added a package system test for
this. Thanks to Griet Laenen at KU Leuven for reporting on this.Added option R.matlab::readMat/v4/textMatrixCollapse controlling
whether MAT v4 text matrixes are collapsed into strings by row
("byrow"; default), by column ("bycolumn") or not at all
("none").
Whenever there is an uncompress error in readMat(), it now tries
to infer what type of compression the buffer has by inspecting the
first two bytes and include the type in the error message.
readMat(..., drop = "singletonLists") would throw an error if the
singleton list dropped contained NULL and that NULL was assigned to
an element of an outer list, resulting in that element being
dropped.ROBUSTNESS: Now Matlab$startServer() asserts that all MATLAB
server files that are indeed successfully copied, iff missing.
open() for Matlab no longer generates warnings on
"socketConnection(...) [...] cannot be opened", which occured
while waiting/polling for the MATLAB server to startup and respond.
MatlabServer.m script would incorrectly consider MATLAB v8
and above as MATLAB v6. Thanks to Frank Stephen at NREL for
reporting on this and providing a patch.Now the R.matlab Package object is also available when the
package is only loaded (but not attached).
ROBUSTNESS: Added package system test asserting that readMat()
can be called without attaching the package.
Package now only imports R.oo (no longer attaches it). Also, the package now imports R.utils (used to suggest and attach it when needed). These updates make the package more "silent".
Dropped obsolete autoload():s.
readMat() throws on failed
decompression also includes the first and last bytes of the data
block that it tried to decompress. This will further help
troubleshooting.R.methodsS3::appendVarArgs().CONSISTENCY: Added argument drop to readMat() to control how
singleton dimensions of for instance nested lists are dropped or
not. The defaults are now such that R.matlab v2.0.4 is consistent
with R.matlab v1.7.1 (R.matlab v2.0.0-2.0.3 were not). Thanks
to Claudia Beleites at IPHT Jena, Germany for reporting on this.
Bumped package dependencies.
readMat() would produce an error when parsing an empty
mxCHAR_CLASS matrix with a 0x0 dimension. It is not clear
whether this is a valid MAT v5 structure or not, but I`ve added a
workaround. Thanks Claudia Beleites at IPHT Jena, Germany for
reporting on this.help("readMat").MATLAB instead of
Matlab.readMat() significant faster. More than 15x
speedup(!) is observed on MAT files with many small structures.
This was achieved by (i) dramatically decreasing the amount of
memory allocation and garbage collection needed by the internal raw
buffer mechanism, by (ii) dropping an internal rm() calls that
was called very often, as well as by (iii) minimizing the number of
times local functions and objects are created.Now readMat() also accepts a raw vector as input.
Now readMat() no longer generates warnings reporting on "In
readBin( con = rawBufferT, what = what, size = size, n = n, signed
= signed, : 'signed = FALSE' is only valid for integers of sizes 1
and 2."
help("readMat"). Added a section on 'Speed
performance'.\usage{} lines are at most 90 characters long.readMat() now support reading 64-bit integers [array type (class)
mxINT64_CLASS and mxUINT64_CLASS] on machines/versions of R
with .Machine$sizeof.long >= 8 or .Machine$sizeof.longlong >= 8.
Package utilizes new startupMessage() of R.oo.
Authors@R field to the DESCRIPTION.R CMD check no longer warns on global assignments.
Bumped up package dependencies.
readMat() could not read sparse matrices containing logical
values, only numerics. This was because the logical bit in the
Array Flags was not utilized by readMat(), which in turn was
because the file format documentation is rather vague on how to use
that bit. This means that readMat() now represents sparse
logical matrices using logical values, except when
sparseMatrixClass = "SparseM", because SparseM matrices can
only hold numeric values. Thanks to Irtisha Sinhg at Cornell
University for reporting on this.It is now possible to specify the decompression method for
readMat().
Now readMat() decompression error messages are more informative.
readMat(..., verbose = -111) gave an error on object zraw not
being found.SOFTWARE QUALITY:
readMat() and writeMat() to formal system tests of the package,
and made those examples simpler.memDecompress() of the
base package instead. However, just in case someone is still
running older versions of R, readMat() does indeed still look for
Rcompression as a fallback..Internal() calls.help(Matlab) on how to connect through
firewalls. Thanks Bas de Regt at University of Adelaide for the
suggestion.R CMD check NOTE that would show up in R v2.15.0 devel.The version 1.5.0 that was submitted to CRAN did not include the adjustments for startup messages. Thus, resubmitting this one to CRAN.
Updated dependencies on R and packages.
GENERALIZATION: Now readMat() utilizes base::memDecompress() to
uncompress compressed data structures, unless running R v2.9.x or
before, in case Rcompression::uncompress() is used, iff
installed. R v2.10.0 was released October 2009. Thanks Stephen
Eglen at University of Cambridge for reminding me about
memDecompress().
CLEANUP: Now using packageStartupMessage() instead of cat().
Added a namespace to the package, which will be more or less a requirement in the next major release of R.
CLEANUP: Now all references to the Rcompression package is
within the local uncompress() function of readMat(). This
makes the code more modular making it easier to implement
alternatives to Rcompression::uncompress().
example(readMat).imaginary (not partial imag) in calls to
complex().Now writeMat() gives an error if duplicated object names are
used, e.g. writeMat(..., A = 1, B = 2, A = 3).
Now writeMat() gives an error, not a warning, if there are
non-named objects.
writeMat() did not work
correctly. Thanks Claudia Beleites for the report.readMat() can read non-named objects, in
MATLAB it is only the load() function call that can read it but
not the plain 'load' call. Because of this, writeMat() will now
give a warning if it detects non-named objects. For instance, in
writeMat("foo.mat", x = a, y) the object a is named (as x)
whereas y is not. To name the objects and avoid the warning, use
writeMat("foo.mat", x = a, y = y). Thanks Claudia Beleites at
University of Trieste for reporting on this.writeMat() that it can only write
'uncompressed' MAT files.Clarified and corrected some sentences in help(Matlab).
Added MATLAB 7.11.0.584 (R2010b) to the list of confirmed version that are compatible with R.matlab.
MatlabServer.m script incorrectly referred to the
InputStreamByteWrapper class as
java.io.InputStreamByteWrapper. This hopefully solves previously
reports on obtaining "??? Undefined variable "java" or class
"java.io.InputStreamByteWrapper". Error in ==> MatlabServer at 262
reader = java.io.InputStreamByteWrapper(4096);". This bug was
introduced in R.matlab v1.2.5 (2009-08-25). Thanks Kenvor
Cothey at GMO LCC for the report.R.utilsR.utils
readMat() would throw an exception on "Error in dim(matrix) <-
dimensionsArray$dim : dims [product 1] do not match the length of
object [0]" in rare cases related to empty strings. Not sure if
those cases are legal, but added an ad hoc workaround for
them. Thanks Claude Flene at University of Turku for reporting
this.MatlabServer script reports it's version when started.Removed some minor ambiguities in help(Matlab) and updated
example(Matlab) with some more troubleshooting statements.
Added MATLAB 7.4.0 (R2007a), 7.7.0.471 (R2008b), and 7.10.0.499 (R2010a) to the list of confirmed version that are compatible with R.matlab. Thanks Ajay Tripathy at UC Berkeley and Marco Colombo at University of Edinburgh for reporting this.
MatlabServer.m saves variables using the function form,
i.e. save(). This solves the problem of having single quotation
marks in the pathname. Thanks Michael Q. Fan at NC State University
for reporting this problem.fixNames of help(readMat). Thanks
Stephen Eglen (University of Cambridge) for the suggestion.readMat() can read also nested miMATRIX
structures. Issue reported by Jonathan Chard at Mango Solutions,
UK.readMat() tries smaller and smaller initial buffers, before
giving in up. Also, it now starts with a buffer 3 (not 10) times
the compressed size. Thanks to Michael Sumner for the report.MatlabServer.m) gave the error
"Undefined function or method ServerSocket for input arguments of
type double.", when launching it via Matlab$startServer(). It
seems like import java.net.*, etc. does not work. A workaround
is to specify the full path for all Java classes,
e.g. java.net.ServerSocket. Thanks Nicolas Stadler for the
report.onWrite == NULL, then there is no longer an outer two-pass
scan for figuring out the size of MAT structure. However, instead
each internal writeDataElement() has to do a two-pass scan in
order to figure out the number of bytes before writing each
element.Arrays with more than two dimension would be written as vectors. Thanks Minho Chae for the report.
From v1.2.2, writeMat() would generate corrupt MAT files.
Reverted writeMat() back to v1.2.1 and updated as above.
example(readMat) so that it gives no error if the
optional Rcompression package is not installed.writeMat() that counts the number of bytes
to be written is skipped if argument onWrite is not given. This
means that writing to file, which is the most common use case, will
no longer involve the counting. Thank to Adam Grossman at Stanford
University for suggesting the modification and providing initial
code. He reports a substantial speedup in writing large files
(38 MB file).Removed internal debugging/asserting code.
Renamed the HISTORY file to NEWS.
readMat()/writeMat() we use an internal character vector to
represent the 8-bit ASCII character set. However, in R one cannot
represent ASCII 0x00 - it resolves to an empty string, and from R
v2.8.0 we will get a warning if we try. For now, we have replaced
the ASCII for 0x00 with the empty string (unless before R v2.6.0).
We might want to move away from using this ASCII vector for
intToChar()/charToInt() and instead make use of
intToUtf8()/utf8ToInt() or alternatively just
rawToChar()/charToRaw().Added support for reading compressed MAT files. It requires that the Rcompression package (Omegahat.org) is available.
Added support for also returning sparse matrices according to the representations given by either the Matrix or the SparseM package.
Added simple UTF-* string support. Conversion should be accurate when the R installation supports iconv.
Added mat-files/StructWithSparseMatrix-v4.mat from JR.
Credits for all of the above new feature goes to Jason Riedy at the Computer Science Dept, UC Berkeley. Thanks also to Duncan Temple-Lang at UC Davis for the Rcompression package.
readMat() does not support
compressed MAT files.Update so that package passes R CMD check on R v2.5.0.
Replaced Sys.putenv() with new Sys.setenv() (to be deprecated
in R v2.5.0).
Now open() throws an error if connection to MATLAB failed (before
FALSE was returned).
Extended the accepted range of ports to [1023,66535].
Added more details on available options in setOption() and how to
avoid timeouts.
The example in help(Matlab) is now using default values for
host and port making the example less confusing.
MATLABSERVER_PORT) was out
of range, a MATLAB syntax error occurred, i.e '... Line: 109
Column: 45 ")" expected, "identifier" found.' Thanks Alexander
Nervedi for the report.port to Matlab$startServer(..., port = NULL) to
start the MATLAB server such that it listens to another port than
the default 9999. Internally the server first tries to obtain the
port number by environment variable MATLABSERVER_PORT. This
feature was requested by Wang Yu at Iowa State University.?Matlab.Matlab class, the package requires the R.utils
where the classes Java and Verbose was moved. The methods for
dealing with user options in the Matlab class are provided by the
Options class also in R.utils. Similarly, if verbose != FALSE in readMat() or writeMat(), then the R.utils package
is required.Forgot to "implement" miSINGLE, i.e. to set knownTypes and
knownWhats for this. Thanks to Craig Aumann for the report.
readMat() did not read 'complex' matrices in MAT v5 correctly.
Thanks to Chris Sims for the report.
readMat() did not read 'text' matrices in MAT v4 correctly.
Thanks to Chris Sims, Princeton University, for the patch.
Added support for user-defined options to Matlab objects via
methods setOption() and getOption(). Current supported options
are "readResult/maxTries" and "readResult/interval" to provide
a "workaround" for the problem with timeout errors in evaluate().
Thanks Thomas Romary, France, for (re-)reporting on this problem.
Added class Options. This will probably move to another package
whenever it becomes stable and widely used.
misc/ with two new directories
mat-files/ and externals/.seq(0) and not seq(length = 0) was used. Updated all
occurrences of this problem.readMat() failed to read some MATLAB struct`s because of a
parsing error.
Internally in readMat() - in readMat5DataStructure() it might
be data readTag() returns a tag referring to a data block of zero
length (nbrOfBytes == 0). Now list(NULL) is returned in this
case.
The MatlabServer script did (indeed) not work with MATLAB v7; a
wrong version was included. Thanks Patrick Drechsler, University
of Wuerzburg for reporting the above two problems.
SIGNFICANT CHANGES:
MatlabServer.m script to automatically set the Java
classpath for MATLAB v7 and higher. This will assure that
InputStreamByteWrapper.class is found as long as it is in the
same directory as the *.m script. Thanks Yichun Wei at University
of Southern California for suggesting this and also for suggesting
additional help instructions.Added a Wishlist and an Acknowledgements section to "about".
Added more information on how to start the MATLAB server.
Matlab$startServer() and MATLAB v7.autoload("hasVarArgs") to R/000.R.Package now passes R CMD check on R v2.1.0 devel without warnings.
Added appendVarArgs = TRUE to setMethodS3(), which specifies
that ... should be added, if missing.
Add argument ... to all methods to make it even more consistent
with any generic function. This is also done for a few methods in R
base packages.
readMat() would not read non-signed bytes correctly.DOCMENTATION:
Updated description of package. Preparing for moving to CRAN.
Added Rdoc comments to internal Verbose classes.
Package passes R CMD check on R v2.1.0 devel too.
Moved the Java class from R.io to this package, since it is
currently only used here. It is used for reading and writing Java
bytes, ints and (UTF) strings when communicating with MATLAB.
Added a setVerbose() method to the Matlab class, which allows
the user to get details at various levels of the MATLAB process.
Added support for reading sparse matrices. Sparse matrices are expanded to regular matrices, which means that some may be too large to be loaded.
WORKAROUND: It sometimes happens that the reply from MATLAB is not
transferred "quick enough" and even if the connection should block
we receive NULL giving an error. The workaround for now is to try
to read the answer again. The symptom was that an error in
readResult() complain about "if (answer == 0)" where answer was
of length 0.
The header of MAT v4 files was not parsed correctly, which made MAT v4 files unrecognized and assumed to be MAT v5 files which in turn could not be read.
Used "sparce" instead of "sparse" in MOPT[4] tag (MAT v4),
which most likely would generate an error for such structure.
<- was used instead of <<- in one new
method. I do not like <<-, but that is how it works.Created an readMat() that can read both MAT v4 and MAT v5
files. The MAT v4 reader was kindly provided by Andy Jacobson at
the Princeton University.
writeMat() was created for consistency, but it does only support
MAT v5.
Note that I have now chosen to rename readMAT() and writeMAT()
to readMat() and writeMat(), respectively. This is in line with
the RCC, cf. readHtml() instead of readHTML(). If you really
need readMAT() and writeMAT() you can easily write your own
wrappers.
require(R.io) is loaded with the package and not when the
first MATLAB client/server method is called. The former approach
was a bit annoying.Java class of the R.io package. Correcting
these bugs made the MATLAB part work again.\keyword{internal}.readMAT() would not work because getBits() previously in
package R.oo was removed. getBits() is now added as a local
function inside readMAT().Initial methods are readMAT() and writeMAT() for reading and
writing MATLAB MAT file structures.
Initial class is Matlab, which provides static methods for communicating with the running MATLAB server.
Package created.