10.70 Save the Dataset

For large datasets we may want to save it to a binary RData file once we have wrangled it into the right shape and collected the metadata. Loading a binary dataset is generally quicker than loading a CSV file—a CSV file with 2 million observations and 800 variables can take 30 minutes to utils::read.csv(), 5 minutes to base::save(), and 30 seconds to base::load().

# Timestamp for the dataset.

dsdate  <- "_" %s+% format(Sys.Date(), "%y%m%d") %T>% print()

## [1] "_260901"

# Filename for the saved dataset

dsrdata <- dsname %s+% dsdate %s+% ".RData" %T>% print()

## [1] "weatherAUS_260901.RData"

# Save relevant R objects to binary RData file.

save(ds, dsname, dspath, dsdate, nobs,
     vars, target, risk, id, ignore, omit,
     inputi, inputs, numi, numc, cati, catc,
     file=dsrdata)

Notice that in addition to the dataset (ds) we also store the collection of metadata. This begins with items such as the name of the dataset, the source file path, the date we obtained the dataset, the number of observations, the variables of interest, the target variable, the name of the risk variable (if any), the identifiers, the variables to ignore and observations to omit. We continue with the indicies of the input variables and their names, the indicies of the numeric variables and their names, and the indicies of the categoric variables and their names.

Each time we wish to use the dataset we can now simply base::load() it into R. The value that is invisibly returned by base::load() is a vector naming the R objects loaded from the binary RData file.

load(dsrdata) %>% print()

##  [1] "ds"     "dsname" "dspath" "dsdate" "nobs"   "vars"   "target" "risk"  
##  [9] "id"     "ignore" "omit"   "inputi" "inputs" "numi"   "numc"   "cati"  
## [17] "catc"

We place the call to base::load() within a call to [(](https://www.rdocumentation.org/packages/base/topics/(){ target=“_blank” } (i.e., we have surrounded the call with round brackets) to ensure the result of the function call is printed. A call to base::load() returns its result invisibly since we are primarily interested in its side-effect. The side-effect is to read to R binary data from disk and to make it available within our current R session.

If you find this curated material useful then you can consider a donation to support it's ongoing availability and give you access to the PDF version of this book. The material has been scoped up by Generative AI without permission or any kind of recompense so do consider a donation if you can afford it. Unlike Generative AI your access to this materials is freely given. Desktop Survival Guides include Data Science, GNU/Linux, and MLHub. Books available on Amazon include Data Mining with Rattle and Essentials of Data Science. Togaware has a 30 year tradition of making popular open source software which includes sold privacy preserving productivity apps, rattle, wajig, and mlhub. Hosted by Togaware, a pioneer of free and open source software since 1984. Copyright © 1995-2022 Graham.Williams@togaware.com Creative Commons Attribution-ShareAlike 4.0