The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Tutorial 2: Using metadata_ondisc_matrix and multimodal_ondisc_matrix

Tutorial 1 covered ondisc_matrix, the core class implemented by ondisc. Thus tutorial covers metadata_ondisc_matrix and multimodal_ondisc_matrix, two additional classes provided by the package. metadata_ondisc_matrix stores cell-specific and feature-specific covariate matrices alongside the expression matrix, and multimodal_ondisc_matrix stores multiple metadata_ondisc_matrices representing different cellular modalities. Together, metadata_ondisc_matrix and multimodal_ondisc_matrix facilitate feature selection, quality control, and other common single-cell data preprocessing tasks.

The `metadata_ondisc_matrix` class

A metadata_ondisc_matrix object consists of three components: (i) an ondisc_matrix representing the expression data, (ii) a data frame storing the cell-specific covariates, and (iii) a data frame storing the feature-specific covariates. The easiest way to initialize a metadata_ondisc_matrix is by calling create_ondisc_matrix_from_mtx on an mtx file and associated metadata files, setting the optional parameter return_metadata_ondisc_matrix to TRUE. Below, we reproduce the example from Tutorial 1, this time returning a metadata_ondisc_matrix instead of a list.

# Set paths to the .mtx and .tsv files
raw_data_dir <- system.file("extdata", package = "ondisc")
mtx_fp <- paste0(raw_data_dir, "/gene_expression.mtx")
barcodes_fp <- paste0(raw_data_dir, "/cell_barcodes.tsv")
features_fp <- paste0(raw_data_dir, "/genes.tsv")

# Specify directory in which to store the .h5 file
temp_dir <- tempdir()

# Initialize metadata_ondisc_matrix
expressions <- create_ondisc_matrix_from_mtx(mtx_fp = mtx_fp,
                                              barcodes_fp = barcodes_fp,
                                              features_fp = features_fp,
                                              on_disk_dir = temp_dir,
                                              return_metadata_ondisc_matrix = TRUE)
#> 
|========                                                                 | 11%
|=================                                                        | 23%
|==========================                                               | 36%
|====================================                                     | 48%
|=============================================                            | 61%
|======================================================                   | 73%
|===============================================================          | 86%
|=========================================================================| 98%
|=========================================================================| 100%
#> 
|========                                                                 | 11%
|=================                                                        | 23%
|==========================                                               | 36%
|====================================                                     | 48%
|=============================================                            | 61%
|======================================================                   | 73%
|===============================================================          | 86%
|=========================================================================| 98%
|=========================================================================| 100%
#> Writing CSC data.
#> Writing CSR data.

The variable expressions is an object of class metadata_ondisc_matrix; expressions contains the fields ondisc_matrix, cell_covariates, and feature_covariates.

# Print the variable
expressions
#> A metadata_ondisc_matrix with the following components:
#>  An ondisc_matrix with 300 features and 900 cells.
#>  A cell covariate matrix with columns n_nonzero, n_umis, p_mito.
#>  A feature covariate matrix with columns mean_expression, coef_of_variation, n_nonzero.

We alternately can initialize a metadata_ondisc_matrix by calling the constructor function of the metadata_ondisc_matrix class; see documentation (via ?metadata_ondisc_matrix) for details.

The `multimodal_ondisc_matrix` class

The multimodal_ondisc_matrix class is used to represent multimodal data. multimodal_ondisc_matrix objects have two fields: (i) a named list of metadata_ondisc_matrices representing different modalities, and (ii) a global (i.e., cross-modality) cell-specific covariate matrix. The ondisc package ships with example CRISPR perturbation data, which we use to initialize a new perturbation modality via a call to create_ondisc_matrix_from_mtx.

# Set paths to the perturbation .mtx and .tsv files
mtx_fp <- paste0(raw_data_dir, "/perturbation.mtx")
barcodes_fp <- paste0(raw_data_dir, "/cell_barcodes.tsv")
features_fp <- paste0(raw_data_dir, "/guides.tsv")

# Initialize metadata_ondisc_matrix
perturbations <- create_ondisc_matrix_from_mtx(mtx_fp = mtx_fp,
                                               barcodes_fp = barcodes_fp,
                                               features_fp = features_fp,
                                               on_disk_dir = temp_dir,
                                               return_metadata_ondisc_matrix = TRUE)
#> 
#> Writing CSC data.
#> Writing CSR data.

Like expressions, the variable perturbations is an object of class metadata_ondisc_matrix. However, because perturbations represents logical perturabtion data instead of integer gene expression data, the cell-specific and feature-specific covariates of perturbations differ from those of expressions.

# These matrices have different columns
head(expressions@cell_covariates)
#>   n_nonzero n_umis     p_mito
#> 1        43    214 0.04672897
#> 2        26    169 0.00000000
#> 3        22    116 0.05172414
#> 4        37    258 0.08139535
#> 5        36    224 0.08035714
#> 6        31    147 0.07482993
head(perturbations@cell_covariates)
#>   n_nonzero
#> 1        21
#> 2        11
#> 3        17
#> 4        14
#> 5         7
#> 6        11

The expressions and perturbations data are multimodal – they are collected from the same set of cells. We can create a multimodal_ondisc_matrix by passing a named list of metadata_ondisc_matrix objects – in this case, expressions and perturbations – to the constructor function of the multimodal_ondisc_matrix class.

modality_list <- list(expressions = expressions, perturbations = perturbations)
crispr_experiment <- multimodal_ondisc_matrix(modality_list)

The variable crispr_experiment is an object of class multimodal_ondisc_matrix. The column names of the global covariate matrix are derived from the names of the modalities.

# print variable
crispr_experiment
#> A multimodal_ondisc_matrix with the following modalities:
#> 1. expressions A metadata_ondisc_matrix with the following components:
#>  An ondisc_matrix with 300 features and 900 cells.
#>  A cell covariate matrix with columns n_nonzero, n_umis, p_mito.
#>  A feature covariate matrix with columns mean_expression, coef_of_variation, n_nonzero.
#> 2. perturbations A metadata_ondisc_matrix with the following components:
#>  An ondisc_matrix with 250 features and 900 cells.
#>  A cell covariate matrix with columns n_nonzero.
#>  A feature covariate matrix with columns n_nonzero.
#> Plus, a global cell-covariate matrix containing 4 covariates and 900 cells.

# show the global covariate matrix
head(crispr_experiment@global_cell_covariates)
#>   expressions_n_nonzero expressions_n_umis expressions_p_mito
#> 1                    43                214         0.04672897
#> 2                    26                169         0.00000000
#> 3                    22                116         0.05172414
#> 4                    37                258         0.08139535
#> 5                    36                224         0.08035714
#> 6                    31                147         0.07482993
#>   perturbations_n_nonzero
#> 1                      21
#> 2                      11
#> 3                      17
#> 4                      14
#> 5                       7
#> 6                      11

The figure below summarizes the relationship between ondisc_matrix, metadata_ondisc_matrix, and multimodal_ondisc_matrix.

Figure: Classes provided by the package. a) ondisc_matrix, b) metadata_ondisc_matrix, c) multimodal_ondisc_matrix

Querying basic information

We can use the functions get_feature_ids, get_feature_names, and get_cell_barcodes to obtain the feature IDs, feature names (if applicable), and cell barcodes, respectively, of a metadata_ondisc_matrix or a multimodal_ondisc_matrix. get_feature_ids and get_feature_names return a list when called on a multimodal_ondisc_matrix, as the different modalities contain different features.

# metadata_ondisc_matrix
cell_barcodes <- get_cell_barcodes(expressions)
feature_ids <- get_feature_ids(expressions)
feature_names <- get_feature_names(expressions)

# multimodal_ondisc_matrix
cell_barcodes <- get_cell_barcodes(crispr_experiment)
feature_ids <- get_feature_ids(crispr_experiment)

We likewise can use dim, nrow, and ncol to query the dimension, number of rows, and number of columns of a metadata_ondisc_matrix or multimodal_ondisc_matrix. dim and nrow again return lists when called on a multimodal_ondisc_matrix.

# metadata_ondisc_matrix
dim(expressions)
#> [1] 300 900
nrow(expressions)
#> [1] 300
ncol(expressions)
#> [1] 900

# multimodal_ondisc_matrix
dim(crispr_experiment)
#> $expressions
#> [1] 300 900
#> 
#> $perturbations
#> [1] 250 900
nrow(crispr_experiment)
#> $expressions
#> [1] 300
#> 
#> $perturbations
#> [1] 250
ncol(crispr_experiment)
#> [1] 900

Subsetting

Similar to ondisc_matrices, metadata_ondisc_matrices and multimodal_ondisc_matrices can be subset using the [ operator. metadata_ondisc_matrices can be subset either by feature or cell, while multimodal_ondisc_matrices can be subset by cell only.

# metadata_ondisc_matrix
# keep cells 100 - 150
expressions_sub <- expressions[,100:150]
# keep genes ENSG00000188305, ENSG00000257284, ENSG00000251655
expressions_sub <- expressions[c("ENSG00000188305", "ENSG00000257284", "ENSG00000251655"),]

# multimodal_ondisc_matrix
# keep all cells except 1 - 100
crispr_experiment_sub <- crispr_experiment[,-c(1:100)]

As with ondisc_matrices, the original objects remain unchanged.

expressions
#> A metadata_ondisc_matrix with the following components:
#>  An ondisc_matrix with 300 features and 900 cells.
#>  A cell covariate matrix with columns n_nonzero, n_umis, p_mito.
#>  A feature covariate matrix with columns mean_expression, coef_of_variation, n_nonzero.
crispr_experiment
#> A multimodal_ondisc_matrix with the following modalities:
#> 1. expressions A metadata_ondisc_matrix with the following components:
#>  An ondisc_matrix with 300 features and 900 cells.
#>  A cell covariate matrix with columns n_nonzero, n_umis, p_mito.
#>  A feature covariate matrix with columns mean_expression, coef_of_variation, n_nonzero.
#> 2. perturbations A metadata_ondisc_matrix with the following components:
#>  An ondisc_matrix with 250 features and 900 cells.
#>  A cell covariate matrix with columns n_nonzero.
#>  A feature covariate matrix with columns n_nonzero.
#> Plus, a global cell-covariate matrix containing 4 covariates and 900 cells.

Notes and tips

To save and load a metadata_ondisc_matrix or multimodal_ondisc_matrix, simply use the functions saveRDS and readRDS, respectively.
It is not possible to apply the double-bracket “pull submatrix into memory” operator to a metadata_ondisc_matrix or multimodal_ondisc_matrix. To apply the double-bracket operator to an ondisc_matrix stored inside a metadata_ondisc_matrix or multimodal_ondisc_matrix, simply access the desired ondisc_matrix first.
The metadata_ondisc_matrix and multimodal_ondisc_matrix classes are inspired by analogous classes in the SingleCellExperiment and Seurat packages. However, unlike SingleCellExperiment and Seurat, ondisc does not (currently) support storing the dense matrix of normalized UMI counts. We view this as a feature: storing the normalized expression matrix is almost never necessary and is hugely expensive (in space and time) for large-scale data.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.

Tutorial 2: Using metadata_ondisc_matrix and multimodal_ondisc_matrix

The metadata_ondisc_matrix class

The multimodal_ondisc_matrix class

Querying basic information

Subsetting

Notes and tips

Tutorial 2: Using `metadata_ondisc_matrix` and `multimodal_ondisc_matrix`

The `metadata_ondisc_matrix` class

The `multimodal_ondisc_matrix` class