Last updated: 2024-01-01

Checks: 2 0

Knit directory: mage_2020_marker-gene-benchmarking/

This reproducible R Markdown analysis was created with workflowr (version 1.7.0). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

R Markdown file: up-to-date

Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Repository version: 2c5ed16

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version 2c5ed16. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated: 


Ignored files:
    Ignored:    .Renviron
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/
    Ignored:    .snakemake/
    Ignored:    NSForest/.Rhistory
    Ignored:    NSForest/NS-Forest_v3_Extended_Binary_Markers_Supplmental.csv
    Ignored:    NSForest/NS-Forest_v3_Full_Results.csv
    Ignored:    NSForest/NSForest3_medianValues.csv
    Ignored:    NSForest/NSForest_v3_Final_Result.csv
    Ignored:    NSForest/__pycache__/
    Ignored:    NSForest/data/
    Ignored:    RankCorr/picturedRocks/__pycache__/
    Ignored:    benchmarks/
    Ignored:    config/
    Ignored:    data/cellmarker/
    Ignored:    data/downloaded_data/
    Ignored:    data/expert_annotations/
    Ignored:    data/expert_mgs/
    Ignored:    data/raw_data/
    Ignored:    data/real_data/
    Ignored:    data/sim_data/
    Ignored:    data/sim_mgs/
    Ignored:    data/special_real_data/
    Ignored:    figures/
    Ignored:    logs/
    Ignored:    results/
    Ignored:    weights/

Unstaged changes:
    Modified:   smash-fork

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

These are the previous versions of the repository in which changes were made to the R Markdown (analysis/index.Rmd) and HTML (public/index.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.

File Version Author Date Message
Rmd 2c5ed16 Jeffrey Pullin 2024-01-01 Tweak website
html c7df94e Jeffrey Pullin 2024-01-01 Build site.
html fcecf65 Jeffrey Pullin 2022-09-09 Build site.
html af96b34 Jeffrey Pullin 2022-08-30 Build site.
html 72ff781 Jeffrey Pullin 2022-05-11 Build site.
Rmd f0da1ad Jeffrey Pullin 2022-05-11 Update landing page
html 0e47874 Jeffrey Pullin 2022-05-04 Build site.
html b5045c1 Jeffrey Pullin 2022-05-02 Build site.
Rmd 048156f Jeffrey Pullin 2022-05-02 Tweak analysis outputs
html 048156f Jeffrey Pullin 2022-05-02 Tweak analysis outputs
html 8b989e1 Jeffrey Pullin 2022-05-02 Build site.
html 0548273 Jeffrey Pullin 2022-05-02 Build site.
Rmd 50bca7c Jeffrey Pullin 2022-05-02 workflowr::wflow_publish(all = TRUE, republish = TRUE)
html 50bca7c Jeffrey Pullin 2022-05-02 workflowr::wflow_publish(all = TRUE, republish = TRUE)
html 5cc008f Jeffrey Pullin 2022-02-09 Build site.
Rmd d1aca16 Jeffrey Pullin 2022-02-09 Refresh website
html d1aca16 Jeffrey Pullin 2022-02-09 Refresh website
html a116967 Jeffrey Pullin 2021-04-13 Build site.
Rmd 581627b Jeffrey Pullin 2021-04-13 Rebuild index
html ca82ce0 Jeffrey Pullin 2021-02-16 Build site.
Rmd 63bce66 Jeffrey Pullin 2021-02-16 Add new marker gene type analysis and update FDR + TPR
html 2863555 Jeffrey Pullin 2021-02-10 Build site.
Rmd 1ad9d6d Jeffrey Pullin 2021-02-10 Add workflowr website
Rmd 7971ea2 Jeffrey Pullin 2020-10-25 Initial commit

Project overview

This project aim to benchmark current methods for the selection of cell-type maker genes in scRNA-seq data. Our findings are summarized in the following preprint:

A comparison of marker gene selection methods for single-cell RNA sequencing data
Jeffrey M Pullin, Davis J McCarthy
bioRxiv 2022.05.09.490241; doi: https://doi.org/10.1101/2022.05.09.490241

Analyses

We assessed the method’s concordance, the characteristics of the marker genes they select, and the magnitude and surprising properties of the p-values they return.

Next, we assessed the method’s performance on simulated data and their predictive performance on real datasets.

We compared the method’s ability to select expert annotated marker genes on a vareity of datasets:

We assessed the impact of various dataset characteritics on methods performance and method’s stability when an exemplar real dataset was down-sampled.

We assessed the methods’ speed and memory usage as well as the quality of their implementations.

Finally, we compared the commonly used Scanpy and Seurat packages, examining the impact of their different methods for calculating log fold-change.

Datasets

A variety of scRNA-seq datasets are used in the used to compare methods. For theses datasets we have performed general analysis, including selecting marker genes. The code used to process the each dataset can be found in the corresponding prep_* R script in the code directory. A selection of datasets (below) were partially analysed to inform aspects of the benchmarking.