biwhitening ebnmf on GTEX data
DongyueXie
2022-10-19
Last updated: 2023-03-20
Checks: 7 0
Knit directory: SMF/
This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.
The command set.seed(20190719) was run prior to running
the code in the R Markdown file. Setting a seed ensures that any results
that rely on randomness, e.g. subsampling or permutations, are
reproducible.
Great job! Recording the operating system, R version, and package versions is critical for reproducibility.
Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.
Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version 8b68c63. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for
the analysis have been committed to Git prior to generating the results
(you can use wflow_publish or
wflow_git_commit). workflowr only checks the R Markdown
file, but you know if there are other scripts or data files that it
depends on. Below is the status of the Git repository when the results
were generated:
Ignored files:
Ignored: .Rhistory
Ignored: .Rproj.user/
Untracked files:
Untracked: data/count_files.RData
Untracked: data/lowgene/
Untracked: data/stmGTEx/
Untracked: output/luis/
Unstaged changes:
Deleted: example.sbatch
Deleted: slurm-65782790.out
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the repository in which changes were
made to the R Markdown
(analysis/biwhitening_ebnmf_gtex.Rmd) and HTML
(docs/biwhitening_ebnmf_gtex.html) files. If you’ve
configured a remote Git repository (see ?wflow_git_remote),
click on the hyperlinks in the table below to view the files as they
were in that past version.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| Rmd | 8b68c63 | DongyueXie | 2023-03-20 | wflow_publish("analysis/biwhitening_ebnmf_gtex.Rmd") |
| html | c2969aa | DongyueXie | 2023-03-19 | Build site. |
| Rmd | e6de2d3 | DongyueXie | 2023-03-19 | wflow_publish("analysis/biwhitening_ebnmf_gtex.Rmd") |
Introduction
library(fastTopics)
# fit the model on
fit = readRDS('/project2/mstephens/dongyue/gtex/V8/analysis/biwhite_ebnmf_fit.rds')
datax = readRDS('/project2/mstephens/dongyue/gtex/V8/data/gtex_v8.rds')
sample_info_tissue = datax$samplesfit_list <- list(L = fit$ldf$l[,-1]%*%diag(fit$ldf$d[-1]),F = fit$ldf$f[,-1])
class(fit_list) <- c("multinom_topic_model_fit", "list")
colors = randomcoloR::distinctColorPalette(30)
structure_plot(fit_list,grouping = sample_info_tissue$SMTS,colors = colors,gap=20,verbose=0)Running tsne on 135 x 29 matrix.Running tsne on 27 x 29 matrix.Running tsne on 110 x 29 matrix.Running tsne on 156 x 29 matrix.Running tsne on 313 x 29 matrix.Running tsne on 51 x 29 matrix.Running tsne on 87 x 29 matrix.Running tsne on 160 x 29 matrix.Running tsne on 90 x 29 matrix.Running tsne on 37 x 29 matrix.Running tsne on 62 x 29 matrix.Running tsne on 91 x 29 matrix.Running tsne on 78 x 29 matrix.Running tsne on 24 x 29 matrix.Running tsne on 36 x 29 matrix.Running tsne on 38 x 29 matrix.Running tsne on 30 x 29 matrix.Running tsne on 21 x 29 matrix.Running tsne on 205 x 29 matrix.Running tsne on 27 x 29 matrix.Running tsne on 37 x 29 matrix.Running tsne on 38 x 29 matrix.Running tsne on 85 x 29 matrix.
| Version | Author | Date |
|---|---|---|
| c2969aa | DongyueXie | 2023-03-19 |
fit_tm = readRDS('/project2/mstephens/dongyue/gtex/V8/analysis/topic_model_fit.rds')
structure_plot(fit_tm,grouping = sample_info_tissue$SMTS,colors = colors,gap=20,verbose=0)Running tsne on 144 x 20 matrix.Running tsne on 33 x 20 matrix.Running tsne on 104 x 20 matrix.Running tsne on 153 x 20 matrix.Running tsne on 302 x 20 matrix.Running tsne on 49 x 20 matrix.Running tsne on 85 x 20 matrix.Running tsne on 160 x 20 matrix.Running tsne on 91 x 20 matrix.Running tsne on 34 x 20 matrix.Running tsne on 63 x 20 matrix.Running tsne on 93 x 20 matrix.Running tsne on 68 x 20 matrix.Running tsne on 28 x 20 matrix.Running tsne on 38 x 20 matrix.Running tsne on 32 x 20 matrix.Running tsne on 35 x 20 matrix.Running tsne on 231 x 20 matrix.Running tsne on 20 x 20 matrix.
Running tsne on 20 x 20 matrix.Running tsne on 45 x 20 matrix.Running tsne on 39 x 20 matrix.Running tsne on 67 x 20 matrix.
| Version | Author | Date |
|---|---|---|
| c2969aa | DongyueXie | 2023-03-19 |
structure_plot(fit_list,grouping = sample_info_tissue$SMTSD,colors = colors,gap=20,verbose=0)Running tsne on 79 x 29 matrix.Running tsne on 56 x 29 matrix.Running tsne on 38 x 29 matrix.Running tsne on 51 x 29 matrix.Running tsne on 25 x 29 matrix.Running tsne on 79 x 29 matrix.Running tsne on 23 x 29 matrix.Running tsne on 28 x 29 matrix.Running tsne on 22 x 29 matrix.Running tsne on 26 x 29 matrix.Running tsne on 22 x 29 matrix.Running tsne on 24 x 29 matrix.Running tsne on 32 x 29 matrix.Running tsne on 25 x 29 matrix.Running tsne on 50 x 29 matrix.Running tsne on 65 x 29 matrix.Running tsne on 21 x 29 matrix.Running tsne on 43 x 29 matrix.Running tsne on 52 x 29 matrix.Running tsne on 41 x 29 matrix.Running tsne on 66 x 29 matrix.Running tsne on 58 x 29 matrix.Running tsne on 63 x 29 matrix.Running tsne on 53 x 29 matrix.Running tsne on 27 x 29 matrix.Running tsne on 61 x 29 matrix.Running tsne on 92 x 29 matrix.Running tsne on 64 x 29 matrix.Running tsne on 25 x 29 matrix.Running tsne on 38 x 29 matrix.Running tsne on 32 x 29 matrix.Running tsne on 31 x 29 matrix.Running tsne on 62 x 29 matrix.Running tsne on 69 x 29 matrix.Running tsne on 21 x 29 matrix.Running tsne on 36 x 29 matrix.Running tsne on 34 x 29 matrix.Running tsne on 39 x 29 matrix.Running tsne on 71 x 29 matrix.Running tsne on 24 x 29 matrix.Running tsne on 94 x 29 matrix.
structure_plot(fit_tm,grouping = sample_info_tissue$SMTSD,colors = colors,gap=20,verbose=0)Running tsne on 81 x 20 matrix.Running tsne on 64 x 20 matrix.Running tsne on 28 x 20 matrix.Running tsne on 51 x 20 matrix.Running tsne on 31 x 20 matrix.Running tsne on 71 x 20 matrix.Running tsne on 21 x 20 matrix.Running tsne on 29 x 20 matrix.Running tsne on 30 x 20 matrix.Running tsne on 28 x 20 matrix.Running tsne on 26 x 20 matrix.Running tsne on 25 x 20 matrix.
Running tsne on 25 x 20 matrix.Running tsne on 33 x 20 matrix.Running tsne on 26 x 20 matrix.Running tsne on 42 x 20 matrix.Running tsne on 47 x 20 matrix.Running tsne on 48 x 20 matrix.Running tsne on 41 x 20 matrix.Running tsne on 45 x 20 matrix.Running tsne on 61 x 20 matrix.Running tsne on 70 x 20 matrix.Running tsne on 37 x 20 matrix.Running tsne on 53 x 20 matrix.Running tsne on 30 x 20 matrix.Running tsne on 67 x 20 matrix.Running tsne on 24 x 20 matrix.Running tsne on 97 x 20 matrix.Running tsne on 71 x 20 matrix.Running tsne on 29 x 20 matrix.Running tsne on 34 x 20 matrix.Running tsne on 21 x 20 matrix.Running tsne on 89 x 20 matrix.Running tsne on 83 x 20 matrix.Running tsne on 24 x 20 matrix.Running tsne on 36 x 20 matrix.Running tsne on 42 x 20 matrix.Running tsne on 40 x 20 matrix.Running tsne on 79 x 20 matrix.Running tsne on 86 x 20 matrix.
source("~/Rpackages/gsmash/code/poisson_STM/structure_plot.R")structure plot. An error says: Error in (function (cond) : error in evaluating the argument ‘x’ in selecting a method for function ‘drop’: Expecting a single value: [extent=1200].
table(sample_info_tissue$SMTS)
library(randomcoloR)
colors = randomcoloR::distinctColorPalette(30)
structure_plot_general(fit$fit$EL,fit$fit$EF,as.factor(sample_info_tissue$SMTS),remove_l0f0 = FALSE,colors = colors,n_samples = 5000)Brain
brain = datax$counts[sample_info_tissue$SMTS=='Brain',]
brain = brain[,colSums(brain) > 0]
fit_brain = fastTopics::fit_topic_model(brain,k=6)
saveRDS(fit_brain,file='/project2/mstephens/dongyue/gtex/V8/analysis/topic_model_brain.rds')
Y_tilde = biwhitening(brain)
fit_sf = scaledflash(Y_tilde$Y,Y_tilde$u,Y_tilde$v,
S2 = NULL,
var.type = 'by_column',
Kmax=10,
tol=0.01,
maxiter = 1000,
ebnm_fn = 'ebnm_pe',
init_fn = 'nnmf_r1',
ebnm_param=NULL,
verbose=TRUE,
nullcheck=TRUE,
sigma2 = NULL,
seed=12345)
saveRDS(fit_sf,file='/project2/mstephens/dongyue/gtex/V8/analysis/biwhite_ebnmf_brain.rds')fit_sf = readRDS('/project2/mstephens/dongyue/gtex/V8/analysis/biwhite_ebnmf_brain.rds')
fit_list <- list(L = fit_sf$ldf$l[,-1]%*%diag(fit_sf$ldf$d[-1]),F = fit_sf$ldf$f[,-1])
class(fit_list) <- c("multinom_topic_model_fit", "list")
structure_plot(fit_list,grouping = sample_info_tissue$SMTSD[sample_info_tissue$SMTS=='Brain'],colors = c('#a6cee3',
'#1f78b4',
'#b2df8a',
'#33a02c',
'#fb9a99',
'#e31a1c',
'#fdbf6f',
'#ff7f00',
'#cab2d6',
'#6a3d9a',
'#ffff99',
'#b15928'),gap=40,verbose=0)Running tsne on 120 x 9 matrix.
Running tsne on 120 x 9 matrix.Running tsne on 186 x 9 matrix.Running tsne on 171 x 9 matrix.Running tsne on 181 x 9 matrix.Running tsne on 193 x 9 matrix.Running tsne on 161 x 9 matrix.Running tsne on 150 x 9 matrix.Running tsne on 154 x 9 matrix.Running tsne on 188 x 9 matrix.Running tsne on 146 x 9 matrix.Running tsne on 116 x 9 matrix.Running tsne on 114 x 9 matrix.
fit_topic = readRDS('/project2/mstephens/dongyue/gtex/V8/analysis/topic_model_brain.rds')
structure_plot(fit_topic,grouping=sample_info_tissue$SMTSD[sample_info_tissue$SMTS=='Brain'],verbose=0)Running tsne on 112 x 6 matrix.Running tsne on 141 x 6 matrix.Running tsne on 191 x 6 matrix.Running tsne on 171 x 6 matrix.Running tsne on 183 x 6 matrix.Running tsne on 202 x 6 matrix.Running tsne on 154 x 6 matrix.Running tsne on 141 x 6 matrix.Running tsne on 151 x 6 matrix.Running tsne on 187 x 6 matrix.Running tsne on 154 x 6 matrix.Running tsne on 115 x 6 matrix.Running tsne on 98 x 6 matrix.
sessionInfo()R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)
Matrix products: default
BLAS: /software/R-4.1.0-no-openblas-el7-x86_64/lib64/R/lib/libRblas.so
LAPACK: /software/R-4.1.0-no-openblas-el7-x86_64/lib64/R/lib/libRlapack.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=C
[4] LC_COLLATE=C LC_MONETARY=C LC_MESSAGES=C
[7] LC_PAPER=C LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=C LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] gridExtra_2.3 ggplot2_3.4.1 fastTopics_0.6-142 workflowr_1.6.2
loaded via a namespace (and not attached):
[1] mcmc_0.9-7 fs_1.5.0 progress_1.2.2
[4] httr_1.4.5 rprojroot_2.0.2 tools_4.1.0
[7] bslib_0.2.5.1 utf8_1.2.3 R6_2.5.1
[10] irlba_2.3.5.1 uwot_0.1.14 lazyeval_0.2.2
[13] colorspace_2.1-0 withr_2.5.0 tidyselect_1.2.0
[16] prettyunits_1.1.1 curl_5.0.0 compiler_4.1.0
[19] git2r_0.28.0 cli_3.6.0 quantreg_5.94
[22] SparseM_1.81 plotly_4.10.1 labeling_0.4.2
[25] sass_0.4.0 scales_1.2.1 SQUAREM_2021.1
[28] quadprog_1.5-8 pbapply_1.7-0 mixsqp_0.3-48
[31] stringr_1.5.0 digest_0.6.31 rmarkdown_2.9
[34] MCMCpack_1.6-3 pkgconfig_2.0.3 htmltools_0.5.4
[37] fastmap_1.1.0 invgamma_1.1 highr_0.9
[40] htmlwidgets_1.6.1 rlang_1.0.6 rstudioapi_0.13
[43] jquerylib_0.1.4 generics_0.1.3 farver_2.1.1
[46] jsonlite_1.8.4 dplyr_1.1.0 magrittr_2.0.3
[49] Matrix_1.5-3 Rcpp_1.0.10 munsell_0.5.0
[52] fansi_1.0.4 lifecycle_1.0.3 stringi_1.6.2
[55] whisker_0.4 yaml_2.3.7 MASS_7.3-54
[58] Rtsne_0.16 grid_4.1.0 parallel_4.1.0
[61] randomcoloR_1.1.0.1 promises_1.2.0.1 ggrepel_0.9.3
[64] crayon_1.5.2 lattice_0.20-44 cowplot_1.1.1
[67] splines_4.1.0 hms_1.1.2 knitr_1.33
[70] pillar_1.8.1 glue_1.6.2 evaluate_0.14
[73] V8_4.2.2 data.table_1.14.8 RcppParallel_5.1.7
[76] vctrs_0.5.2 httpuv_1.6.1 MatrixModels_0.5-1
[79] gtable_0.3.1 purrr_1.0.1 tidyr_1.3.0
[82] ashr_2.2-54 xfun_0.24 coda_0.19-4
[85] later_1.3.0 survival_3.2-11 viridisLite_0.4.1
[88] truncnorm_1.0-8 tibble_3.1.8 cluster_2.1.2
[91] ellipsis_0.3.2