tima::run_tima()
#> + par_def_pre_fea_tab dispatched
#> ✔ par_def_pre_fea_tab completed [7ms, 860 B]
#> + par_def_pre_tax dispatched
#> ✔ par_def_pre_tax completed [1ms, 1.51 kB]
#> + par_def_fil_ann dispatched
#> ✔ par_def_fil_ann completed [0ms, 1.34 kB]
#> + par_def_pre_lib_spe dispatched
#> ✔ par_def_pre_lib_spe completed [1ms, 1.57 kB]
#> + par_def_pre_lib_rt dispatched
#> ✔ par_def_pre_lib_rt completed [1ms, 2.20 kB]
#> + par_def_pre_ann_gnp dispatched
#> ✔ par_def_pre_ann_gnp completed [1ms, 1.42 kB]
#> + yaml_paths dispatched
#> ✔ yaml_paths completed [1ms, 16.25 kB]
#> + par_def_pre_lib_sop_clo dispatched
#> ✔ par_def_pre_lib_sop_clo completed [1ms, 523 B]
#> + par_def_pre_ann_spe dispatched
#> ✔ par_def_pre_ann_spe completed [1ms, 1.46 kB]
#> + par_def_ann_spe dispatched
#> ✔ par_def_ann_spe completed [1ms, 2.14 kB]
#> + par_def_pre_lib_sop_big dispatched
#> ✔ par_def_pre_lib_sop_big completed [0ms, 314 B]
#> + par_def_pre_fea_edg dispatched
#> ✔ par_def_pre_fea_edg completed [0ms, 706 B]
#> + par_def_pre_ann_mzm dispatched
#> ✔ par_def_pre_ann_mzm completed [0ms, 1.43 kB]
#> + par_def_wei_ann dispatched
#> ✔ par_def_wei_ann completed [1ms, 5.34 kB]
#> + par_def_pre_ann_sir dispatched
#> ✔ par_def_pre_ann_sir completed [1ms, 2.08 kB]
#> + par_def_pre_lib_sop_lot dispatched
#> ✔ par_def_pre_lib_sop_lot completed [1ms, 494 B]
#> + par_def_cre_com dispatched
#> ✔ par_def_cre_com completed [1ms, 375 B]
#> + par_def_pre_lib_sop_hmd dispatched
#> ✔ par_def_pre_lib_sop_hmd completed [1ms, 492 B]
#> + par_def_cre_edg_spe dispatched
#> ✔ par_def_cre_edg_spe completed [1ms, 1.42 kB]
#> + par_def_ann_mas dispatched
#> ✔ par_def_ann_mas completed [0ms, 6.09 kB]
#> + par_def_pre_lib_sop_ecm dispatched
#> ✔ par_def_pre_lib_sop_ecm completed [0ms, 492 B]
#> + par_def_pre_fea_com dispatched
#> ✔ par_def_pre_fea_com completed [1ms, 358 B]
#> + par_def_pre_lib_sop_mer dispatched
#> ✔ par_def_pre_lib_sop_mer completed [0ms, 5.91 kB]
#> + paths dispatched
#> ✔ paths completed [1ms, 3.04 kB]
#> + lib_sop_ecm dispatched
#> [2026-03-26 21:22:00.823] [INFO ] > Starting: download_file [url=https://ecmdb.ca/download/ecmdb.json.zip, destination=data/source/libraries/sop/ecmdb.json.zip]
#> [2026-03-26 21:22:01.177] [INFO ] [OK] Completed: download_file [size_bytes=1334921] (320ms)
#> ✔ lib_sop_ecm completed [388ms, 1.33 MB]
#> + xrefs_compounds dispatched
#> [2026-03-26 21:22:01.281] [INFO ] Fetching compound cross-references from Wikidata / QLever
#> [2026-03-26 21:22:01.282] [INFO ] > Starting: get_compounds_xrefs [(no parameters)]
#> [2026-03-26 21:22:28.957] [INFO ] > Starting: export_output [file=data/interim/xrefs/compounds.tsv.gz, n_rows=4646335]
#> [2026-03-26 21:22:32.818] [INFO ] [OK] Completed: export_output [size_bytes=44877698] (3.9s)
#> [2026-03-26 21:22:32.819] [INFO ] [OK] Completed: get_compounds_xrefs [n_rows=4646335] (31.5s)
#> ✔ xrefs_compounds completed [31.5s, 44.88 MB]
#> + lib_spe_exp_mb_pre_pos dispatched
#> [2026-03-26 21:22:33.845] [INFO ] > Starting: download_file [url=https://github.com/adafede/SpectRalLibRaRies/raw/main/data/interim/libraries/spectra/exp/massbank_202510_pos.rds, destination=data/interim/libraries/spectra/exp/massbank_202510_pos.rds]
#> [2026-03-26 21:22:34.485] [INFO ] [OK] Completed: download_file [size_bytes=17589271] (640ms)
#> ✔ lib_spe_exp_mb_pre_pos completed [642ms, 17.59 MB]
#> + par_pre_par dispatched
#> ✔ par_pre_par completed [0ms, 1.54 kB]
#> + lib_spe_exp_gnp_pre_sop dispatched
#> [2026-03-26 21:22:35.365] [INFO ] > Starting: download_file [url=https://github.com/Adafede/SpectRalLibRaRies/raw/main/data/interim/libraries/sop/gnps_11566051_prepared.tsv.gz, destination=data/interim/libraries/sop/gnps_11566051_prepared.tsv.gz]
#> [2026-03-26 21:22:35.608] [INFO ] [OK] Completed: download_file [size_bytes=1311218] (243ms)
#> ✔ lib_spe_exp_gnp_pre_sop completed [245ms, 1.31 MB]
#> + lib_spe_exp_gnp_pre_neg dispatched
#> [2026-03-26 21:22:35.733] [INFO ] > Starting: download_file [url=https://github.com/adafede/SpectRalLibRaRies/raw/main/data/interim/libraries/spectra/exp/gnps_11566051_neg.rds, destination=data/interim/libraries/spectra/exp/gnps_11566051_neg.rds]
#> Downloading 97% ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0s
#> Downloading 100% ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0s
#> [2026-03-26 21:22:38.220] [INFO ] [OK] Completed: download_file [size_bytes=91762056] (2.5s)
#> ✔ lib_spe_exp_gnp_pre_neg completed [2.5s, 91.76 MB]
#> + lib_spe_exp_mb_pre_neg dispatched
#> [2026-03-26 21:22:38.408] [INFO ] > Starting: download_file [url=https://github.com/adafede/SpectRalLibRaRies/raw/main/data/interim/libraries/spectra/exp/massbank_202510_neg.rds, destination=data/interim/libraries/spectra/exp/massbank_202510_neg.rds]
#> [2026-03-26 21:22:39.152] [INFO ] [OK] Completed: download_file [size_bytes=5982479] (744ms)
#> ✔ lib_spe_exp_mb_pre_neg completed [746ms, 5.98 MB]
#> + lib_spe_exp_mer_pre_neg dispatched
#> [2026-03-26 21:22:39.296] [INFO ] > Starting: download_file [url=https://github.com/adafede/SpectRalLibRaRies/raw/main/data/interim/libraries/spectra/exp/merlin_16984129_neg.rds, destination=data/interim/libraries/spectra/exp/merlin_16984129_neg.rds]
#> [2026-03-26 21:22:40.326] [INFO ] [OK] Completed: download_file [size_bytes=53015463] (1s)
#> ✔ lib_spe_exp_mer_pre_neg completed [1s, 53.02 MB]
#> + lib_spe_is_wik_pre_neg dispatched
#> [2026-03-26 21:22:40.483] [INFO ] > Starting: download_file [url=https://github.com/taxonomicallyinformedannotation/tima-isdb-neg/raw/main/wikidata_5607185_neg.rds, destination=data/interim/libraries/spectra/is/wikidata_5607185_neg.rds]
#> Downloading 9% ■■■■ 10s
#> Downloading 17% ■■■■■■ 9s
#> Downloading 46% ■■■■■■■■■■■■■■■ 6s
#> Downloading 71% ■■■■■■■■■■■■■■■■■■■■■■ 3s
#> Downloading 100% ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0s
#> [2026-03-26 21:22:51.547] [INFO ] [OK] Completed: download_file [size_bytes=687328159] (11.1s)
#> ✔ lib_spe_is_wik_pre_neg completed [11.1s, 687.33 MB]
#> + lib_spe_is_wik_pre_pos dispatched
#> [2026-03-26 21:22:51.935] [INFO ] > Starting: download_file [url=https://github.com/taxonomicallyinformedannotation/tima-isdb-pos/raw/main/wikidata_5607185_pos.rds, destination=data/interim/libraries/spectra/is/wikidata_5607185_pos.rds]
#> Downloading 8% ■■■ 11s
#> Downloading 16% ■■■■■■ 12s
#> Downloading 29% ■■■■■■■■■■ 13s
#> Downloading 43% ■■■■■■■■■■■■■■ 11s
#> Downloading 55% ■■■■■■■■■■■■■■■■■ 9s
#> Downloading 64% ■■■■■■■■■■■■■■■■■■■■ 8s
#> Downloading 80% ■■■■■■■■■■■■■■■■■■■■■■■■■ 4s
#> Downloading 98% ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0s
#> Downloading 100% ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0s
#> [2026-03-26 21:23:12.824] [INFO ] [OK] Completed: download_file [size_bytes=863950396] (20.9s)
#> ✔ lib_spe_is_wik_pre_pos completed [20.9s, 863.95 MB]
#> + test_spectra_mini dispatched
#> ✔ test_spectra_mini completed [0ms, 7.77 MB]
#> + lib_sop_hmd_fam_raw dispatched
#> [2026-03-26 21:23:13.426] [INFO ] > Starting: download_file [url=https://www.csfmetabolome.ca/system/downloads/current/csf_metabolites_structures.zip, destination=data/source/libraries/sop/csfmetabolome/structures.zip]
#> [2026-03-26 21:23:13.535] [WARN ] file download failed (attempt 1/3), retrying in 1s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.csfmetabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:14.611] [WARN ] file download failed (attempt 2/3), retrying in 2s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.csfmetabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:16.736] [WARN ] HMDB download failed. Creating minimal placeholder SDF file.
#> [2026-03-26 21:23:16.791] [INFO ] > Starting: download_file [url=https://www.fecalmetabolome.ca/system/downloads/current/feces_metabolites_structures.zip, destination=data/source/libraries/sop/fecalmetabolome/structures.zip]
#> [2026-03-26 21:23:16.908] [WARN ] file download failed (attempt 1/3), retrying in 1s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.fecalmetabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:17.984] [WARN ] file download failed (attempt 2/3), retrying in 2s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.fecalmetabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:20.123] [WARN ] HMDB download failed. Creating minimal placeholder SDF file.
#> [2026-03-26 21:23:20.128] [INFO ] > Starting: download_file [url=https://www.salivametabolome.ca/system/downloads/current/saliva_metabolites_structures.zip, destination=data/source/libraries/sop/salivametabolome/structures.zip]
#> [2026-03-26 21:23:20.222] [WARN ] file download failed (attempt 1/3), retrying in 1s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.salivametabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:21.298] [WARN ] file download failed (attempt 2/3), retrying in 2s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.salivametabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:23.411] [WARN ] HMDB download failed. Creating minimal placeholder SDF file.
#> [2026-03-26 21:23:23.415] [INFO ] > Starting: download_file [url=https://www.serummetabolome.ca/system/downloads/current/serum_metabolites_structures.zip, destination=data/source/libraries/sop/serummetabolome/structures.zip]
#> [2026-03-26 21:23:23.541] [WARN ] file download failed (attempt 1/3), retrying in 1s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.serummetabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:24.616] [WARN ] file download failed (attempt 2/3), retrying in 2s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.serummetabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:26.728] [WARN ] HMDB download failed. Creating minimal placeholder SDF file.
#> [2026-03-26 21:23:26.733] [INFO ] > Starting: download_file [url=https://www.sweatmetabolome.ca/system/downloads/current/sweat_metabolites_structures.zip, destination=data/source/libraries/sop/sweatmetabolome/structures.zip]
#> [2026-03-26 21:23:26.815] [WARN ] file download failed (attempt 1/3), retrying in 1s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.sweatmetabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:27.893] [WARN ] file download failed (attempt 2/3), retrying in 2s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.sweatmetabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:30.032] [WARN ] HMDB download failed. Creating minimal placeholder SDF file.
#> [2026-03-26 21:23:30.036] [INFO ] > Starting: download_file [url=https://www.urinemetabolome.ca/system/downloads/current/urine_metabolites_structures.zip, destination=data/source/libraries/sop/urinemetabolome/structures.zip]
#> [2026-03-26 21:23:30.139] [WARN ] file download failed (attempt 1/3), retrying in 1s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.urinemetabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:31.213] [WARN ] file download failed (attempt 2/3), retrying in 2s: Failed to perform HTTP request.
#> Caused by error in `curl::curl_fetch_disk()`:
#> ! SSL peer certificate or SSH remote key was not OK [www.urinemetabolome.ca]:
#> SSL certificate problem: certificate has expired
#> [2026-03-26 21:23:33.324] [WARN ] HMDB download failed. Creating minimal placeholder SDF file.
#> [2026-03-26 21:23:33.328] [INFO ] > Starting: download_file [url=https://mcdb.ca/system/downloads/current/milk_metabolites_structures.zip, destination=data/source/libraries/sop/mcdb/structures.zip]
#> [2026-03-26 21:23:33.554] [INFO ] [OK] Completed: download_file [size_bytes=1375818] (225ms)
#> [2026-03-26 21:23:33.555] [INFO ] > Starting: download_file [url=https://smpdb.ca/downloads/smpdb_structures.zip, destination=data/source/libraries/sop/smpdb/structures.zip]
#> [2026-03-26 21:23:34.193] [INFO ] [OK] Completed: download_file [size_bytes=23382536] (638ms)
#> [2026-03-26 21:23:34.195] [INFO ] > Starting: download_file [url=https://mimedb.org/system/downloads/2.0/mimedb.sdf.zip, destination=data/source/libraries/sop/mimedb/structures.zip]
#> [2026-03-26 21:23:34.470] [WARN ] file download failed (attempt 1/3), retrying in 1s: HTTP 403 Forbidden.
#> [2026-03-26 21:23:35.510] [WARN ] file download failed (attempt 2/3), retrying in 2s: HTTP 403 Forbidden.
#> [2026-03-26 21:23:37.588] [WARN ] HMDB download failed. Creating minimal placeholder SDF file.
#> [2026-03-26 21:23:37.592] [INFO ] > Starting: download_file [url=https://t3db.ca/system/downloads/current/structures.zip, destination=data/source/libraries/sop/t3db/structures.zip]
#> [2026-03-26 21:23:37.793] [INFO ] [OK] Completed: download_file [size_bytes=1676995] (201ms)
#> [2026-03-26 21:23:37.794] [INFO ] > Starting: download_file [url=https://bovinedb.ca/system/downloads/current/structures.zip, destination=data/source/libraries/sop/bovinedb/structures.zip]
#> [2026-03-26 21:23:38.349] [INFO ] [OK] Completed: download_file [size_bytes=19260214] (555ms)
#> [2026-03-26 21:23:38.351] [INFO ] > Starting: download_file [url=https://www.ymdb.ca/system/downloads/current/ymdb.sdf.zip, destination=data/source/libraries/sop/ymdb/structures.zip]
#> [2026-03-26 21:23:38.450] [INFO ] [OK] Completed: download_file [size_bytes=1200611] (100ms)
#> [2026-03-26 21:23:38.452] [INFO ] > Starting: download_file [url=https://cannabisdatabase.ca/simple/download_compound_as_sdf, destination=data/source/libraries/sop/cannabisdatabase/compounds.sdf]
#> [2026-03-26 21:23:39.811] [INFO ] [OK] Completed: download_file [size_bytes=42496276] (1.4s)
#> ✔ lib_sop_hmd_fam_raw completed [26.4s, 89.39 MB]
#> + lib_spe_exp_mb_pre_sop dispatched
#> [2026-03-26 21:23:39.983] [INFO ] > Starting: download_file [url=https://github.com/Adafede/SpectRalLibRaRies/raw/main/data/interim/libraries/sop/massbank_202510_prepared.tsv.gz, destination=data/interim/libraries/sop/massbank_202510_prepared.tsv.gz]
#> [2026-03-26 21:23:40.169] [INFO ] [OK] Completed: download_file [size_bytes=499913] (186ms)
#> ✔ lib_spe_exp_mb_pre_sop completed [188ms, 499.91 kB]
#> + lib_spe_exp_mer_pre_sop dispatched
#> [2026-03-26 21:23:40.304] [INFO ] > Starting: download_file [url=https://github.com/Adafede/SpectRalLibRaRies/raw/main/data/interim/libraries/sop/merlin_16984129_prepared.tsv.gz, destination=data/interim/libraries/sop/merlin_16984129_prepared.tsv.gz]
#> [2026-03-26 21:23:40.550] [INFO ] [OK] Completed: download_file [size_bytes=2338408] (246ms)
#> ✔ lib_spe_exp_mer_pre_sop completed [248ms, 2.34 MB]
#> + lib_spe_is_wik_pre_sop dispatched
#> [2026-03-26 21:23:40.680] [INFO ] > Starting: download_file [url=https://github.com/taxonomicallyinformedannotation/tima-example-files/raw/main/wikidata_spectral_5607185_prepared.tsv.gz, destination=data/interim/libraries/sop/wikidata_5607185_prepared.tsv.gz]
#> [2026-03-26 21:23:41.100] [INFO ] [OK] Completed: download_file [size_bytes=37904410] (420ms)
#> ✔ lib_spe_is_wik_pre_sop completed [422ms, 37.90 MB]
#> + lib_sop_hmd dispatched
#> [2026-03-26 21:23:41.245] [INFO ] > Starting: download_file [url=https://hmdb.ca/system/downloads/current/structures.zip, destination=data/source/libraries/sop/hmdb/structures.zip]
#> [2026-03-26 21:23:41.326] [WARN ] file download failed (attempt 1/3), retrying in 1s: HTTP 403 Forbidden.
#> [2026-03-26 21:23:42.369] [WARN ] file download failed (attempt 2/3), retrying in 2s: HTTP 403 Forbidden.
#> [2026-03-26 21:23:44.450] [WARN ] HMDB download failed. Creating minimal placeholder SDF file.
#> ✔ lib_sop_hmd completed [3.2s, 337 B]
#> + lib_spe_exp_gnp_pre_pos dispatched
#> [2026-03-26 21:23:44.586] [INFO ] > Starting: download_file [url=https://github.com/adafede/SpectRalLibRaRies/raw/main/data/interim/libraries/spectra/exp/gnps_11566051_pos.rds, destination=data/interim/libraries/spectra/exp/gnps_11566051_pos.rds]
#> Downloading 21% ■■■■■■■ 4s
#> Downloading 72% ■■■■■■■■■■■■■■■■■■■■■■■ 1s
#> Downloading 100% ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0s
#> [2026-03-26 21:23:49.983] [INFO ] [OK] Completed: download_file [size_bytes=341151138] (5.4s)
#> ✔ lib_spe_exp_gnp_pre_pos completed [5.4s, 341.15 MB]
#> + lib_sop_lot dispatched
#> [2026-03-26 21:23:50.248] [INFO ] Retrieving latest version from Zenodo: 10.5281/zenodo.5794106
#> [2026-03-26 21:23:51.809] [INFO ] Downloading 230106_frozen_metadata.csv.gz from https://doi.org/10.5281/zenodo.5794106
#> [2026-03-26 21:23:51.811] [INFO ] > Starting: download_file [url=https://zenodo.org/records/7534071/files/230106_frozen_metadata.csv.gz, destination=data/source/libraries/sop/lotus.csv.gz]
#> [2026-03-26 21:23:55.884] [INFO ] [OK] Completed: download_file [size_bytes=92979778] (4.1s)
#> [2026-03-26 21:23:55.885] [INFO ] Download completed: data/source/libraries/sop/lotus.csv.gz
#> ✔ lib_sop_lot completed [5.6s, 92.98 MB]
#> + par_pre_par2 dispatched
#> ✔ par_pre_par2 completed [0ms, 27.02 kB]
#> + lib_spe_exp_mer_pre_pos dispatched
#> [2026-03-26 21:23:56.209] [INFO ] > Starting: download_file [url=https://github.com/adafede/SpectRalLibRaRies/raw/main/data/interim/libraries/spectra/exp/merlin_16984129_pos.rds, destination=data/interim/libraries/spectra/exp/merlin_16984129_pos.rds]
#> Downloading 34% ■■■■■■■■■■■ 2s
#> Downloading 35% ■■■■■■■■■■■■ 2s
#> Downloading 100% ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0s
#> [2026-03-26 21:23:58.984] [INFO ] [OK] Completed: download_file [size_bytes=158504396] (2.8s)
#> ✔ lib_spe_exp_mer_pre_pos completed [2.8s, 158.50 MB]
#> + par_fin_par dispatched
#> ✔ par_fin_par completed [0ms, 332 B]
#> + lib_sop_hmd_fam_pre dispatched
#> [2026-03-26 21:23:59.320] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=CSFMETABOLOME, input=data/source/libraries/sop/csfmetabolome/structures.zip, tag=csf]
#> [2026-03-26 21:23:59.432] [WARN ] Empty dataframe in select_sop_columns
#> [2026-03-26 21:23:59.437] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/csfmetabolome_prepared.tsv.gz, n_rows=0]
#> [2026-03-26 21:23:59.439] [INFO ] [OK] Completed: export_output [size_bytes=255] (1ms)
#> [2026-03-26 21:23:59.440] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=0] (120ms)
#> [2026-03-26 21:23:59.440] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=FECALMETABOLOME, input=data/source/libraries/sop/fecalmetabolome/structures.zip, tag=fecal]
#> [2026-03-26 21:23:59.467] [WARN ] Empty dataframe in select_sop_columns
#> [2026-03-26 21:23:59.472] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/fecalmetabolome_prepared.tsv.gz, n_rows=0]
#> [2026-03-26 21:23:59.474] [INFO ] [OK] Completed: export_output [size_bytes=255] (1ms)
#> [2026-03-26 21:23:59.475] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=0] (34ms)
#> [2026-03-26 21:23:59.476] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=SALIVAMETABOLOME, input=data/source/libraries/sop/salivametabolome/structures.zip, tag=saliva]
#> [2026-03-26 21:23:59.503] [WARN ] Empty dataframe in select_sop_columns
#> [2026-03-26 21:23:59.508] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/salivametabolome_prepared.tsv.gz, n_rows=0]
#> [2026-03-26 21:23:59.509] [INFO ] [OK] Completed: export_output [size_bytes=255] (1ms)
#> [2026-03-26 21:23:59.510] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=0] (34ms)
#> [2026-03-26 21:23:59.511] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=SERUMMETABOLOME, input=data/source/libraries/sop/serummetabolome/structures.zip, tag=serum]
#> [2026-03-26 21:23:59.538] [WARN ] Empty dataframe in select_sop_columns
#> [2026-03-26 21:23:59.543] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/serummetabolome_prepared.tsv.gz, n_rows=0]
#> [2026-03-26 21:23:59.544] [INFO ] [OK] Completed: export_output [size_bytes=255] (1ms)
#> [2026-03-26 21:23:59.545] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=0] (34ms)
#> [2026-03-26 21:23:59.546] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=SWEATMETABOLOME, input=data/source/libraries/sop/sweatmetabolome/structures.zip, tag=sweat]
#> [2026-03-26 21:23:59.573] [WARN ] Empty dataframe in select_sop_columns
#> [2026-03-26 21:23:59.579] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/sweatmetabolome_prepared.tsv.gz, n_rows=0]
#> [2026-03-26 21:23:59.580] [INFO ] [OK] Completed: export_output [size_bytes=255] (1ms)
#> [2026-03-26 21:23:59.581] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=0] (34ms)
#> [2026-03-26 21:23:59.582] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=URINEMETABOLOME, input=data/source/libraries/sop/urinemetabolome/structures.zip, tag=urine]
#> [2026-03-26 21:23:59.609] [WARN ] Empty dataframe in select_sop_columns
#> [2026-03-26 21:23:59.614] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/urinemetabolome_prepared.tsv.gz, n_rows=0]
#> [2026-03-26 21:23:59.615] [INFO ] [OK] Completed: export_output [size_bytes=255] (1ms)
#> [2026-03-26 21:23:59.616] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=0] (34ms)
#> [2026-03-26 21:23:59.617] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=MCDB, input=data/source/libraries/sop/mcdb/structures.zip, tag=milk]
#> [2026-03-26 21:24:00.109] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/mcdb_prepared.tsv.gz, n_rows=2360]
#> [2026-03-26 21:24:00.123] [INFO ] [OK] Completed: export_output [size_bytes=93659] (14ms)
#> [2026-03-26 21:24:00.124] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=2360] (507ms)
#> [2026-03-26 21:24:00.125] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=SMPDB, input=data/source/libraries/sop/smpdb/structures.zip, tag=pathway]
#> [2026-03-26 21:24:13.591] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/smpdb_prepared.tsv.gz, n_rows=49817]
#> [2026-03-26 21:24:13.727] [INFO ] [OK] Completed: export_output [size_bytes=1586276] (135ms)
#> [2026-03-26 21:24:13.728] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=49817] (13.6s)
#> [2026-03-26 21:24:13.729] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=MIMEDB, input=data/source/libraries/sop/mimedb/structures.zip, tag=microbiome]
#> [2026-03-26 21:24:13.757] [WARN ] Empty dataframe in select_sop_columns
#> [2026-03-26 21:24:13.761] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/mimedb_prepared.tsv.gz, n_rows=0]
#> [2026-03-26 21:24:13.763] [INFO ] [OK] Completed: export_output [size_bytes=255] (1ms)
#> [2026-03-26 21:24:13.764] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=0] (34ms)
#> [2026-03-26 21:24:13.764] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=T3DB, input=data/source/libraries/sop/t3db/structures.zip, tag=toxin]
#> [2026-03-26 21:24:14.261] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/t3db_prepared.tsv.gz, n_rows=3526]
#> [2026-03-26 21:24:14.276] [INFO ] [OK] Completed: export_output [size_bytes=130540] (15ms)
#> [2026-03-26 21:24:14.277] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=3526] (513ms)
#> [2026-03-26 21:24:14.278] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=BOVINEDB, input=data/source/libraries/sop/bovinedb/structures.zip, tag=NA]
#> [2026-03-26 21:24:27.715] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/bovinedb_prepared.tsv.gz, n_rows=51684]
#> [2026-03-26 21:24:27.860] [INFO ] [OK] Completed: export_output [size_bytes=1728074] (145ms)
#> [2026-03-26 21:24:27.862] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=51684] (13.6s)
#> [2026-03-26 21:24:27.863] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=YMDB, input=data/source/libraries/sop/ymdb/structures.zip, tag=NA]
#> [2026-03-26 21:24:28.236] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/ymdb_prepared.tsv.gz, n_rows=2025]
#> [2026-03-26 21:24:28.249] [INFO ] [OK] Completed: export_output [size_bytes=90095] (12ms)
#> [2026-03-26 21:24:28.250] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=2025] (387ms)
#> [2026-03-26 21:24:28.251] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=CANNABISDATABASE, input=data/source/libraries/sop/cannabisdatabase/compounds.sdf, tag=NA]
#> [2026-03-26 21:24:29.189] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/cannabisdatabase_prepared.tsv.gz, n_rows=6220]
#> [2026-03-26 21:24:29.213] [INFO ] [OK] Completed: export_output [size_bytes=237379] (23ms)
#> [2026-03-26 21:24:29.214] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=6220] (963ms)
#> ✔ lib_sop_hmd_fam_pre completed [29.9s, 3.87 MB]
#> + par_fin_par2 dispatched
#> ✔ par_fin_par2 completed [2ms, 3.33 kB]
#> + par_usr_pre_fea_tab dispatched
#> ✔ par_usr_pre_fea_tab completed [1.8s, 274 B]
#> + par_usr_pre_tax dispatched
#> ✔ par_usr_pre_tax completed [1.6s, 438 B]
#> + par_usr_fil_ann dispatched
#> ✔ par_usr_fil_ann completed [1.7s, 739 B]
#> + par_usr_cre_edg_spe dispatched
#> ✔ par_usr_cre_edg_spe completed [1.7s, 452 B]
#> + par_usr_pre_ann_gnp dispatched
#> ✔ par_usr_pre_ann_gnp completed [1.7s, 708 B]
#> + par_usr_pre_lib_sop_clo dispatched
#> ✔ par_usr_pre_lib_sop_clo completed [1.7s, 267 B]
#> + par_usr_pre_ann_spe dispatched
#> ✔ par_usr_pre_ann_spe completed [1.7s, 731 B]
#> + par_usr_pre_lib_sop_big dispatched
#> ✔ par_usr_pre_lib_sop_big completed [1.7s, 107 B]
#> + par_usr_pre_ann_mzm dispatched
#> ✔ par_usr_pre_ann_mzm completed [1.7s, 710 B]
#> + par_usr_pre_lib_rt dispatched
#> ✔ par_usr_pre_lib_rt completed [1.9s, 487 B]
#> + par_usr_wei_ann dispatched
#> ✔ par_usr_wei_ann completed [1.7s, 1.80 kB]
#> + par_usr_pre_lib_sop_lot dispatched
#> ✔ par_usr_pre_lib_sop_lot completed [1.7s, 174 B]
#> + par_usr_pre_ann_sir dispatched
#> ✔ par_usr_pre_ann_sir completed [1.7s, 934 B]
#> + par_usr_cre_com dispatched
#> ✔ par_usr_cre_com completed [1.7s, 200 B]
#> + par_usr_pre_lib_spe dispatched
#> ✔ par_usr_pre_lib_spe completed [1.7s, 298 B]
#> + par_usr_ann_mas dispatched
#> ✔ par_usr_ann_mas completed [1.7s, 2.75 kB]
#> + par_usr_pre_lib_sop_mer dispatched
#> ✔ par_usr_pre_lib_sop_mer completed [1.7s, 2.58 kB]
#> + par_usr_pre_lib_sop_ecm dispatched
#> ✔ par_usr_pre_lib_sop_ecm completed [1.7s, 176 B]
#> + par_usr_ann_spe dispatched
#> ✔ par_usr_ann_spe completed [1.7s, 1.03 kB]
#> + par_usr_pre_lib_sop_hmd dispatched
#> ✔ par_usr_pre_lib_sop_hmd completed [1.7s, 178 B]
#> + par_usr_pre_fea_com dispatched
#> ✔ par_usr_pre_fea_com completed [1.7s, 200 B]
#> + par_usr_pre_fea_edg dispatched
#> ✔ par_usr_pre_fea_edg completed [1.7s, 328 B]
#> + par_pre_fea_tab dispatched
#> ✔ par_pre_fea_tab completed [2ms, 278 B]
#> + par_pre_tax dispatched
#> ✔ par_pre_tax completed [1ms, 329 B]
#> + par_fil_ann dispatched
#> ✔ par_fil_ann completed [1ms, 361 B]
#> + par_cre_edg_spe dispatched
#> ✔ par_cre_edg_spe completed [2ms, 389 B]
#> + par_pre_ann_gnp dispatched
#> ✔ par_pre_ann_gnp completed [2ms, 336 B]
#> + par_pre_lib_sop_clo dispatched
#> ✔ par_pre_lib_sop_clo completed [1ms, 232 B]
#> + par_pre_ann_spe dispatched
#> ✔ par_pre_ann_spe completed [2ms, 334 B]
#> + par_pre_lib_sop_big dispatched
#> ✔ par_pre_lib_sop_big completed [1ms, 155 B]
#> + par_pre_ann_mzm dispatched
#> ✔ par_pre_ann_mzm completed [1ms, 341 B]
#> + par_pre_lib_rt dispatched
#> ✔ par_pre_lib_rt completed [2ms, 375 B]
#> + par_wei_ann dispatched
#> ✔ par_wei_ann completed [3ms, 968 B]
#> + par_pre_lib_sop_lot dispatched
#> ✔ par_pre_lib_sop_lot completed [1ms, 186 B]
#> + par_pre_ann_sir dispatched
#> ✔ par_pre_ann_sir completed [1ms, 447 B]
#> + par_cre_com dispatched
#> ✔ par_cre_com completed [1ms, 191 B]
#> + par_pre_lib_spe dispatched
#> ✔ par_pre_lib_spe completed [1ms, 404 B]
#> + par_ann_mas dispatched
#> ✔ par_ann_mas completed [2ms, 1.14 kB]
#> + par_pre_lib_sop_mer dispatched
#> ✔ par_pre_lib_sop_mer completed [2ms, 717 B]
#> + par_pre_lib_sop_ecm dispatched
#> ✔ par_pre_lib_sop_ecm completed [1ms, 191 B]
#> + par_ann_spe dispatched
#> ✔ par_ann_spe completed [2ms, 497 B]
#> + par_pre_lib_sop_hmd dispatched
#> ✔ par_pre_lib_sop_hmd completed [2ms, 191 B]
#> + par_pre_fea_com dispatched
#> ✔ par_pre_fea_com completed [1ms, 184 B]
#> + par_pre_fea_edg dispatched
#> ✔ par_pre_fea_edg completed [1ms, 244 B]
#> + par_pre_fea_tab_fil_fea_raw dispatched
#> ✔ par_pre_fea_tab_fil_fea_raw completed [1ms, 451.55 kB]
#> + lib_sop_clo_pre dispatched
#> [2026-03-26 21:25:13.997] [INFO ] Preparing closed structure-organism pairs library
#> [2026-03-26 21:25:13.998] [WARN ] Closed resource not accessible at: ~/Git/lotus-processor/data/processed/240412_closed_metadata.csv.gz. Returning empty template instead.
#> [2026-03-26 21:25:14.015] [INFO ] Exporting parameters to: data/interim/params/260326_212514_prepare_libraries_sop_closed.yaml
#> [2026-03-26 21:25:14.017] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/closed_prepared.tsv.gz, n_rows=1]
#> [2026-03-26 21:25:14.019] [INFO ] [OK] Completed: export_output [size_bytes=276] (1ms)
#> ✔ lib_sop_clo_pre completed [24ms, 276 B]
#> + lib_sop_big_pre dispatched
#> [2026-03-26 21:25:14.165] [INFO ] Preparing BiGG structure-organism pairs
#> [2026-03-26 21:25:37.068] [INFO ] > Starting: process_smiles [n_structures=1420]
#> [2026-03-26 21:25:37.069] [INFO ] Processing SMILES with RDKit
#> Downloading uv...Done!
#> Downloading cpython-3.12.13-linux-x86_64-gnu (download) (32.3MiB)
#> Downloaded cpython-3.12.13-linux-x86_64-gnu (download)
#> Downloading numpy (15.9MiB)
#> Downloading pillow (6.7MiB)
#> Downloading rdkit (35.0MiB)
#> Downloaded pillow
#> Downloaded numpy
#> Downloaded rdkit
#> Installed 5 packages in 34ms
#> [2026-03-26 21:25:42.244] [INFO ] Processing 1419 new SMILES with RDKit
#> [2026-03-26 21:25:42.247] [INFO ] Starting SMILES processing pipeline
#> [2026-03-26 21:25:42.248] [INFO ] Input: /tmp/RtmpBbz93e/file24f83b689514.smi
#> [2026-03-26 21:25:42.248] [INFO ] Output: /tmp/RtmpBbz93e/file24f857dc4c93.csv.gz
#> [2026-03-26 21:25:42.248] [INFO ] Input file validated: /tmp/RtmpBbz93e/file24f83b689514.smi
#> [2026-03-26 21:25:42.249] [INFO ] Output file validated: /tmp/RtmpBbz93e/file24f857dc4c93.csv.gz
#> [2026-03-26 21:25:42.249] [INFO ] Processing parameters: workers=8, batch_size=1000, progress_interval=10000
#> [2026-03-26 21:25:42.249] [INFO ] SMILES supplier initialized
#> [2026-03-26 21:25:44.278] [INFO ] Processing complete. Total molecules processed: 1419
#> [2026-03-26 21:25:44.319] [INFO ] Successfully processed 1419 SMILES
#> [2026-03-26 21:25:44.388] [INFO ] [OK] Completed: process_smiles [n_processed=1419] (7.3s)
#> [2026-03-26 21:25:50.646] [INFO ] > Starting: process_smiles [n_structures=2084]
#> [2026-03-26 21:25:50.647] [INFO ] Processing SMILES with RDKit
#> [2026-03-26 21:25:50.660] [INFO ] Processing 1241 new SMILES with RDKit
#> [2026-03-26 21:25:50.661] [INFO ] Starting SMILES processing pipeline
#> [2026-03-26 21:25:50.661] [INFO ] Input: /tmp/RtmpBbz93e/file24f82338e76c.smi
#> [2026-03-26 21:25:50.661] [INFO ] Output: /tmp/RtmpBbz93e/file24f87bcbbe40.csv.gz
#> [2026-03-26 21:25:50.662] [INFO ] Input file validated: /tmp/RtmpBbz93e/file24f82338e76c.smi
#> [2026-03-26 21:25:50.662] [INFO ] Output file validated: /tmp/RtmpBbz93e/file24f87bcbbe40.csv.gz
#> [2026-03-26 21:25:50.662] [INFO ] Processing parameters: workers=8, batch_size=1000, progress_interval=10000
#> [2026-03-26 21:25:50.662] [INFO ] SMILES supplier initialized
#> [2026-03-26 21:25:52.479] [INFO ] Processing complete. Total molecules processed: 1241
#> [2026-03-26 21:25:52.514] [INFO ] Successfully processed 1241 SMILES
#> [2026-03-26 21:25:52.522] [INFO ] [OK] Completed: process_smiles [n_processed=1241] (1.9s)
#> [2026-03-26 21:25:52.606] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/bigg_prepared.tsv.gz, n_rows=2358]
#> [2026-03-26 21:25:52.620] [INFO ] [OK] Completed: export_output [size_bytes=85489] (14ms)
#> ✔ lib_sop_big_pre completed [38.5s, 85.49 kB]
#> + lib_rt dispatched
#> [2026-03-26 21:25:52.971] [INFO ] Preparing retention time libraries
#> [2026-03-26 21:25:52.982] [WARN ] No retention time library found, returning empty retention time and sop tables.
#> [2026-03-26 21:25:53.029] [INFO ] Exporting parameters to: data/interim/params/260326_212553_prepare_libraries_rt.yaml
#> [2026-03-26 21:25:53.031] [INFO ] > Starting: export_output [file=data/interim/libraries/rt/prepared.tsv.gz, n_rows=1]
#> [2026-03-26 21:25:53.032] [INFO ] [OK] Completed: export_output [size_bytes=86] (1ms)
#> [2026-03-26 21:25:53.035] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/rt_prepared.tsv.gz, n_rows=1]
#> [2026-03-26 21:25:53.037] [INFO ] [OK] Completed: export_output [size_bytes=105] (1ms)
#> ✔ lib_rt completed [67ms, 191 B]
#> + lib_sop_lot_pre dispatched
#> [2026-03-26 21:25:53.324] [INFO ] > Starting: prepare_libraries_sop_lotus [input=data/source/libraries/sop/lotus.csv.gz]
#> [2026-03-26 21:26:00.874] [INFO ] [OK] Completed: prepare_libraries_sop_lotus [n_pairs=791809] (7.5s)
#> [2026-03-26 21:26:00.876] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/lotus_prepared.tsv.gz, n_rows=791809]
#> [2026-03-26 21:26:04.392] [INFO ] [OK] Completed: export_output [size_bytes=46518841] (3.5s)
#> ✔ lib_sop_lot_pre completed [11.1s, 46.52 MB]
#> + lib_spe_exp_int_pre dispatched
#> [2026-03-26 21:26:05.161] [INFO ] > Starting: prepare_libraries_spectra [library_name=internal, n_input_files=1]
#> [2026-03-26 21:26:05.166] [WARN ] Input file(s) not found; creating empty library template
#> [2026-03-26 21:26:06.514] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/internal_prepared.tsv.gz, n_rows=1]
#> [2026-03-26 21:26:06.516] [INFO ] [OK] Completed: export_output [size_bytes=106] (2ms)
#> [2026-03-26 21:26:06.584] [INFO ] Exporting parameters to: data/interim/params/260326_212606_prepare_libraries_spectra.yaml
#> [2026-03-26 21:26:06.586] [INFO ] [OK] Completed: prepare_libraries_spectra [n_structures=1, n_spectra_total=2, files_exported=3] (1.4s)
#> ✔ lib_spe_exp_int_pre completed [1.4s, 1.31 kB]
#> + lib_sop_mer_str_pro dispatched
#> [2026-03-26 21:26:07.107] [INFO ] > Starting: download_file [url=https://github.com/taxonomicallyinformedannotation/tima-example-files/raw/main/processed.csv.gz, destination=data/interim/libraries/sop/merged/structures/processed.csv.gz]
#> [2026-03-26 21:26:07.871] [INFO ] [OK] Completed: download_file [size_bytes=81527879] (764ms)
#> ✔ lib_sop_mer_str_pro completed [770ms, 81.53 MB]
#> + lib_sop_ecm_pre dispatched
#> [2026-03-26 21:26:08.302] [INFO ] Preparing ECMDB structure-organism pairs
#> [2026-03-26 21:26:08.969] [INFO ] Exporting parameters to: data/interim/params/260326_212608_prepare_libraries_sop_ecmdb.yaml
#> [2026-03-26 21:26:08.971] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/ecmdb_prepared.tsv.gz, n_rows=3760]
#> [2026-03-26 21:26:08.990] [INFO ] [OK] Completed: export_output [size_bytes=177466] (19ms)
#> ✔ lib_sop_ecm_pre completed [691ms, 177.47 kB]
#> + par_ann_spe_fil_spe_raw dispatched
#> ✔ par_ann_spe_fil_spe_raw completed [0ms, 7.77 MB]
#> + lib_sop_hmd_pre dispatched
#> [2026-03-26 21:26:09.797] [INFO ] > Starting: prepare_libraries_sop_hmdb_like [source=HMDB, input=data/source/libraries/sop/hmdb/structures.zip, tag=NA]
#> [2026-03-26 21:26:09.828] [WARN ] Empty dataframe in select_sop_columns
#> [2026-03-26 21:26:09.834] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/hmdb_prepared.tsv.gz, n_rows=0]
#> [2026-03-26 21:26:09.836] [INFO ] [OK] Completed: export_output [size_bytes=255] (2ms)
#> [2026-03-26 21:26:09.837] [INFO ] [OK] Completed: prepare_libraries_sop_hmdb_like [n_pairs=0] (40ms)
#> ✔ lib_sop_hmd_pre completed [41ms, 255 B]
#> + input_features dispatched
#> ✔ input_features completed [0ms, 451.55 kB]
#> + lib_rt_rts dispatched
#> ✔ lib_rt_rts completed [1ms, 86 B]
#> + lib_rt_sop dispatched
#> ✔ lib_rt_sop completed [0ms, 105 B]
#> + lib_spe_exp_int_pre_sop dispatched
#> ✔ lib_spe_exp_int_pre_sop completed [0ms, 106 B]
#> + lib_spe_exp_int_pre_pos dispatched
#> ✔ lib_spe_exp_int_pre_pos completed [1ms, 601 B]
#> + lib_spe_exp_int_pre_neg dispatched
#> ✔ lib_spe_exp_int_pre_neg completed [0ms, 601 B]
#> + input_spectra dispatched
#> ✔ input_spectra completed [0ms, 7.77 MB]
#> + fea_pre dispatched
#> [2026-03-26 21:26:12.697] [INFO ] > Starting: prepare_features_tables [input=data/source/example_features.csv, candidates=1]
#> [2026-03-26 21:26:12.818] [INFO ] Prepared 5328 feature-sample pairs
#> [2026-03-26 21:26:12.820] [INFO ] [OK] Completed: prepare_features_tables [n_features=5328] (123ms)
#> [2026-03-26 21:26:12.845] [INFO ] Exporting parameters to: data/interim/params/260326_212612_prepare_features_tables.yaml
#> [2026-03-26 21:26:12.847] [INFO ] > Starting: export_output [file=data/interim/features/example_features.tsv.gz, n_rows=5328]
#> [2026-03-26 21:26:12.861] [INFO ] [OK] Completed: export_output [size_bytes=95629] (14ms)
#> ✔ fea_pre completed [167ms, 95.63 kB]
#> + lib_sop_mer dispatched
#> [2026-03-26 21:26:13.225] [INFO ] > Starting: prepare_libraries_sop_merged [n_libraries=24, filter_enabled=FALSE, filter_level=none]
#> [2026-03-26 21:26:27.371] [INFO ] Splitting SOP library into standardized components
#> [2026-03-26 21:26:31.826] [INFO ] > Starting: process_smiles [n_structures=1467154]
#> [2026-03-26 21:26:31.827] [INFO ] Processing SMILES with RDKit
#> [2026-03-26 21:26:39.373] [INFO ] Processing 25 new SMILES with RDKit
#> [2026-03-26 21:26:39.375] [INFO ] Starting SMILES processing pipeline
#> [2026-03-26 21:26:39.375] [INFO ] Input: /tmp/RtmpBbz93e/file24f86ec6bfa5.smi
#> [2026-03-26 21:26:39.376] [INFO ] Output: /tmp/RtmpBbz93e/file24f86b09b537.csv.gz
#> [2026-03-26 21:26:39.376] [INFO ] Input file validated: /tmp/RtmpBbz93e/file24f86ec6bfa5.smi
#> [2026-03-26 21:26:39.376] [INFO ] Output file validated: /tmp/RtmpBbz93e/file24f86b09b537.csv.gz
#> [2026-03-26 21:26:39.376] [INFO ] Processing parameters: workers=8, batch_size=1000, progress_interval=10000
#> [2026-03-26 21:26:39.376] [INFO ] SMILES supplier initialized
#> [21:26:39] Explicit valence for atom # 1 N, 3, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 1
#> [21:26:39] ERROR: Explicit valence for atom # 1 N, 3, is greater than permitted
#> [21:26:39] Explicit valence for atom # 31 O, 3, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 4
#> [21:26:39] ERROR: Explicit valence for atom # 31 O, 3, is greater than permitted
#> [21:26:39] Explicit valence for atom # 4 N, 4, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 5
#> [21:26:39] ERROR: Explicit valence for atom # 4 N, 4, is greater than permitted
#> [21:26:39] Explicit valence for atom # 26 N, 4, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 6
#> [21:26:39] ERROR: Explicit valence for atom # 26 N, 4, is greater than permitted
#> [21:26:39] Explicit valence for atom # 0 Ar, 1, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 7
#> [21:26:39] ERROR: Explicit valence for atom # 0 Ar, 1, is greater than permitted
#> [21:26:39] Explicit valence for atom # 0 O, 3, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 8
#> [21:26:39] ERROR: Explicit valence for atom # 0 O, 3, is greater than permitted
#> [21:26:39] Explicit valence for atom # 7 O, 3, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 9
#> [21:26:39] ERROR: Explicit valence for atom # 7 O, 3, is greater than permitted
#> [21:26:39] Explicit valence for atom # 6 Al, 6, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 10
#> [21:26:39] ERROR: Explicit valence for atom # 6 Al, 6, is greater than permitted
#> [21:26:39] Explicit valence for atom # 2 Al, 4, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 11
#> [21:26:39] ERROR: Explicit valence for atom # 2 Al, 4, is greater than permitted
#> [21:26:39] Explicit valence for atom # 3 Al, 5, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 12
#> [21:26:39] ERROR: Explicit valence for atom # 3 Al, 5, is greater than permitted
#> [21:26:39] Explicit valence for atom # 9 C, 5, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 13
#> [21:26:39] ERROR: Explicit valence for atom # 9 C, 5, is greater than permitted
#> [21:26:39] Explicit valence for atom # 1 F, 2, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 14
#> [21:26:39] ERROR: Explicit valence for atom # 1 F, 2, is greater than permitted
#> [21:26:39] Explicit valence for atom # 0 Br, 1, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 15
#> [21:26:39] ERROR: Explicit valence for atom # 0 Br, 1, is greater than permitted
#> [21:26:39] Explicit valence for atom # 1 Br, 5, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 16
#> [21:26:39] ERROR: Explicit valence for atom # 1 Br, 5, is greater than permitted
#> [21:26:39] Explicit valence for atom # 1 Br, 3, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 17
#> [21:26:39] ERROR: Explicit valence for atom # 1 Br, 3, is greater than permitted
#> [21:26:39] Explicit valence for atom # 4 Sb, 8, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 18
#> [21:26:39] ERROR: Explicit valence for atom # 4 Sb, 8, is greater than permitted
#> [21:26:39] Explicit valence for atom # 0 P, 11, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 19
#> [21:26:39] ERROR: Explicit valence for atom # 0 P, 11, is greater than permitted
#> [21:26:39] Can't kekulize mol. Unkekulized atoms: 6 7 8 9 10 11 12 13 14
#> [21:26:39] ERROR: Could not sanitize molecule on line 20
#> [21:26:39] ERROR: Can't kekulize mol. Unkekulized atoms: 6 7 8 9 10 11 12 13 14
#> [21:26:39] Explicit valence for atom # 0 He, 1, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 21
#> [21:26:39] ERROR: Explicit valence for atom # 0 He, 1, is greater than permitted
#> [21:26:39] Explicit valence for atom # 0 He, 1, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 22
#> [21:26:39] ERROR: Explicit valence for atom # 0 He, 1, is greater than permitted
#> [21:26:39] Explicit valence for atom # 0 He, 1, is greater than permitted
#> [21:26:39] ERROR: Could not sanitize molecule on line 23
#> [21:26:39] ERROR: Explicit valence for atom # 0 He, 1, is greater than permitted
#> [21:26:39] Explicit valence for atom # 4 P, 7, is greater than permitted
#> [2026-03-26 21:26:39.382] [WARNING] Failed to process SMILES '[H][C@](O)(CO[P-]([O])(=O)=O)C=O': Explicit valence for atom # 4 P, 7, is greater than permitted
#> [21:26:39] Explicit valence for atom # 7 Si, 6, is greater than permitted
#> [2026-03-26 21:26:39.384] [WARNING] Failed to process SMILES 'C1COCC[NH+]1C[Si-]23(OC(C(O2)(C4=CC=CC=C4)C5=CC=CC=C5)(C6=CC=CC=C6)C7=CC=CC=C7)OC(C(O3)(C8=CC=CC=C8)C9=CC=CC=C9)(C1=CC=CC=C1)C1=CC=CC=C1': Explicit valence for atom # 7 Si, 6, is greater than permitted
#> [21:26:39] Explicit valence for atom # 7 As, 7, is greater than permitted
#> [2026-03-26 21:26:39.385] [WARNING] Failed to process SMILES 'C1=CC=C2C(=C1)O[As-]34(O2)(OC5=CC=CC=C5O3)OC6=CC=CC=C6O4': Explicit valence for atom # 7 As, 7, is greater than permitted
#> [21:26:39] Explicit valence for atom # 56 P, 7, is greater than permitted
#> [2026-03-26 21:26:39.385] [WARNING] Failed to process SMILES 'CC(C)=CCCC(C)=CCCC(C)=CCCC(C)=CCCC(C)=CCCC(C)=CCCC(C)=CCCC(C)=CCCC(C)=CCCC(C)=CCCC(C)=CCO[P-]([O])(=O)=O': Explicit valence for atom # 56 P, 7, is greater than permitted
#> [2026-03-26 21:26:39.385] [WARNING] Batch processing: 4/4 molecules failed
#> [2026-03-26 21:26:39.386] [INFO ] Processing complete. Total molecules processed: 0
#> [2026-03-26 21:26:39.414] [INFO ] Successfully processed 0 SMILES
#> [2026-03-26 21:26:46.367] [INFO ] [OK] Completed: process_smiles [n_processed=1370110] (14.5s)
#> [2026-03-26 21:27:04.192] [INFO ] Referenced structure-organism pairs (723,703)
#> [2026-03-26 21:27:08.536] [INFO ] Structures: 288,738 stereoisomers, 1,014,890 without stereochemistry, 1,090,547 constitutional isomers
#> [2026-03-26 21:27:41.787] [INFO ] Unique organisms (36,801)
#> [2026-03-26 21:27:41.878] [INFO ] Processing 919 organism name(s) for OTT taxonomy lookup
#> [2026-03-26 21:27:42.328] [INFO ] Querying OTT API in 10 batches
#> [2026-03-26 21:27:47.073] [INFO ] Retrieving detailed taxonomy for 13 unique OTT IDs
#> [2026-03-26 21:27:48.861] [INFO ] Got OTTaxonomy!
#> [2026-03-26 21:27:48.898] [INFO ] Exporting parameters to: data/interim/params/260326_212748_prepare_libraries_sop_merged.yaml
#> [2026-03-26 21:27:48.901] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/merged/keys.tsv.gz, n_rows=723703]
#> [2026-03-26 21:27:49.838] [INFO ] [OK] Completed: export_output [size_bytes=14591949] (938ms)
#> [2026-03-26 21:27:49.840] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/merged/organisms/taxonomies/ott.tsv.gz, n_rows=35896]
#> [2026-03-26 21:27:49.928] [INFO ] [OK] Completed: export_output [size_bytes=939127] (87ms)
#> [2026-03-26 21:27:49.930] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/merged/structures/stereo.tsv.gz, n_rows=1305314]
#> [2026-03-26 21:27:53.122] [INFO ] [OK] Completed: export_output [size_bytes=40812325] (3.2s)
#> [2026-03-26 21:27:53.124] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/merged/structures/metadata.tsv.gz, n_rows=1344159]
#> [2026-03-26 21:27:54.851] [INFO ] [OK] Completed: export_output [size_bytes=34187322] (1.7s)
#> [2026-03-26 21:27:54.853] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/merged/structures/names.tsv.gz, n_rows=294729]
#> [2026-03-26 21:27:55.341] [INFO ] [OK] Completed: export_output [size_bytes=8849024] (489ms)
#> [2026-03-26 21:27:55.343] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/merged/structures/taxonomies/classyfire.tsv.gz, n_rows=146393]
#> [2026-03-26 21:27:55.491] [INFO ] [OK] Completed: export_output [size_bytes=2489314] (148ms)
#> [2026-03-26 21:27:55.493] [INFO ] > Starting: export_output [file=data/interim/libraries/sop/merged/structures/taxonomies/npc.tsv.gz, n_rows=141815]
#> [2026-03-26 21:27:55.781] [INFO ] [OK] Completed: export_output [size_bytes=2389944] (288ms)
#> [2026-03-26 21:27:55.782] [INFO ] [OK] Completed: prepare_libraries_sop_merged [n_pairs=723703, n_structures=1305314, n_organisms=35896, files_exported=7] (1m 43s)
#> ✔ lib_sop_mer completed [1m 42.6s, 104.26 MB]
#> + fea_edg_spe dispatched
#> [2026-03-26 21:27:57.018] [INFO ] > Starting: create_edges_spectra [method=gnps, threshold=0.7, n_input_files=1]
#> [2026-03-26 21:27:57.020] [INFO ] Creating spectral similarity network edges
#> [2026-03-26 21:27:57.021] [INFO ] Importing spectra from: data/source/example_spectra.mgf
#> [2026-03-26 21:27:57.048] [INFO ] Reading MGF file (7.41 MB) with optimized parser: data/source/example_spectra.mgf
#> [2026-03-26 21:27:58.761] [INFO ] Processed 10000 spectra...
#> [2026-03-26 21:27:59.958] [INFO ] Total spectra read: 16282
#> [2026-03-26 21:28:05.770] [INFO ] Loaded 16282 spectra from file
#> [2026-03-26 21:28:05.802] [INFO ] Combining replicate spectra by FEATURE_ID
#> [2026-03-26 21:28:08.486] [INFO ] Combined replicates: 12195 -> 4087 spectra
#> [2026-03-26 21:28:08.488] [INFO ] Sanitizing 4087 spectra (cutoff: 0)
#> [2026-03-26 21:28:09.612] [INFO ] Sanitization complete: 3331/4087 spectra retained (81.5%, 756 removed)
#> [2026-03-26 21:28:09.613] [INFO ] Import complete: 3331 spectra ready for analysis
#> [2026-03-26 21:28:09.614] [INFO ] ======================================
#> [2026-03-26 21:28:09.615] [INFO ] Take yourself a break, you deserve it.
#> [2026-03-26 21:28:09.616] [INFO ] ======================================
#> [2026-03-26 21:28:09.617] [INFO ] > Starting: create_edges [n_spectra=3331, method=gnps, threshold=0.7, min_peaks=6]
#> [2026-03-26 21:28:21.238] [INFO ] Processed 500 / 3330 queries
#> [2026-03-26 21:28:30.846] [INFO ] Processed 1000 / 3330 queries
#> [2026-03-26 21:28:38.579] [INFO ] Processed 1500 / 3330 queries
#> [2026-03-26 21:28:44.253] [INFO ] Processed 2000 / 3330 queries
#> [2026-03-26 21:28:48.110] [INFO ] Processed 2500 / 3330 queries
#> [2026-03-26 21:28:50.212] [INFO ] Processed 3000 / 3330 queries
#> [2026-03-26 21:28:50.641] [INFO ] Here is the distribution of edge similarity scores (0.1 bins) BEFORE filtering:
#> [2026-03-26 21:28:50.644] [INFO ]
#> bin N
#> [0,0.1] 4066572
#> (0.1,0.2] 899766
#> (0.2,0.3] 323606
#> (0.3,0.4] 134325
#> (0.4,0.5] 60461
#> (0.5,0.6] 30399
#> (0.6,0.7] 16141
#> (0.7,0.8] 8374
#> (0.8,0.9] 4674
#> (0.9,1] 1797
#> [2026-03-26 21:28:50.647] [INFO ] [OK] Completed: create_edges [n_edges=7265, n_comparisons=5546115, pass_rate=0.1%] (41s)
#> [2026-03-26 21:28:50.720] [INFO ] Exporting parameters to: data/interim/params/260326_212850_create_edges_spectra.yaml
#> [2026-03-26 21:28:50.722] [INFO ] > Starting: export_output [file=data/interim/features/example_edgesSpectra.tsv, n_rows=9237]
#> [2026-03-26 21:28:50.725] [INFO ] [OK] Completed: export_output [size_bytes=437871] (3ms)
#> [2026-03-26 21:28:50.726] [INFO ] [OK] Completed: create_edges_spectra [n_edges=9237] (53.7s)
#> ✔ fea_edg_spe completed [53.7s, 437.87 kB]
#> + ann_spe_neg dispatched
#> [2026-03-26 21:28:51.208] [INFO ] ============================================================
#> [2026-03-26 21:28:51.209] [INFO ] Data Sanitizing: Pre-flight Checks
#> [2026-03-26 21:28:51.210] [INFO ] ============================================================
#> [2026-03-26 21:28:51.211] [INFO ] Checking MGF file...
#> [2026-03-26 21:28:51.690] [INFO ] [OK] MGF file: 12195 MS2 spectra found
#> [2026-03-26 21:28:51.691] [INFO ] ============================================================
#> [2026-03-26 21:28:51.692] [INFO ] [OK] All pre-flight checks passed!
#> [2026-03-26 21:28:51.693] [INFO ] Data validation complete. Ready to proceed.
#> [2026-03-26 21:28:51.694] [INFO ] ============================================================
#> [2026-03-26 21:28:51.695] [INFO ] Starting spectral annotation in neg mode
#> [2026-03-26 21:28:51.696] [INFO ] Importing spectra from: data/source/example_spectra.mgf
#> [2026-03-26 21:28:51.697] [INFO ] Reading MGF file (7.41 MB) with optimized parser: data/source/example_spectra.mgf
#> [2026-03-26 21:28:53.980] [INFO ] Processed 10000 spectra...
#> [2026-03-26 21:28:55.136] [INFO ] Total spectra read: 16282
#> [2026-03-26 21:29:01.004] [INFO ] Loaded 16282 spectra from file
#> [2026-03-26 21:29:01.021] [INFO ] Combining replicate spectra by FEATURE_ID
#> [2026-03-26 21:29:01.025] [INFO ] Combined replicates: 0 -> 0 spectra
#> [2026-03-26 21:29:01.026] [WARN ] No spectra to sanitize
#> [2026-03-26 21:29:01.027] [INFO ] Import complete: 0 spectra ready for analysis
#> [2026-03-26 21:29:01.028] [WARN ] No query spectra loaded
#> [2026-03-26 21:29:01.030] [INFO ] Exporting parameters to: data/interim/params/260326_212901_annotate_spectra.yaml
#> [2026-03-26 21:29:01.031] [WARN ] Returning empty annotation template
#> [2026-03-26 21:29:01.034] [INFO ] > Starting: export_output [file=data/interim/annotations/example_spectralMatches_neg.tsv.gz, n_rows=1]
#> [2026-03-26 21:29:01.035] [INFO ] [OK] Completed: export_output [size_bytes=247] (1ms)
#> ✔ ann_spe_neg completed [9.9s, 247 B]
#> + ann_spe_pos dispatched
#> [2026-03-26 21:29:01.633] [INFO ] ============================================================
#> [2026-03-26 21:29:01.634] [INFO ] Data Sanitizing: Pre-flight Checks
#> [2026-03-26 21:29:01.635] [INFO ] ============================================================
#> [2026-03-26 21:29:01.635] [INFO ] Checking MGF file...
#> [2026-03-26 21:29:02.114] [INFO ] [OK] MGF file: 12195 MS2 spectra found
#> [2026-03-26 21:29:02.116] [INFO ] ============================================================
#> [2026-03-26 21:29:02.117] [INFO ] [OK] All pre-flight checks passed!
#> [2026-03-26 21:29:02.117] [INFO ] Data validation complete. Ready to proceed.
#> [2026-03-26 21:29:02.118] [INFO ] ============================================================
#> [2026-03-26 21:29:02.119] [INFO ] Starting spectral annotation in pos mode
#> [2026-03-26 21:29:02.120] [INFO ] Importing spectra from: data/source/example_spectra.mgf
#> [2026-03-26 21:29:02.121] [INFO ] Reading MGF file (7.41 MB) with optimized parser: data/source/example_spectra.mgf
#> [2026-03-26 21:29:03.999] [INFO ] Processed 10000 spectra...
#> [2026-03-26 21:29:05.318] [INFO ] Total spectra read: 16282
#> [2026-03-26 21:29:11.494] [INFO ] Loaded 16282 spectra from file
#> [2026-03-26 21:29:11.518] [INFO ] Combining replicate spectra by FEATURE_ID
#> [2026-03-26 21:29:12.445] [INFO ] Combined replicates: 12195 -> 4087 spectra
#> [2026-03-26 21:29:12.446] [INFO ] Sanitizing 4087 spectra (cutoff: 0)
#> [2026-03-26 21:29:13.617] [INFO ] Sanitization complete: 3331/4087 spectra retained (81.5%, 756 removed)
#> [2026-03-26 21:29:13.618] [INFO ] Import complete: 3331 spectra ready for analysis
#> [2026-03-26 21:29:13.620] [INFO ] Importing spectra from: data/interim/libraries/spectra/is/wikidata_5607185_pos.rds
#> [2026-03-26 21:29:39.216] [INFO ] Loaded 998198 spectra from file
#> [2026-03-26 21:29:41.532] [INFO ] Import complete: 998198 spectra ready for analysis
#> [2026-03-26 21:29:41.534] [INFO ] Importing spectra from: data/interim/libraries/spectra/exp/internal_pos.rds
#> [2026-03-26 21:29:41.535] [INFO ] Loaded 1 spectra from file
#> [2026-03-26 21:29:41.538] [INFO ] Import complete: 0 spectra ready for analysis
#> [2026-03-26 21:29:41.539] [INFO ] Importing spectra from: data/interim/libraries/spectra/exp/gnps_11566051_pos.rds
#> [2026-03-26 21:29:47.533] [INFO ] Loaded 272264 spectra from file
#> [2026-03-26 21:29:47.692] [INFO ] Import complete: 272263 spectra ready for analysis
#> [2026-03-26 21:29:47.693] [INFO ] Importing spectra from: data/interim/libraries/spectra/exp/massbank_202510_pos.rds
#> [2026-03-26 21:29:48.288] [INFO ] Loaded 62855 spectra from file
#> [2026-03-26 21:29:48.331] [INFO ] Import complete: 62855 spectra ready for analysis
#> [2026-03-26 21:29:48.333] [INFO ] Importing spectra from: data/interim/libraries/spectra/exp/merlin_16984129_pos.rds
#> [2026-03-26 21:29:54.295] [INFO ] Loaded 328190 spectra from file
#> [2026-03-26 21:29:54.500] [INFO ] Import complete: 328190 spectra ready for analysis
#> [2026-03-26 21:30:09.144] [INFO ]
#> library spectra unique_connectivities
#> ISDB - Wikidata 998198 998198
#> merlin 328190 42482
#> gnps 272263 22507
#> massbank 62855 6939
#> [2026-03-26 21:30:09.948] [INFO ] > Starting: calculate_entropy_similarity [n_library=514526, n_query=3331, method=gnps]
#> [2026-03-26 21:30:09.949] [INFO ] Calculating entropy and similarity for 3331 spectra
#> [2026-03-26 21:30:20.212] [INFO ] Processed 500 / 3331 queries
#> [2026-03-26 21:30:28.335] [INFO ] Processed 1000 / 3331 queries
#> [2026-03-26 21:30:34.389] [INFO ] Processed 1500 / 3331 queries
#> [2026-03-26 21:30:40.537] [INFO ] Processed 2000 / 3331 queries
#> [2026-03-26 21:30:45.645] [INFO ] Processed 2500 / 3331 queries
#> [2026-03-26 21:30:50.506] [INFO ] Processed 3000 / 3331 queries
#> [2026-03-26 21:30:53.370] [WARN ] Sanitized 59094/514526 library spectra on-demand before similarity scoring.
#> [2026-03-26 21:30:53.371] [INFO ] Processed 3331 / 3331 queries
#> [2026-03-26 21:30:53.396] [INFO ] [OK] Completed: calculate_entropy_similarity [n_comparisons=982591] (43.4s)
#> [2026-03-26 21:30:53.408] [INFO ] > Starting: harmonize_adducts [n_rows=514526]
#> [2026-03-26 21:30:55.213] [INFO ] [OK] Completed: harmonize_adducts [n_unique_before=65, n_unique_after=65] (1.8s)
#> [2026-03-26 21:30:56.164] [INFO ] Here is the distribution of annotation similarity scores (0.1 bins):
#> [2026-03-26 21:30:56.166] [INFO ]
#> bin N
#> [0,0.1] 485888
#> (0.1,0.2] 56788
#> (0.2,0.3] 22507
#> (0.3,0.4] 10409
#> (0.4,0.5] 4982
#> (0.5,0.6] 2213
#> (0.6,0.7] 921
#> (0.7,0.8] 624
#> (0.8,0.9] 487
#> (0.9,1] 245
#> [2026-03-26 21:30:56.220] [INFO ] 303641 Candidates annotated on 3177 features (threshold >= 0).
#> [2026-03-26 21:30:56.224] [INFO ] Exporting parameters to: data/interim/params/260326_213056_annotate_spectra.yaml
#> [2026-03-26 21:30:56.225] [INFO ] > Starting: export_output [file=data/interim/annotations/example_spectralMatches_pos.tsv.gz, n_rows=585064]
#> [2026-03-26 21:30:57.876] [INFO ] [OK] Completed: export_output [size_bytes=34593842] (1.7s)
#> ✔ ann_spe_pos completed [1m 56.3s, 34.59 MB]
#> + lib_mer_org_tax_ott dispatched
#> ✔ lib_mer_org_tax_ott completed [1ms, 939.13 kB]
#> + lib_mer_str_tax_npc dispatched
#> ✔ lib_mer_str_tax_npc completed [0ms, 2.39 MB]
#> + lib_mer_key dispatched
#> ✔ lib_mer_key completed [0ms, 14.59 MB]
#> + lib_mer_str_met dispatched
#> ✔ lib_mer_str_met completed [0ms, 34.19 MB]
#> + lib_mer_str_nam dispatched
#> ✔ lib_mer_str_nam completed [0ms, 8.85 MB]
#> + lib_mer_str_stereo dispatched
#> ✔ lib_mer_str_stereo completed [0ms, 40.81 MB]
#> + lib_mer_str_tax_cla dispatched
#> ✔ lib_mer_str_tax_cla completed [1ms, 2.49 MB]
#> + edg_spe dispatched
#> ✔ edg_spe completed [0ms, 437.87 kB]
#> + tax_pre dispatched
#> [2026-03-26 21:31:02.663] [INFO ] > Starting: prepare_taxa [taxon=NULL]
#> [2026-03-26 21:31:02.831] [INFO ] Processing 2 organism name(s) for OTT taxonomy lookup
#> [2026-03-26 21:31:03.126] [INFO ] Querying OTT API in 1 batches
#> [2026-03-26 21:31:03.340] [INFO ] Retrying failed queries using genus names only
#> [2026-03-26 21:31:03.347] [INFO ] Retrying with 1 genus names: blk
#> [2026-03-26 21:31:03.734] [INFO ] Retrieving detailed taxonomy for 1 unique OTT IDs
#> [2026-03-26 21:31:04.030] [INFO ] Got OTTaxonomy!
#> [2026-03-26 21:31:04.470] [INFO ] [OK] Completed: prepare_taxa [n_features=5328] (1.8s)
#> [2026-03-26 21:31:04.501] [INFO ] Exporting parameters to: data/interim/params/260326_213104_prepare_taxa.yaml
#> [2026-03-26 21:31:04.503] [INFO ] > Starting: export_output [file=data/interim/taxa/example_taxed.tsv.gz, n_rows=5328]
#> [2026-03-26 21:31:04.510] [INFO ] [OK] Completed: export_output [size_bytes=19697] (7ms)
#> ✔ tax_pre completed [1.8s, 19.70 kB]
#> + ann_spe_exp_gnp_pre dispatched
#> [2026-03-26 21:31:04.946] [INFO ] > Starting: prepare_annotations_gnps [n_files=1]
#> [2026-03-26 21:31:04.947] [WARN ] No GNPS annotations found, returning an empty file instead
#> [2026-03-26 21:31:04.949] [INFO ] [OK] Completed: prepare_annotations_gnps [n_annotations=1] (3ms)
#> [2026-03-26 21:31:04.968] [INFO ] Exporting parameters to: data/interim/params/260326_213104_prepare_annotations_gnps.yaml
#> [2026-03-26 21:31:04.969] [INFO ] > Starting: export_output [file=data/interim/annotations/example_gnpsPrepared.tsv.gz, n_rows=1]
#> [2026-03-26 21:31:04.971] [INFO ] [OK] Completed: export_output [size_bytes=247] (1ms)
#> ✔ ann_spe_exp_gnp_pre completed [32ms, 247 B]
#> + ann_spe_exp_mzm_pre dispatched
#> [2026-03-26 21:31:05.353] [INFO ] > Starting: prepare_annotations_mzmine [n_files=1]
#> [2026-03-26 21:31:05.354] [WARN ] No mzmine annotations found, returning an empty file instead
#> [2026-03-26 21:31:05.356] [INFO ] [OK] Completed: prepare_annotations_mzmine [n_annotations=1] (3ms)
#> [2026-03-26 21:31:05.370] [INFO ] Exporting parameters to: data/interim/params/260326_213105_prepare_annotations_mzmine.yaml
#> [2026-03-26 21:31:05.372] [INFO ] > Starting: export_output [file=data/interim/annotations/example_mzminePrepared.tsv.gz, n_rows=1]
#> [2026-03-26 21:31:05.374] [INFO ] [OK] Completed: export_output [size_bytes=247] (1ms)
#> ✔ ann_spe_exp_mzm_pre completed [23ms, 247 B]
#> + ann_spe_pre dispatched
#> [2026-03-26 21:31:05.756] [INFO ] Preparing spectral matching annotations from 2 file(s)
#> [2026-03-26 21:31:13.984] [INFO ] > Starting: complement_metadata [n_input=585064]
#> [2026-03-26 21:32:14.155] [INFO ] [OK] Completed: complement_metadata [n_enriched=599730] (1m)
#> [2026-03-26 21:32:14.172] [INFO ] Exporting parameters to: data/interim/params/260326_213214_prepare_annotations_spectra.yaml
#> [2026-03-26 21:32:14.174] [INFO ] > Starting: export_output [file=data/interim/annotations/example_spectralMatchesPrepared.tsv.gz, n_rows=599730]
#> [2026-03-26 21:32:15.812] [INFO ] [OK] Completed: export_output [size_bytes=37159900] (1.6s)
#> ✔ ann_spe_pre completed [1m 10.1s, 37.16 MB]
#> + ann_ms1_pre dispatched
#> [2026-03-26 21:32:16.582] [INFO ] > Starting: annotate_masses [ms_mode=pos, tolerance_ppm=10, tolerance_rt=0.02]
#> [2026-03-26 21:32:16.583] [INFO ] Starting mass-based annotation
#> [2026-03-26 21:32:16.584] [INFO ] ============================================================
#> [2026-03-26 21:32:16.584] [INFO ] Data Sanitizing: Pre-flight Checks
#> [2026-03-26 21:32:16.585] [INFO ] ============================================================
#> [2026-03-26 21:32:16.586] [INFO ] Checking features file...
#> [2026-03-26 21:32:16.624] [INFO ] [OK] Features file: 5328 rows, 5 columns
#> [2026-03-26 21:32:16.625] [INFO ] ============================================================
#> [2026-03-26 21:32:16.626] [INFO ] [OK] All pre-flight checks passed!
#> [2026-03-26 21:32:16.627] [INFO ] Data validation complete. Ready to proceed.
#> [2026-03-26 21:32:16.628] [INFO ] ============================================================
#> [2026-03-26 21:32:16.666] [INFO ] Processing 5328 features for annotation
#> [2026-03-26 21:32:31.637] [INFO ] Already 2112 adducts previously detected
#> [2026-03-26 21:32:31.638] [INFO ] > Starting: harmonize_adducts [n_rows=5328]
#> [2026-03-26 21:32:31.648] [INFO ] [OK] Completed: harmonize_adducts [n_unique_before=13, n_unique_after=13] (10ms)
#> [2026-03-26 21:32:31.710] [INFO ] Here are the top 10 observed m/z differences inside the RT windows:
#> [2026-03-26 21:32:31.712] [INFO ]
#> bin N
#> (4.8501,5.0366] 352
#> (21.822,22.009] 283
#> (16.973,17.16] 208
#> (17.906,18.092] 192
#> (15.854,16.041] 172
#> (39.914,40.1] 143
#> (38.981,39.168] 137
#> (34.878,35.065] 115
#> (77.962,78.148] 114
#> (1.8659,2.0524] 108
#> [2026-03-26 21:32:31.712] [INFO ] These differences may help identify potential preprocessing issues
#> [2026-03-26 21:32:34.872] [WARN ] Some adducts were unproperly detected, defaulting to (de)protonated
#> [2026-03-26 21:33:18.735] [INFO ] > Starting: decorate_masses [n_annotations=175774]
#> [2026-03-26 21:33:18.774] [INFO ] MS1 annotations: 42738 unique structures across 4017 features
#> [2026-03-26 21:33:18.775] [INFO ] [OK] Completed: decorate_masses [n_structures=42738, n_features=4017] (40ms)
#> [2026-03-26 21:33:18.830] [INFO ] Exporting parameters to: data/interim/params/260326_213318_annotate_masses.yaml
#> [2026-03-26 21:33:18.832] [INFO ] > Starting: export_output [file=data/interim/features/example_edgesMasses.tsv, n_rows=2653]
#> [2026-03-26 21:33:18.834] [INFO ] [OK] Completed: export_output [size_bytes=81706] (2ms)
#> [2026-03-26 21:33:20.267] [INFO ] > Starting: complement_metadata [n_input=175774]
#> [2026-03-26 21:33:38.133] [INFO ] [OK] Completed: complement_metadata [n_enriched=175823] (17.9s)
#> [2026-03-26 21:33:38.135] [INFO ] > Starting: export_output [file=data/interim/annotations/example_ms1Prepared.tsv.gz, n_rows=175823]
#> [2026-03-26 21:33:38.943] [INFO ] [OK] Completed: export_output [size_bytes=12062954] (807ms)
#> [2026-03-26 21:33:38.944] [INFO ] [OK] Completed: annotate_masses [n_annotations=175774, n_edges=2653] (1m 22s)
#> ✔ ann_ms1_pre completed [1m 22.4s, 12.14 MB]
#> + ann_sir_pre dispatched
#> [2026-03-26 21:33:39.946] [INFO ] > Starting: prepare_annotations_sirius [version=6]
#> [2026-03-26 21:33:40.111] [INFO ] > Starting: complement_metadata [n_input=2571]
#> [2026-03-26 21:33:46.814] [INFO ] [OK] Completed: complement_metadata [n_enriched=2571] (6.7s)
#> [2026-03-26 21:33:46.826] [INFO ] [OK] Completed: prepare_annotations_sirius [n_canopus=15, n_formulas=15, n_structures=2571] (6.9s)
#> [2026-03-26 21:33:46.852] [INFO ] Exporting parameters to: data/interim/params/260326_213346_prepare_annotations_sirius.yaml
#> [2026-03-26 21:33:46.855] [INFO ] > Starting: export_output [file=data/interim/annotations/example_canopusPrepared.tsv.gz, n_rows=15]
#> [2026-03-26 21:33:46.856] [INFO ] [OK] Completed: export_output [size_bytes=830] (2ms)
#> [2026-03-26 21:33:46.858] [INFO ] > Starting: export_output [file=data/interim/annotations/example_formulaPrepared.tsv.gz, n_rows=15]
#> [2026-03-26 21:33:46.860] [INFO ] [OK] Completed: export_output [size_bytes=493] (2ms)
#> [2026-03-26 21:33:46.861] [INFO ] > Starting: export_output [file=data/interim/annotations/example_siriusPrepared.tsv.gz, n_rows=2571]
#> [2026-03-26 21:33:46.873] [INFO ] [OK] Completed: export_output [size_bytes=82968] (12ms)
#> ✔ ann_sir_pre completed [6.9s, 84.29 kB]
#> + ann_ms1_pre_ann dispatched
#> ✔ ann_ms1_pre_ann completed [0ms, 12.06 MB]
#> + ann_ms1_pre_edg dispatched
#> ✔ ann_ms1_pre_edg completed [1ms, 81.71 kB]
#> + ann_sir_pre_str dispatched
#> ✔ ann_sir_pre_str completed [0ms, 82.97 kB]
#> + ann_sir_pre_can dispatched
#> ✔ ann_sir_pre_can completed [1ms, 830 B]
#> + ann_sir_pre_for dispatched
#> ✔ ann_sir_pre_for completed [0ms, 493 B]
#> + fea_edg_pre dispatched
#> [2026-03-26 21:33:50.245] [INFO ] > Starting: prepare_features_edges [n_edge_types=2]
#> [2026-03-26 21:33:50.286] [INFO ] [OK] Completed: prepare_features_edges [n_edges=14226] (41ms)
#> [2026-03-26 21:33:50.308] [INFO ] Exporting parameters to: data/interim/params/260326_213350_prepare_features_edges.yaml
#> [2026-03-26 21:33:50.310] [INFO ] > Starting: export_output [file=data/interim/features/example_edges.tsv, n_rows=14226]
#> [2026-03-26 21:33:50.313] [INFO ] [OK] Completed: export_output [size_bytes=644049] (4ms)
#> ✔ fea_edg_pre completed [70ms, 644.05 kB]
#> + ann_fil dispatched
#> [2026-03-26 21:33:50.709] [INFO ] > Starting: filter_annotations [n_annotation_files=5, tolerance_rt=Inf]
#> [2026-03-26 21:33:50.711] [INFO ] Filtering annotations
#> [2026-03-26 21:33:50.754] [INFO ] Processing 5328 unique features for annotation filtering
#> [2026-03-26 21:33:54.576] [INFO ] Removing MS1 annotations superseded by spectral matches
#> [2026-03-26 21:33:58.051] [INFO ] Removed 70134 redundant MS1 annotations
#> [2026-03-26 21:33:58.052] [INFO ] Total annotations before RT filtering: 707992
#> [2026-03-26 21:33:59.476] [INFO ] RT tolerance is Inf: joining RT library without filtering
#> [2026-03-26 21:33:59.875] [INFO ] Removed 0 annotations based on retention time tolerance
#> [2026-03-26 21:34:01.465] [INFO ] Exporting parameters to: data/interim/params/260326_213401_filter_annotations.yaml
#> [2026-03-26 21:34:01.467] [INFO ] > Starting: export_output [file=data/interim/annotations/example_annotationsFiltered.tsv.gz, n_rows=708623]
#> [2026-03-26 21:34:03.562] [INFO ] [OK] Completed: export_output [size_bytes=39563621] (2.1s)
#> [2026-03-26 21:34:03.563] [INFO ] [OK] Completed: filter_annotations [n_filtered=708623] (12.9s)
#> ✔ ann_fil completed [12.9s, 39.56 MB]
#> + fea_com dispatched
#> [2026-03-26 21:34:04.296] [INFO ] > Starting: create_components [n_input_files=1]
#> [2026-03-26 21:34:04.297] [INFO ] Creating components from 1 edge file(s)
#> [2026-03-26 21:34:04.310] [INFO ] Loaded 12330 edges connecting 4572 unique features
#> [2026-03-26 21:34:04.319] [INFO ] Found 1493 components
#> [2026-03-26 21:34:04.330] [INFO ] Component sizes - Min: 1, Max: 2083, Mean: 3.1
#> [2026-03-26 21:34:04.347] [INFO ] Exporting parameters to: data/interim/params/260326_213404_create_components.yaml
#> [2026-03-26 21:34:04.349] [INFO ] > Starting: export_output [file=data/interim/features/example_components.tsv, n_rows=4572]
#> [2026-03-26 21:34:04.351] [INFO ] [OK] Completed: export_output [size_bytes=39000] (2ms)
#> [2026-03-26 21:34:04.352] [INFO ] Components written to: data/interim/features/example_components.tsv
#> [2026-03-26 21:34:04.353] [INFO ] [OK] Completed: create_components [n_components=1493, n_features=4572] (57ms)
#> ✔ fea_com completed [60ms, 39 kB]
#> + int_com dispatched
#> ✔ int_com completed [0ms, 39 kB]
#> + fea_com_pre dispatched
#> [2026-03-26 21:34:05.170] [INFO ] > Starting: prepare_features_components [n_files=1]
#> [2026-03-26 21:34:05.175] [INFO ] [OK] Completed: prepare_features_components [n_assignments=4572] (5ms)
#> [2026-03-26 21:34:05.191] [INFO ] Exporting parameters to: data/interim/params/260326_213405_prepare_features_components.yaml
#> [2026-03-26 21:34:05.193] [INFO ] > Starting: export_output [file=data/interim/features/example_componentsPrepared.tsv, n_rows=4572]
#> [2026-03-26 21:34:05.195] [INFO ] [OK] Completed: export_output [size_bytes=38995] (2ms)
#> ✔ fea_com_pre completed [27ms, 38.99 kB]
#> + ann_wei dispatched
#> [2026-03-26 21:34:05.598] [INFO ] Starting annotation weighting and scoring
#> [2026-03-26 21:34:05.599] [INFO ] > Starting: weight_annotations [n_candidates_neighbors=16, n_candidates_final=1]
#> [2026-03-26 21:34:22.799] [INFO ]
#> candidate_library n
#> ISDB - Wikidata 523258
#> TIMA MS1 79238
#> merlin 35991
#> gnps 21396
#> massbank 4404
#> SIRIUS 2571
#> [2026-03-26 21:34:29.996] [INFO ] > Starting: weight_bio [n_annotations=662494, n_sop=726501]
#> [2026-03-26 21:34:29.998] [INFO ] Weighting 662494 annotations by biological source
#> [2026-03-26 21:34:35.041] [INFO ] [OK] Completed: weight_bio [n_weighted=662494] (5s)
#> [2026-03-26 21:34:35.043] [INFO ] > Starting: decorate_bio [n_annotations=662494]
#> [2026-03-26 21:34:35.592] [INFO ] Taxonomically informed metabolite annotation reranked:
#> Kingdom level: 152707 candidates (39688 unique)
#> Phylum level: 151662 candidates (39243 unique)
#> Class level: 124560 candidates (33627 unique)
#> Order level: 31020 candidates (9173 unique)
#> Family level: 24908 candidates (7380 unique)
#> Tribe level: 5715 candidates (1247 unique)
#> Genus level: 4839 candidates (985 unique)
#> Species level: 2842 candidates (477 unique)
#> Variety level: 440 candidates (93 unique)
#> Biota level: 440 candidates (93 unique)
#> [2026-03-26 21:34:35.594] [INFO ] [OK] Completed: decorate_bio [n_processed=662494] (551ms)
#> [2026-03-26 21:34:35.595] [INFO ] > Starting: clean_bio [n_annotations=662494, minimal_consistency=0]
#> [2026-03-26 21:34:52.725] [INFO ] [OK] Completed: clean_bio [n_cleaned=661096] (17.1s)
#> [2026-03-26 21:34:52.727] [INFO ] > Starting: weight_chemo [n_input=661096]
#> [2026-03-26 21:34:52.728] [INFO ] Weighting 661096 annotations by chemical consistency
#> [2026-03-26 21:34:55.165] [INFO ] [OK] Completed: weight_chemo [n_weighted=661096] (2.4s)
#> [2026-03-26 21:34:55.167] [INFO ] > Starting: decorate_chemo [n_annotations=661096]
#> [2026-03-26 21:34:55.333] [INFO ] Chemically informed metabolite annotation reranked:
#> Classyfire:
#> Kingdom level: 15805 candidates (10157 unique)
#> Superclass level: 13076 candidates (8158 unique)
#> Class level: 11681 candidates (7153 unique)
#> Parent level: 8663 candidates (5305 unique)
#> NPClassifier:
#> Pathway level: 14031 candidates (9011 unique)
#> Superclass level: 12739 candidates (8205 unique)
#> Class level: 9638 candidates (5962 unique)
#> [2026-03-26 21:34:55.335] [INFO ] [OK] Completed: decorate_chemo [n_processed=661096] (168ms)
#> [2026-03-26 21:34:57.918] [INFO ] > Starting: clean_chemo [n_annotations=661096, candidates_final=1, high_confidence=FALSE]
#> [2026-03-26 21:35:16.351] [INFO ] Sampling candidates for 3095 features with more than 7 candidates per score
#> [2026-03-26 21:35:16.353] [INFO ] > Starting: filter_high_confidence [n_input=12943, context=filtered]
#> [2026-03-26 21:35:16.370] [INFO ] [filtered] Removed 11713 low-confidence candidates (90.5% of 12943 total)
#> [2026-03-26 21:35:16.371] [INFO ] [filtered] 1230 high-confidence candidates remaining (9.5%)
#> [2026-03-26 21:35:16.373] [INFO ] [OK] Completed: filter_high_confidence [n_filtered=1230, n_removed=11713] (20ms)
#> [2026-03-26 21:35:17.429] [INFO ] Summarizing annotation results
#> [2026-03-26 21:35:17.533] [INFO ] Annotated features: 596/596 (100.0%)
#> [2026-03-26 21:35:33.655] [INFO ] Summarizing annotation results
#> [2026-03-26 21:35:47.897] [INFO ] Annotated features: 4695/5328 (88.1%)
#> [2026-03-26 21:35:49.901] [INFO ] [OK] Completed: clean_chemo [n_final_full=450535, n_final_filtered=1123, n_final_mini=5855, n_features=596] (52s)
#> [2026-03-26 21:35:49.903] [INFO ] [OK] Completed: weight_annotations [n_annotations=NULL] (1m 44s)
#> [2026-03-26 21:35:49.928] [INFO ] Exporting parameters to: data/processed/20260326_213549_example/260326_213549_prepare_params.yaml
#> [2026-03-26 21:35:49.951] [INFO ] Exporting parameters to: data/processed/20260326_213549_example/260326_213549_prepare_params_advanced.yaml
#> [2026-03-26 21:35:49.953] [INFO ] > Starting: export_output [file=data/processed/20260326_213549_example/example_results_mini.tsv, n_rows=5855]
#> [2026-03-26 21:35:49.958] [INFO ] [OK] Completed: export_output [size_bytes=1502422] (5ms)
#> [2026-03-26 21:35:49.960] [INFO ] > Starting: export_output [file=data/processed/20260326_213549_example/example_results_filtered.tsv, n_rows=1123]
#> [2026-03-26 21:35:49.964] [INFO ] [OK] Completed: export_output [size_bytes=1066351] (4ms)
#> [2026-03-26 21:35:49.965] [INFO ] > Starting: export_output [file=data/processed/20260326_213549_example/example_results.tsv, n_rows=450535]
#> [2026-03-26 21:35:50.621] [INFO ] [OK] Completed: export_output [size_bytes=226776826] (656ms)
#> [2026-03-26 21:35:50.623] [INFO ] Results exported: example_results.tsv
#> ✔ ann_wei completed [1m 45s, 227.84 MB]
#> ✔ ended pipeline [13m 52.2s, 138 completed, 0 skipped]
#> There were 14 warnings (use warnings() to see them)3 Performing Taxonomically Informed Metabolite Annotation
Adriano Rutz 
This vignette describes how Taxonomically Informed Metabolite Annotation is performed. If you followed all previous steps successfully, this should be a piece of cake, you deserve it!
The final exported file is formatted in order to be easily imported in Cytoscape to further explore your data!
We hope you enjoyed using TIMA and are pleased to hear from you!
For any remark or suggestion, please fill an issue or feel free to contact us directly.
Reuse
Citation
BibTeX citation:
@online{rutz2026,
author = {Rutz, Adriano},
title = {3 {Performing} {Taxonomically} {Informed} {Metabolite}
{Annotation}},
date = {2026-03-26},
url = {https://taxonomicallyinformedannotation.github.io/tima/vignettes/articles/III-processing.html},
langid = {en}
}
For attribution, please cite this work as:
Rutz, Adriano. 2026. “3 Performing Taxonomically Informed
Metabolite Annotation.” March 26. https://taxonomicallyinformedannotation.github.io/tima/vignettes/articles/III-processing.html.