download#

(s3prl.util.download)

Thread-safe file downloading and cacheing

Authors

  • Leo 2022

  • Cheng Liang 2022

get_dir#

s3prl.util.download.get_dir()[source][source]#

set_dir#

s3prl.util.download.set_dir(d)[source][source]#

download#

s3prl.util.download.download(filepath: Path, url, refresh: bool, new_enough_secs: float = 2.0)[source]#

If refresh is True, check the latest modfieid time of the filepath. If the file is new enough (no older than new_enough_secs), than directly use it. If the file is older than new_enough_secs, than re-download the file. This function is useful when multi-processes are all downloading the same large file

urls_to_filepaths#

s3prl.util.download.urls_to_filepaths(*args, refresh=False, download: bool = True)[source]#

Preprocess the URL specified in *args into local file paths after downloading

Parameters:

URLs (Any number of) –

Returns:

Same number of downloaded file paths