- class eradiate.data.SafeOnlineDataStore(base_url, path, registry_fname='registry.txt')#
Serve files located online, with integrity check.
This class basically wraps a
- fetch(filename, downloader=None)#
Fetch a file from the data store. This method wraps
pooch.Pooch.fetch()and automatically selects compressed files when they are available.
Path) – Absolute path where the retrieved resource is located.
If a compressed resource exists, it will be served automatically. For instance, if
"foo.nc"is requested and
foo.nc.gzis registered, the latter will be downloaded, decompressed and served as
- is_registered(filename, allow_compressed=True)#
Check if a file is registered, with an option to look for compressed data.
Path) – The file name which matched filename.
ValueError – If filename could not be matched with any entry in the registry.
Purge local storage location. The default behaviour is very aggressive and will wipe out the entire directory contents.
str, optional) – If set to
"registered", files in the registry, as well as the registry file itself, will not be deleted. Finer control is possible by passing a list of exclusion rules (paths relative to the store’s local storage root, shell wildcards allowed).
keep="registered"keeps registered files to minimise the amount of data to be downloaded upon future queries the the data store. This means, for instance, that if data is registered and downloaded as a compressed file, then served decompressed, the compressed file will be kept, while the decompressed file will be deleted.
This is a destructive operation, make sure you know what you’re doing!
Delete the registry file.
Get the absolute path to the registry file and make sure that it is written to the local cache.
Get a list of registered files.
Reload the registry file from the local cache.
bool, optional) – If
True, the existing registry file will be deleted and downloaded again.
- property path#
Absolute path to the local data storage folder.
- property registry_path#
Absolute path to the registry file.