datalad_next.patches.replace_ora_remote
Patch datalad.distributed.ora_remote.ORARemote
This patch replaces the class datalad.distributed.ora_remote.ORARemote
with an updated version that should work properly on Linux, OSX, and Windows.
The main difference to the original code is that all path-operations are
performed on URL-paths. Those are represented by instances of PurePosixPath.
All subclasses of BaseIO
, i.e. LocalIO
, SSHRemoteIO
,
and HTTPRemoteIO
, are extended to contain the method
url2transport_path()
. This method converts an URL-path into the correct
path for the transport, i.e. the IO abstraction.
Before methods on a subclass of BaseIO
that require a path are called,
the generic URL-path is converted into the correct path for the IO-class by
calling url2transport_path()
on the respective IO-class.
The patch keeps changes to the necessary minimum. That means the source is mostly identical to the original. Besides the changes described above, more debug output was added.
NOTE: this patch only provides ORARemote
. The patches that add a
url2transport_path()
-method to LocalIO
and to HTTPRemoteIO
are contained in module datalad_next.patches.add_method_url2localpath
. The
reason to keep them separate is that the patch from module
datalad_next.patches.replace_create_sibling_ria
require them as well.
For SSHRemoteIO
the method is included in the patch definition of
SSHRemoteIO
, which is contained in the module
datalad_next.patches.replace_sshremoteio
.
- class datalad_next.patches.replace_ora_remote.ORARemote(annex)[source]
Bases:
SpecialRemote
This is the class of RIA remotes.
- checkpresent(key)[source]
Requests the remote to check if a key is present in it.
- Parameters:
key (str)
- Returns:
True if the key is present in the remote. False if the key is not present.
- Return type:
bool
- Raises:
RemoteError -- If the presence of the key couldn't be determined, eg. in case of connection error.
- dataset_tree_version = '1'
- get_store()[source]
checks the remote end for an existing store and dataset
Furthermore reads and stores version and config flags, layout locations, etc. If this doesn't raise, the remote end should be fine to work with.
- getcost()[source]
Requests the remote to return a use cost. Higher costs are more expensive.
cheapRemoteCost = 100 nearlyCheapRemoteCost = 110 semiExpensiveRemoteCost = 175 expensiveRemoteCost = 200 veryExpensiveRemoteCost = 1000 (taken from Config/Cost.hs)
- Returns:
Indicates the cost of the remote.
- Return type:
int
- initremote()[source]
Gets called when git annex initremote or git annex enableremote are run. This is where any one-time setup tasks can be done, for example creating the remote folder. Note: This may be run repeatedly over time, as a remote is initialized in different repositories, or as the configuration of a remote is changed. So any one-time setup tasks should be done idempotently.
- Raises:
RemoteError -- If the remote could not be initialized.
- property io
- known_versions_dst = ['1']
- known_versions_objt = ['1', '2']
- object_tree_version = '2'
- prepare()[source]
Tells the remote that it's time to prepare itself to be used. Gets called whenever git annex is about to access any of the below methods, so it shouldn't be too expensive. Otherwise it will slow down operations like git annex whereis or git annex info.
Internet connection can be established here, though it's recommended to defer this until it's actually needed.
- Raises:
RemoteError -- If the remote could not be prepared.
- property push_io
- remove(key)[source]
Requests the remote to remove a key's contents.
- Parameters:
key (str)
- Raises:
RemoteError -- If the key couldn't be deleted from the remote.
- transfer_retrieve(key, filename)[source]
Get the file identified by key from the remote and store it in local_file.
While the transfer is running, the remote can repeatedly call annex.progress(size) to indicate the number of bytes already stored. This will influence the progress shown to the user.
- Parameters:
key (str) -- The Key to get from the remote.
local_file (str) -- Path where to store the file. Note that in some cases, local_file may contain whitespace.
- Raises:
RemoteError -- If the file could not be received from the remote.
- transfer_store(key, filename)[source]
Store the file in local_file to a unique location derived from key.
It's important that, while a Key is being stored, checkpresent(key) not indicate it's present until all the data has been transferred. While the transfer is running, the remote can repeatedly call annex.progress(size) to indicate the number of bytes already stored. This will influence the progress shown to the user.
- Parameters:
key (str) -- The Key to be stored in the remote. In most cases, this is going to be the remote file name. It should be at least be unambiguously derived from it.
local_file (str) -- Path to the file to upload. Note that in some cases, local_file may contain whitespace. Note that local_file should not influence the filename used on the remote.
- Raises:
RemoteError -- If the file could not be stored to the remote.
- verify_ds_in_store()[source]
Check whether the dataset exists in store and reports a layout version we know
The layout is recorded in 'dataset_somewhere_beneath_base_path/ria-layout-version.' If the version found on the remote end isn't supported and force-write isn't configured, sets the remote to read-only operation.
- verify_store()[source]
Check whether the store exists and reports a layout version we know
The layout of the store is recorded in base_path/ria-layout-version. If the version found on the remote end isn't supported and force-write isn't configured, sets the remote to read-only operation.
- whereis(key)[source]
Asks the remote to provide additional information about ways to access the content of a key stored in it, such as eg, public urls. This will be displayed to the user by eg, git annex whereis. Note that users expect git annex whereis to run fast, without eg, network access.
- Parameters:
key (str)
- Returns:
Information about the location of the key, eg. public urls.
- Return type:
str