skyplane.api.dataplane#

Classes

Dataplane(clientid, topology, provisioner, ...)

A Dataplane represents a concrete Skyplane network, including topology and VMs.

DataplaneAutoDeprovision(dataplane)

class skyplane.api.dataplane.Dataplane(clientid, topology, provisioner, transfer_config, debug=False)[source]#

Bases: object

A Dataplane represents a concrete Skyplane network, including topology and VMs.

Parameters:

  • clientid (str) – the uuid of the local host to create the dataplane

  • topology (ReplicationTopology) – the calculated topology during the transfer

  • provisioner (Provisioner) – the provisioner to launch the VMs

  • transfer_config (TransferConfig) – the configuration during the transfer

  • debug (bool, optional) – whether to enable debug mode, defaults to False

auto_deprovision()[source]#

Returns a context manager that will automatically call deprovision upon exit.

Return type:

DataplaneAutoDeprovision

check_error_logs()[source]#

Get the error log from remote gateways if there is any error.

Return type:

Dict[str, List[str]]

deprovision(max_jobs=64, spinner=False)[source]#

Deprovision the remote gateways

Parameters:

  • max_jobs (int) – maximum number of jobs to deprovision the remote gateways (default: 64)

  • spinner (bool) – Whether to show the spinner during the job (default: False)

estimate_total_cost()[source]#

Estimate total cost of queued jobs

provision(allow_firewall=True, gateway_docker_image='public.ecr.aws/s6m1p0n8/skyplane:edge', gateway_log_dir=None, authorize_ssh_pub_key=None, max_jobs=16, spinner=False)[source]#

Provision the transfer gateways.

Parameters:

  • allow_firewall (bool) – whether to apply firewall rules in the gatweway network (default: True)

  • gateway_docker_image (str) – Docker image token in github

  • gateway_log_dir (PathLike) – path to the log directory in the remote gatweways

  • authorize_ssh_pub_key (str) – authorization ssh key to the remote gateways

  • max_jobs (int) – maximum number of provision jobs to launch concurrently (default: 16)

  • spinner (bool) – whether to show the spinner during the job (default: False)

queue_copy(src, dst, recursive=False)[source]#

Add a copy job to job list. Return the uuid of the job.

Parameters:

  • src (str) – source prefix to copy from

  • dst (str) – the destination of the transfer

  • recursive (bool) – if true, will copy objects at folder prefix recursively (default: False)

Return type:

str

queue_sync(src, dst)[source]#

Add a sync job to job list. Return the uuid of the job.

Parameters:

  • src (str) – Source prefix to copy from

  • dst (str) – The destination of the transfer

  • recursive (bool) – If true, will copy objects at folder prefix recursively (default: False)

Return type:

str

run(hooks=None)[source]#

Start the transfer in the main thread. Wait until the transfer is complete.

Parameters:

hooks (TransferHook) – Tracks the status of the transfer

run_async(hooks=None)[source]#

Start the transfer asynchronously. The main thread will not be blocked.

Parameters:

hooks (TransferHook) – Tracks the status of the transfer

Return type:

TransferProgressTracker

sink_gateways()[source]#

Returns a list of sink gateway nodes

Return type:

List[Server]

source_gateways()[source]#

Returns a list of source gateway nodes

Return type:

List[Server]