ads.opctl.distributed package#
Subpackages#
- ads.opctl.distributed.common package
- Submodules
- ads.opctl.distributed.common.abstract_cluster_provider module
ClusterProvider
ClusterProvider.DEFAULT_CODE_DIR
ClusterProvider.SYNC_SCRIPT_PATH
ClusterProvider.basic_configuration()
ClusterProvider.check_cluster_status()
ClusterProvider.configuration()
ClusterProvider.create_sync_script()
ClusterProvider.execution_failed()
ClusterProvider.expected_worker_count()
ClusterProvider.export_config_files()
ClusterProvider.export_configuration()
ClusterProvider.fetch_all_worker_info()
ClusterProvider.fetch_code()
ClusterProvider.find_self_ip()
ClusterProvider.get_oci_auth()
ClusterProvider.get_sync_loc()
ClusterProvider.get_sync_script()
ClusterProvider.profile_cmd()
ClusterProvider.reached_endoflife()
ClusterProvider.ready()
ClusterProvider.run_code()
ClusterProvider.setup_configuration()
ClusterProvider.setup_extra_configs()
ClusterProvider.start()
ClusterProvider.start_main()
ClusterProvider.start_ps()
ClusterProvider.start_worker()
ClusterProvider.stop()
ClusterProvider.stop_filesystem
ClusterProvider.sync()
ClusterProvider.tearable()
ClusterProvider.when_ready()
- ads.opctl.distributed.common.abstract_framework_spec_builder module
- ads.opctl.distributed.common.cluster_config_helper module
- ads.opctl.distributed.common.cluster_provider_factory module
- ads.opctl.distributed.common.cluster_runner module
- ads.opctl.distributed.common.framework_factory module
- Module contents
Submodules#
ads.opctl.distributed.certificates module#
ads.opctl.distributed.cli module#
ads.opctl.distributed.cmds module#
- ads.opctl.distributed.cmds.dask_cmd(code_mount, oci_key_mount, config)[source]#
CLI command to run Dask distributed framework on local
- Parameters:
code_mount – source_folder to be mounted
oci_key_mount – oci keys to be mounted
config – Job config
Returns – List: CLI command
------- –
- ads.opctl.distributed.cmds.dev_mode_base_uri(cluster_type, version)[source]#
Temporary method to mock initialization using bucket.
- ads.opctl.distributed.cmds.docker_build_cmd(ini)[source]#
Builds the docker image
- Parameters:
ini – config.ini file dictionary
Returns – None
------- –
- ads.opctl.distributed.cmds.get_cmd(ini)[source]#
Docker build command
- Parameters:
ini – config.ini file dictionary
Returns – Docker build command
------- –
- ads.opctl.distributed.cmds.horovod_cmd(code_mount, oci_key_mount, config)[source]#
CLI command to run horovod distributed framework on local
- Parameters:
code_mount – source_folder to be mounted
oci_key_mount – oci keys to be mounted
config – Job config
Returns – List: CLI command
------- –
- ads.opctl.distributed.cmds.increment_tag(ini)[source]#
Increments the tag of the image
- Parameters:
ini – config.ini file dictionary
Returns – updated ini dict
------- –
- ads.opctl.distributed.cmds.increment_tag_in_ini(ini)[source]#
increments tag of image and update latest in config.ini file
- Parameters:
ini – config.ini file dictionary
Returns – updated config.ini file
------- –
- ads.opctl.distributed.cmds.local_run(config, ini)[source]#
Local run distributed framework on local based on input args
- Parameters:
ini – config.ini file dictionary
config – Job run config
Returns – None
------- –
- ads.opctl.distributed.cmds.pytorch_cmd(code_mount, oci_key_mount, config)[source]#
CLI command to run Pytorch distributed framework on local
- Parameters:
code_mount – source_folder to be mounted
oci_key_mount – oci keys to be mounted
config – Job config
Returns – List: CLI command
------- –
- ads.opctl.distributed.cmds.refresh_ini(ini, ini_file)[source]#
write updated config.ini in local
- Parameters:
ini – config.ini file dictionary
ini_file – Name of the config file
- ads.opctl.distributed.cmds.run_cmd(cmd)[source]#
Run cli command :param cmd: input cli command :param Returns: None :param ——-:
- ads.opctl.distributed.cmds.show_config_info(job_id, work_dir, cluster_file_name, worker_info, **kwargs)[source]#
- ads.opctl.distributed.cmds.tensorflow_cmd(code_mount, oci_key_mount, config)[source]#
CLI command to run Tensorflow distributed framework on local
- Parameters:
code_mount – source_folder to be mounted
oci_key_mount – oci keys to be mounted
config – Job config
Returns – List: CLI command
------- –
- ads.opctl.distributed.cmds.update_config_image(config)[source]#
updates image name in config
- Parameters:
config – Job config
Returns – updated config dict
------- –
- ads.opctl.distributed.cmds.update_image(config, ini)[source]#
Update Image name in job config with latest build image
- Parameters:
config – Job run config
ini – config.ini file dictionary
Returns – Updated Job run config
------- –
- ads.opctl.distributed.cmds.update_ini(tag, registry, dockerfile, source_folder, config, nobuild)[source]#
Stores and updates input args in config.ini in Local
- Parameters:
tag – tag of image
registry – registry to push to
dockerfile – relative path to Dockerfile
source_folder – source scripts folder that will be mounted during a local run
config – Job config
nobuild – flag for building the docker
Returns – dict of config.ini file
------- –