This document provides a detailed description of every command of the OctoAI CLI. Remember that you can install the CLI by following CLI & SDK Installation.

account

Commands related to your OctoAI account.

account info

List account details for identifying yourself, useful for debugging or reporting your account information.

octoai account info

EMAIL               KEY            NAME           PROVIDER
[email protected]      XXXXXXXXXXXX   First Last     google

build

Build a Docker image from your OctoAI project.

Flags

• -d, --dockerfile <string> Dockerfile overload to use for building; if not provided, one will be generated.
• -g, --generate Only generate a Dockerfile rather than building the image, which is the default behavior.
• -i, --image <IMAGE> Image URI to use for final tagging.
• -s, --service-module <MODULE> The Python module containing the octoai.Service implementation.
• --setup Run Service.setup() at image build time.

check-config

Print the configuration of the CLI, useful for checking your token and configuration settings, or for debugging the behavior of your builds.

completion

octoai completion <SHELL>

Generate the autocompletion script for the specified shell. Supports one of: bash, fish, powershell, zsh. For more information on how to use it, our CLI is built with Cobra.

deploy

Deploy your endpoint to OctoAI. By default, we use your settings from .octoai.yaml, but flags can be passed to on this command to override the file settings.

Flags:

• --concurrency-per-replica <INT> Maximum concurrency per replica (default is -1).
• --description <DESC> Give this endpoint a human readable description.
• -d, --display-name Give this endpoint a human readable display name in the web UI.
• -e, --env [<KEY>=<VALUE>] Environment variables to pass to the endpoint; issue once per var: --env FOO=bar --env BAZ=qux.
• --hardware <HW_TYPE> The hardware instance to deploy to.
• -i, --image <IMAGE> A URI to the Image to deploy.
• --max-replicas <INT> Maximum number of replicas to scale up to (default -1)
• --min-replicas <INT> Minimum number of replicas to scale down to (default -1)
• -n, --name <STRING> Name of the endpoint
• --regcred <REGCRED_KEY> Registry credentials key for private image pulls.
• --scaledown-secs <INT> Number of seconds of inactivity before scaling down the endpoint (default -1).
• --secrets <SECRET_KEY> Secrets to load into the endpoint; issue once per secret: --secrets secretKey1 --secrets secretKey2.
• -t, --tag <TAG> Tag to use for the image.
• --visibility <public|private> Controls whether the endpoint is public or needs authentication.

endpoint

Commands for creating, editing, or updating your endpoints.

endpoint create

Create an endpoint with the provided flags.

Flags:

• --concurrency-per-replica <INT> Maximum concurrency per replica (default is -1).
• --description <DESC> Give this endpoint a human readable description.
• --pause Pause this endpoint on creation.
• -d, --display-name Give this endpoint a human readable display name in the web UI.
• -e, --env [<KEY>=<VALUE>] Environment variables to pass to the endpoint; issue once per var: --env FOO=bar --env BAZ=qux.
• --hardware <HW_TYPE> The hardware instance to deploy to. Options include "gpu.t4.medium" "gpu.a10g.medium" "gpu.a100-80.medium"
• --health-check-path <STRING> Server path for health checks
• -i, --image <IMAGE> A URI to the Image to deploy.
• --max-replicas <INT> Maximum number of replicas to scale up to (default -1)
• --min-replicas <INT> Minimum number of replicas to scale down to (default -1)
• -n, --name <STRING> Name of the endpoint
• -p, --port <INT> Port to route requests to on the container (default 8080)
• --regcred <REGCRED_KEY> Registry credentials key for private image pulls.
• --scaledown-secs <INT> Number of seconds of inactivity before scaling down the endpoint (default -1).
• --secrets <SECRET_KEY> Secrets to load into the endpoint; issue once per secret: --secrets secretKey1 --secrets secretKey2.
• -t, --tag <TAG> Tag to use for the image.
• --visibility <public|private> Controls whether the endpoint is public or needs authentication.

endpoint get

Get information on the endpoint

octoai endpoint get --name <NAME> [-o <OUTPUT_FORMAT>]

endpoint list

List endpoints within this account

octoai endpoint list  [-o <OUTPUT_FORMAT>]

endpoint update

Update an endpoint

endpoint delete

octoai endpoint delete --name <NAME> [--force]

endpoint start

Start the endpoint, scale to minimum replicas and start routing traffic to it.

octoai endpoint start --name <ENDPOINT_NAME>

endpoint pause

Pause the endpoint, scale down the replicas to 0 and stop routing traffic to it.

octoai endpoint pause --name <ENDPOINT_NAME>

events

Show the events associated with a given endpoint.

Flags

• -f, --follow Tail event stream using our streaming events API. You will only events which occur after you start this command.
• -n, --name <string> Name of the endpoint to monitor.
• -o, --output-format OutputFormat Output format (default table).

help

Help about any command, provides a short summary of this page in the CLI.

init

octoai init

Initialize a brand new project from a scaffold, it will prompt the user for new repository configuration. The command will walk you through configuring your endpoint. See below for an example of creating a new project.

login

Cache auth credentials for the OCTOAI_TOKEN generated from How to create an OctoAI API token.

logs

Show the logs for a given endpoint.

Flags

• -f, --follow Tail endpoint logs using our streaming logs API. You will only events which occur after you start this command.
• -n, --name <NAME> Name of the endpoint to monitor.
• -o, --output-format OutputFormat Output format (default table).

public-endpoints

Information on available public endpoints

public-endpoints list

List the set of public endpoints available for use.

octoai public-endpoints list [--output-format OutputFormat]

push

Push an image

octoai push --image <IMAGE> --tag <TAG>

Flags

• -i, --image <IMAGE> The Image URI path to push to.
• -t, --tag <TAG> The local tag of the image to push.

regcred

Manage registry credentials for deploying images from private registries.

regcred create

Creates a registry credential in your account.

octoai regcred create --key <KEY> --username <USER> --password <PASS>

regcred get

Get a registry credential from your account.

octoai regcred get --key <KEY>

regcred list

List your account's registry credentials.

octoai regcred list

regcred update

Update a registry credential in your account.

octoai regcred update --key <KEY> --username <USER> --password <PASS>

regcred delete

Delete a registry credential from your account.

octoai regcred delete --key <KEY>

run

Run the most recently built container locally for testing before deployment.

Flags

• -b, --background Whether to run container in the background.
• --command <CMD> Inference command to run. Surround in quotes to pass arguments to the command.
• -s, --env <stringArray> Environment variables to pass to the container; issue once per var: --env FOO=bar --env BAZ=qux
• --gpus <string> Identifier of GPU device, or 'all' to use all GPUs.
• -i, --image <string> Image to run.
• -p, --port <int> Host port to bind the container port to. (default 8080)
• --timeout int How long to wait for container to start and healthcheck to be ready before timing out. (default 300)
• --container-name <string> Assign a name to the container.

secret

Manage your secrets for your endpoints. These allow you to pass extra API keys, or any secret configuration to your containers. See Setting up account-wide secrets for your custom endpoints for more information about this.

secret create

Create a secret in your account with a given key and a value.

octoai secret create --key <SECRET_KEY> --value <SECRET_VALUE>

secret delete

Delete a secret from your account.

octoai secret delete --key <SECRET_KEY>

secret get

Fetch a secret from your account.

octoai secret delete --key <SECRET_KEY> [--show] [--output-format OutputFormat]

secret list

List the secrets of your account.

octoai secret list [--output-format OutputFormat]

secret update

Update a secret in your account.

octoai secret update --key <SECRET_KEY> --value <SECRET_VALUE>

version

Print the CLI version information useful for checking your build information or reporting issues.