August 30, 2023

  • Added Llama2 70B quickstart template endpoint at: We can also host custom Llama2 LoRAs/ checkpoints for you-- please reach out on Discord if you're interested.
  • Enabled users to upload data via URL in the authoring experience (CLI + Python SDK)
  • Added real-time streaming capabilities to our Whisper audio flow, with a React hook called useWhisper for ease of integration into web/mobile apps. You can learn about how to use this feature here:
  • Changed the domain for all newly created endpoints from to Existing endpoints on octoai.cloudwill still work, but we suggest that you start changing your code to call endpoints from instead of, since we'll also update existing endpoints in about a month.