• Popular Checkpoints like DreamShaper and Realistic Vision, Low Rank Adaptations (LoRAs), and Textual Inversions (navigate to the SD 1.5 template and review the drop downs for a full list of fine tuned options). Note: LoRA weights must sum up to 1.
• Additional image dimensions
• Updated user interface

Added more graceful concurrency handling: when users send more than N concurrent request to an endpoint with N replicas actively running, we will queue all extra requests instead of failing them. This queuing behavior has been activated for selected customers, and will be gradually rolled out over this week and next week. You will temporarily see a new replica spin up while the rollout is occurring on your endpoint.

Updated our Python SDK from 0.1.2 to 0.2.0--it now support both streaming and async inference requests.

Added diarization to our Whisper template endpoint and rectified the list of languages supported. Diarization enables use cases where you'd like to identify the speaker of each segment in a speech recording. You can view the full API spec at https://whisper-demo-kk0powt97tmb.octoai.cloud/docs. Here's an example of how to use the template with diarization:

• Python

  import requests
  import base64
  
  def download_file(url, filename):
      response = requests.get(url)
      if response.status_code == 200:
          with open(filename, "wb") as f:
              f.write(response.content)
          print(f"File downloaded successfully as {filename}.")
      else:
          print(f"Failed to download the file. Status code: {response.status_code}")
  
  
  def make_post_request(filename):
      with open(filename, "rb") as f:
          encoded_audio = base64.b64encode(f.read()).decode("utf-8")
  
      headers = {
          "Content-Type": "application/json"
      }
      data = {
          "audio": encoded_audio,
          "task": "transcribe",
          "diarize": True
      }
  
      response = requests.post("https://whisper-demo-kk0powt97tmb.octoai.cloud/predict", json=data, headers=headers)
  
      if response.status_code == 200:
          # Handle the successful response here
          json_response = response.json()
  
          for seg in json_response["response"]["segments"]:
              print(seg)
  
      else:
          print(f"Request failed with status code: {response.status_code}")
  
  if __name__ == "__main__":
      url = "<YOUR_FILE_HERE>.wav"
      filename = "sample.wav"
  
      download_file(url, filename)
  
      make_post_request(filename)
  

• Tomorrow, June 13th, any existing endpoints will be set to min replicas=0 so that you are not billed for an instance unintentionally left active and running. Be prepared for a cold start before your first inference and reset to min replicas=1 if you prefer to keep the instance warm.
• Every user who logs in during public beta will receive credits for 2 free compute hrs on A100 (or 10+ hrs on A10!) to use in their first two weeks.
• The first 500 users to create a new endpoint will receive credits for 12 free compute hrs on A100 (or 50+ hrs on A10!) to use within their first month.

• Our new Python client (supports synchronous inference). Read more about it here.
• Our HTTP REST API now supports both synchronous and asynchronous calls allowing users to request inference without persisting a connection, poll for status, and retrieve the completed prediction data. This is most effective when managing longer running requests.Read more about it here.