Skip to main content

Get CUDA Information

This gives information about your cuda usage.

*Note: This is only available when Marqo is running in COMBINED model. If you are running Marqo with separate API and Inference containers, this endpoint will return incorrect information.

GET /device/cuda

Example​

CUDA​

=== "Marqo Open Source" === "cURL"

curl -XGET http://localhost:8882/device/cuda

=== "Python"

results = mq.index("my-index").get_cuda_info()

=== "Marqo Cloud" For Marqo Cloud, you will need to access the endpoint of your index and replace your_endpoint with this. To do this, visit Find Your Endpoint. You will also need your API Key. To obtain this key visit Find Your API Key. === "cURL"

curl -XGET your_endpoint/device/cuda \
-H 'x-api-key: XXXXXXXXXXXXXXX'

=== "Python"

results = mq.index("my-index").get_cuda_info()

Response: 200 OK​

{
   "cuda_devices": [
     {
       "device_id": 0,
       "device_name": "Tesla T4",
       "memory_used": "1.7 GiB",
       "total_memory": "14.6 GiB",
       "utilization": "11.0 %",
       "memory_used_percent": "25.0 %"
     }
   ]
}