Version

Build the platform

On this page

This is the platform team’s side of Modelplane. You set up the gateway that fronts your models, give the control plane cloud credentials, and register your first GPU cluster: a hardware profile published as an InferenceClass and an InferenceCluster that offers it.

In the next step, the ML team will create a model deployment that schedules against this capacity without knowing which cluster it runs on.

Prerequisites

An AWS account with permissions to create EKS clusters, VPCs, and IAM roles
AWS access key ID and secret access key

A GCP service account JSON key, granted these roles on the project:

Role	Needed for
`roles/container.admin`	the cluster and its node pools
`roles/compute.admin`	the VPC network and subnet
`roles/serviceusage.serviceUsageAdmin`	enabling the APIs the cluster needs
`roles/iam.serviceAccountAdmin`	the node service account
`roles/iam.serviceAccountKeyAdmin`	the node service account’s key
`roles/iam.serviceAccountUser`	attaching that account to the nodes
`roles/resourcemanager.projectIamAdmin`	granting the node account `container.admin`

The last one is worth a look before you hand the key over. Modelplane grants the node service account roles/container.admin, so the credential doing the provisioning has to be able to set project IAM policy.

An Azure account with permissions to create AKS clusters and managed identities
An Azure service principal JSON with clientId, clientSecret, subscriptionId, and tenantId

A Nebius account with permissions to create clusters
A Nebius service account JSON key and your project ID

A Vultr account with access to GPU plans
A Vultr API key

Configure cloud credentials

Give the control plane credentials so it can provision clusters in your cloud account.

Create an AWS credentials file:

[default]
aws_access_key_id = 
aws_secret_access_key =

Create a Kubernetes secret:

kubectl create secret generic aws-creds \
  --from-file=credentials= \
  -n crossplane-system

Apply the ClusterProviderConfig referencing your secret:

# Points the AWS provider at the credentials Secret you created. Named default,
# so InferenceClusters with an EKS source use it without further configuration.
apiVersion: aws.m.upbound.io/v1beta1
kind: ClusterProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: aws-creds
      key: credentials

Create a Kubernetes secret:

kubectl create secret generic gcp-creds \
  --from-file=credentials=.json \
  -n crossplane-system

Apply the ClusterProviderConfig, setting projectID to your GCP project:

# Points the GCP provider at the credentials Secret you created. Named default,
# so InferenceClusters with a GKE source use it without further configuration.
apiVersion: gcp.m.upbound.io/v1beta1
kind: ClusterProviderConfig
metadata:
  name: default
spec:
  projectID: my-gcp-project  # replace with your GCP project
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: gcp-creds
      key: credentials

curl -fsSL /examples/getting-started/clusterproviderconfig-gke.yaml \
  | sed 's/my-gcp-project//' \
  | kubectl apply -f -

Create a Kubernetes secret from your service principal JSON:

kubectl create secret generic azure-credentials \
  --from-file=credentials.json=.json \
  -n crossplane-system

Apply the ClusterProviderConfig referencing your secret:

# Points the Azure providers at the credentials Secret you created. Named
# default, so InferenceClusters with an AKS source use it without further
# configuration. The Secret carries a service principal JSON with
# clientId, clientSecret, subscriptionId, and tenantId.
apiVersion: azure.m.upbound.io/v1beta1
kind: ClusterProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: azure-credentials
      key: credentials.json

Create a Kubernetes secret from your service account JSON:

kubectl create secret generic nebius-credentials \
  --from-file=credentials.json=.json \
  -n crossplane-system

Apply the ClusterProviderConfig, setting projectID to your Nebius project:

# Points the Nebius provider at the credentials Secret you created. Named
# default, so InferenceClusters with a Nebius source use it without further
# configuration - Modelplane also reuses its credentials Secret to
# authenticate to the clusters it provisions.
apiVersion: nebius.m.upbound.io/v1beta1
kind: ClusterProviderConfig
metadata:
  name: default
spec:
  identity:
    type: ServiceAccount
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: nebius-credentials
      key: credentials.json
  projectID: project-e00example

curl -fsSL /examples/getting-started/clusterproviderconfig-nebius.yaml \
  | sed 's/project-e00example//' \
  | kubectl apply -f -

Create a Kubernetes secret from your API key:

kubectl create secret generic vultr-credentials \
  --from-literal=api-key= \
  -n crossplane-system

Apply the ClusterProviderConfig referencing your secret:

# Points the Vultr provider at the API key Secret you created. Named
# default, so InferenceClusters with a Vultr source use it without further
# configuration. The API key only provisions clusters - VKE kubeconfigs
# embed static client certificates, so consumers never need it to reach
# the clusters Modelplane provisions.
apiVersion: vultr.m.upbound.io/v1beta1
kind: ClusterProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: vultr-credentials
      key: api-key

Publish hardware and register the cluster

The InferenceClass describes a hardware profile and how to provision it. The InferenceCluster registers a cluster that offers it. Apply both:

apiVersion: modelplane.ai/v1alpha1
kind: InferenceClass
metadata:
  name: l4-1x-g6
spec:
  description: "EKS g6.xlarge, 1x NVIDIA L4"
  provisioning:
    provider: EKS
    eks:
      instanceType: g6.xlarge
      diskSizeGb: 50
      accelerator:
        type: nvidia-l4
        count: 1
  devices:
  - name: gpu
    claim: DRA
    driver: gpu.nvidia.com
    deviceClassName: gpu.nvidia.com
    count: 1
    attributes:
      architecture: { string: Ada Lovelace }
    capacity:
      memory: { value: "23034Mi" }   # L4's real reported VRAM (not the nominal 24GB)
---
apiVersion: modelplane.ai/v1alpha1
kind: InferenceCluster
metadata:
  name: eks-us-east
  labels:
    modelplane.ai/region: us-east
spec:
  cluster:
    source: EKS
    eks:
      region: us-east-1
  nodePools:
  - name: gpu-l4
    className: l4-1x-g6
    nodeCount: 1
    minNodeCount: 1
    maxNodeCount: 1
    zones:
    - us-east-1b