DEV Community: Kupe Cloud

Private container images, connected to your Kupe clusters | Kupe Cloud

Billy Walker — Mon, 26 Oct 2026 12:51:16 +0000

A private image needs somewhere to live and a way for your cluster to pull it. The storage is usually straightforward. Access has a habit of spreading: a credential in CI, a pull Secret in one namespace, another copy when a service moves, and a rotation to remember later. Your existing registry may automate some of that already. We wanted Kupe to take care of it for the clusters it manages.

Kupe now includes an opt-in private container registry for each account. You push images with your usual tools, connect the clusters that need them, and deploy. Connected clusters can pull your images without an imagePullSecrets entry in your manifests. Kupe manages the pull credentials.

The registry belongs to your account

Your images sit alongside the clusters and managed services. Each account can turn on one registry, and you choose which of its clusters to connect. You can use the same image across development and production without setting up a separate registry for each cluster.

The registry has its own lifecycle. Deleting a cluster removes that cluster's connection; it doesn't delete the images. When you create its replacement, you can connect it to the same registry. That keeps an application build available independently of any one cluster.

Access follows your Kupe account too. Members sign in with their existing identity, and CI uses access tokens with pull-only or push-and-pull access. Removing a member revokes the sessions and tokens they created. There is no separate set of registry users to keep in sync when someone joins or leaves.

Kupe operates the registry for you, with no separate installation to maintain or account to administer. The console, CLI, API and Terraform provider all work with the same registry and cluster connections.

Connect once, pull across the cluster

A connection gives a cluster access to the account's images. Once it's ready, new pods can pull from the registry in any namespace. Creating another namespace doesn't give you another pull Secret to distribute, and rotating the connection's credentials doesn't require changes to your application manifests.

The pull credential stays on the host, outside your tenant cluster. Workloads and people with access to that cluster cannot read or copy it. That was a deliberate choice: a cluster needs permission to fetch an image, but the application running from that image doesn't need the credential itself.

There is a tradeoff here. A connection grants pull access to every repository in the account's registry, across the whole cluster. This first version has no per-namespace pull restrictions. If two teams must not be able to pull each other's images, they need separate accounts or a registry setup with that finer boundary. Namespaces alone don't provide it.

Builds running inside a cluster can also receive push access in selected namespaces. That uses a Secret inside those namespaces, so it needs more care: workloads able to use it can push to the registry, including replacing existing tags. We keep that permission separate from the cluster's ordinary pull access.

Keep the build tools you use

The registry works with standard OCI clients and stores container images, multi-platform images, Helm charts and signatures. An existing pipeline can push to Kupe using an access token; you don't need to adopt a Kupe-specific build system.

For a team managing infrastructure in Git, Terraform can declare the registry and its cluster connections alongside the clusters themselves. If you prefer the console or CLI, those expose the same choices. The useful outcome is the same: the image is stored in your account, and the clusters you've connected can run it.

You can browse repositories, images and tags in the console. Storage use, push and pull activity, access tokens and cluster connections also appear in the Registry dashboard in Grafana. Alerts cover storage filling up, expiring tokens and connections that haven't become ready. Registry operations belong with the rest of the platform's monitoring; you shouldn't need another monitoring setup just to see whether your image store is filling up.

What this first version leaves to you

The registry handles image storage and access today. A few limits are worth knowing before you rely on it:

It doesn't scan stored images yet. Kupe's workload vulnerability reports cover images deployed to clusters.
Tags are mutable, and there are no automatic retention rules. You decide what to keep and delete.
The registry isn't backed up, and deleted images cannot be restored. Keep the source and build pipeline needed to publish them again.

Deletion protection is on by default. You can set a storage limit within your plan's allowance; reaching it stops new pushes while existing images remain available to pull.

Scanning on push, per-image SBOMs, retention rules and immutable tags are on the roadmap. They're planned additions, so choose the registry based on what it does today.

To try it, enable Registry in the console and connect one of your clusters. The registry guide covers pushing your first image and setting up CI, with Terraform examples if that's how you manage your account. If the path leaves you managing a pull Secret by hand, we'd like to hear where: support@kupe.cloud.

Managed PostgreSQL for Kubernetes apps | Kupe Cloud

Billy Walker — Wed, 21 Oct 2026 13:05:09 +0000

Your app needs Postgres. The familiar Kubernetes route is to install a database operator, choose its size, arrange TLS, put credentials somewhere the app can reach, and keep the whole arrangement upgraded. That is real work, and we would rather not make every team repeat it on every cluster. It also means the database competes with your app for the cluster's CPU and memory, and you secure it on your own.

Kupe now provides managed PostgreSQL databases. You create a database for your account, then bind a role into the namespace where an app runs. The namespace receives a Service and a credentials Secret; the app mounts the Secret and uses the supplied connection URI. We built the feature around that last step. Creating a database is the easy half. The harder half is getting credentials to the right workload without pasting them into a manifest.

A database that outlives a cluster

A managed database belongs to your account rather than to one cluster. A cluster can be deleted and recreated, and your data should not go with it.

You choose CPU, memory and storage for an instance, with PostgreSQL 18 as the default engine version and PostgreSQL 17 also available. High availability adds a second instance with a synchronous standby. Storage can grow, but it cannot shrink; a database keeps its PostgreSQL major version, while minor updates are applied automatically.

There are a few guardrails worth knowing before you create one:

Deletion protection starts on. Turn it off before deleting a database.
Deleted storage is retained for seven days for support-assisted recovery. There is no self-service restore in v1.
A database can alert you when storage is running low, when one instance is down while another is serving, or when no instance is up. Those alerts sit with cluster alerts in the console.

Bind the database where the app runs

A binding joins one database role to one namespace in one cluster. It is the contract that tells Kupe where to deliver access. The cluster does not need to exist when you create the binding: if it is deleted, the binding waits and delivers again when a cluster with the same name returns.

In each bound namespace, Kupe creates a Service named after the database and a credentials Secret. The default Secret name is {database}-{role}. It follows the servicebinding.io key convention and includes host, port, database, username, password, sslmode, ca.crt, uri, type and provider.

The useful bit is the uri. It is already configured for TLS certificate verification with sslmode=verify-full, using the database's own CA. Connections without TLS are refused. Mount the Secret as a volume at /bindings/<secret-name>, for example /bindings/orders-db-readwrite, and set SERVICE_BINDING_ROOT=/bindings. The uri only works at that path, because its CA setting points at the mounted ca.crt. The database guide shows the workload configuration. Your application can then consume the binding without pasting a password or CA bundle into a Deployment manifest.

The Secret tracks the database. When a password rotates or the CA is renewed, every bound Secret receives the new value. Kubernetes updates mounted Secret files in place; an application that reads its password once at startup needs a restart to pick up the change.

Network access follows the same binding. Pods in the bound namespace of the bound cluster can connect; there is no public endpoint. Remove the binding and Kupe removes the Service, Secret and network path. That does not rotate the role password. If access may already have escaped the namespace, rotate the role credentials as well.

Use roles to separate migrations from the service

Managed PostgreSQL v1 has two roles:

owner can change the schema. It belongs with migrations.
readwrite can read and write the owner's tables but cannot change their schema. It belongs with the running service.

Put owner in orders-migrate and readwrite in orders-api, and the API namespace has no schema-changing credential to mount. That is the reason bindings are per namespace rather than a single database Secret copied around a cluster.

From the CLI, the database and both bindings take three commands:

kupe database create orders-db --cpu 250m --memory 512Mi --storage 5Gi
kupe database binding add orders-db --cluster dev --namespace orders-migrate --role owner
kupe database binding add orders-db --cluster dev --namespace orders-api --role readwrite

create and binding add wait until the requested object is ready. The CLI reference also covers resizing, high availability, deletion protection, credential rotation and waiting in scripts.

The same binding in the console, CLI or Terraform

The console, CLI and Terraform provider create the same resource. Choose the one that fits how you already manage infrastructure.

In the console, open a database and choose Add Binding, then select the cluster, namespace and role. A binding row shows Ready once its Service and Secret are in place.

For infrastructure managed in pull requests, the Terraform provider has kupe_database, kupe_database_binding and a kupe_database data source. Here is the binding portion of a database definition:

resource "kupe_database_binding" "migrations" {
  database   = kupe_database.orders.name
  cluster    = kupe_cluster.prod.name
  namespaces = ["orders-migrate"]
  role       = "owner"
}

resource "kupe_database_binding" "api" {
  database   = kupe_database.orders.name
  cluster    = kupe_cluster.prod.name
  namespaces = ["orders-api"]
  role       = "readwrite"
}

The provider rejects v1 changes that cannot work, including shrinking storage or changing the PostgreSQL major version. It also respects deletion protection. If a namespace already contains a same-named Service or Secret that Kupe did not create, the binding remains failed rather than overwriting somebody else's object. The binding stays failed until you remove or rename your object, which beats an apply that silently replaces a Secret your workload depends on.

The public managed-database sample puts this shape to use: a migration Job uses the owner role, a Go API uses readwrite, and its frontend has no database binding at all.

The edges of v1

This first version has clear limits. It is PostgreSQL only, supports one PostgreSQL database per instance, has no custom roles or extensions, no built-in connection pooling, no backups or self-service restore, and no in-place major version upgrades.

We are working towards backups with point-in-time restore, more control over roles and in-place major upgrades. Those changes are additive: an application built on a v1 binding will not need a different connection model when they arrive.

Start with a small database and one namespace. Bind readwrite, mount the Secret, and connect using its uri. The get-started guide has the full path; if you run into something rough while trying it, email support@kupe.cloud.

Create a Kubernetes cluster from the command line | Kupe Cloud

Billy Walker — Tue, 13 Oct 2026 17:02:19 +0000

Getting a Kubernetes cluster shouldn't involve a wizard, a downloaded file and a kubeconfig you'll have lost by Friday. With the Kupe CLI it's four commands, and the last one is kubectl:

kupe auth login
kupe cluster create dev --cpu-limit 2 --memory-limit 8Gi --storage-limit 50Gi
kupe cluster kubeconfig dev --merge --exec
kubectl get pods

This post walks through what each of those does, why we built the CLI the way we did, and how the same commands hold up when a CI job runs them with nobody watching. If you already use kubectl, most of it should feel familiar. That was the plan.

Install and sign in

You'll need a Kupe Cloud account first, and sign-up is open during the alpha.

The quickest install is Homebrew (brew install kupecloud/tap/kupe) or the install script (curl -fsSL https://get.kupe.cloud | sh). The CLI docs cover Windows and every other route.

kupe auth login uses the OAuth device flow, the same pattern a smart TV uses when it shows you a code and tells you to finish on your phone. The CLI prints a short code and a URL, you approve in any browser, and the CLI collects the tokens. Your password never passes through it, and because there's no local web server involved it behaves the same over SSH or in a dev container. Tokens live in your operating system's keyring where one exists, and refresh themselves on later commands.

Create a cluster

kupe cluster create dev --cpu-limit 2 --memory-limit 8Gi --storage-limit 50Gi

The three limits have no defaults. They cap what workloads in the cluster can use and count against your plan's pool; kupe plan list shows what your plan allows.

Underneath, cluster creation is asynchronous. The API accepts the request straight away and the cluster moves through phases (pending, provisioning, running) while the platform builds it. The CLI watches that phase and only returns once the cluster is running, so the next line of a script can assume the cluster exists. If you'd rather have your prompt back immediately, pass --wait=false and come back later with kupe cluster wait dev --for running.

Connect kubectl

kupe cluster kubeconfig dev --merge --exec

Without --merge, the command prints a kubeconfig to stdout for you to redirect into a file. With it, the cluster is added to the kubeconfig you already use, under a context named kupe-<tenant>-<cluster>, and becomes your current context, so there's no second file to keep track of.

The --exec flag changes what goes in that kubeconfig. By default it embeds your token, which makes the file a secret and means it stops working when the token does. With --exec there is no token in the file at all. Instead, the file tells kubectl to run kupe auth get-token whenever it needs a credential, using Kubernetes' standard exec credential plugin mechanism (hence the flag name). The AWS, Google and Azure CLIs connect kubectl to their managed clusters the same way.

The trade-off is that kubectl now needs kupe on its PATH. That suits a laptop. For a throwaway container where you can't count on the binary being there, leave --exec off and treat the kubeconfig as the credential it is.

Why it looks like kubectl (mostly)

Our design notes start from two rules. The first is that a developer at a terminal and a CI job run the same commands, and the binary adapts to which one it is. Spinners, colour and confirmation prompts only appear on a terminal. What the CLI writes to stdout is the same either way, with progress and errors kept on stderr, so kupe cluster list -o json | jq never finds a spinner in its input.

The second rule is that we shouldn't surprise kubectl users, since you're probably switching between the two all day. So -o takes table, wide, json, yaml, name and go-template=..., and exit codes and error shapes follow kubectl wherever it has a convention.

We deliberately broke from kubectl in a few places, and each time there was an alternative we weighed:

Noun first. kubectl puts the verb first (kubectl get pods) because it's a generic client for dozens of resource kinds that all share the same verbs. Kupe has a handful of nouns and you'll type kupe cluster <tab> far more often than kupe <verb>, so kupe cluster create gives completion a smaller list to offer. gh, fly and hcloud read the same way.
-o json, not --json. gh and fly use a --json flag. We kept kubectl's -o because yaml, name, wide and go-template all belong under one umbrella.
Our own printer. We looked at reusing the printing library from Kubernetes itself, but it expects Kubernetes objects. Our resources are plain structs, so we wrote a small printer instead.

Contexts keep you honest

kupe auth login --tenant acme-staging --context staging
kupe config use-context staging
kupe config current-context

kupe config view shows the active configuration with the tokens redacted, which makes it safe to paste into a support request. Every command also takes --tenant and --context (or KUPE_TENANT and KUPE_CONTEXT), which is what you'll reach for in scripts.

Running it in CI

A pipeline can't open a browser, so it signs in with an API key instead. Mint one scoped to the tenant, give it an expiry, and store the output as a CI secret:

kupe apikey create --name ci --role admin --expires-at 90d

The raw key is printed once and can't be retrieved again. Pass it to the CLI as KUPE_API_TOKEN; the --token flag also works but leaves the value in shell history and process listings, which is why the help text itself warns against it.

Here is a job that creates a cluster, deploys to it, and always cleans up:

#!/usr/bin/env bash
set -euo pipefail

export KUPE_API_TOKEN="$CI_KUPE_TOKEN"
export KUPE_TENANT="acme"

cluster="ci-${GITHUB_SHA:0:7}"
trap 'kupe cluster delete "$cluster" --yes' EXIT

kupe cluster create "$cluster" \
  --cpu-limit 2 --memory-limit 8Gi --storage-limit 50Gi \
  --wait-timeout 10m

kupe cluster kubeconfig "$cluster" --merge --force
kubectl --context "kupe-${KUPE_TENANT}-${cluster}" apply -f example.yaml

Three details do the work here. create waits by default, so there's no sleep to guess at. Outside a terminal, delete refuses to run without --yes, so a stray redirect can't delete infrastructure. And delete waits as well, so a script that recreates the same name straight away isn't racing the old cluster on its way out. We learned that one from our own tests, when a destroy-and-recreate collided with the previous cluster's teardown. The platform copes with that now, but waiting is still the safer default.

The exit codes are part of the contract too, and they're stable across releases. 3 means authentication failed, 4 means not found, 5 means conflict, and 8 means a --wait-timeout ran out. That lets a script tell a bad token from a slow cluster without parsing error text. The full table is in the repository.

We rely on that contract ourselves. The CLI's live test suite builds the real binary and runs it against the deployed API: whoami, tenant and member commands, a bad token failing without leaking into the output, and an opt-in create-and-delete of a real cluster. If a command's output or exit code changes, that's where we find out.

-o name helps with housekeeping too. A cluster per commit counts against your plan's cluster limit, so this clears out any CI clusters a cancelled job left behind:

kupe cluster list -o name | grep '^ci-' | xargs -n1 kupe cluster delete --yes

Where to start

The CLI is imperative, which makes it good for trying things out and for the glue between CI steps, and a poor record of what exists. If you want clusters reviewable in a pull request, use the Terraform provider. If you're writing a service, call the API. If your team needs a shared view, use the console.

For a first run, create a throwaway cluster, merge its kubeconfig, run a few kubectl commands, then delete it and watch the delete wait for you. If you hit any issues while experimenting with the CLI, get in touch and we'll do our best to help.

Kupe Cloud alpha is open

Billy Walker — Wed, 07 Oct 2026 18:33:28 +0000

Signup for Kupe Cloud is open. There is no invite code, no card and no charge during alpha.

Alpha is where we find out whether Kupe does what we built it to do: give a team a cluster with GitOps, secrets, metrics, logs and alerting already wired, so they can get on with shipping. We kept it closed for the first two months. The core paths went to a small group of people who already run Kubernetes for a living, because they notice when something is off and can say why. We spent the same two months trying to break the platform ourselves, on production, after every change.

This post is about what that found, and what changed before we opened signup.

We became the first tenant

The end-to-end suite has sixteen phases and a full run takes just under forty minutes. It goes through the public signup service rather than an admin shortcut, creates the cluster through the public API and deploys through Argo CD inside that cluster. Only the residue sweep afterwards runs with admin access, because somebody has to check the cupboard after the party. Everything on the customer path runs as a customer would.

We ran three load profiles alongside it: steady traffic at a small demo app, a workload that outgrows its cluster, and concurrent tenant onboarding through the real signup route. The traffic run handled 298,708 requests over fifteen minutes, with a p95 of 140 ms and no failed requests through the external gateway and TLS. That result came from a tiny demo app on the smallest cluster size. It says the path holds under that test; it does not predict your application's latency.

The cluster-growth tests answered a different question: does the platform add capacity while the workload is still asking for it? In one run, the HPA moved from two to thirteen replicas and the platform added a node under load. A separate scale-out run onboarded two tenants concurrently and left no residue. These are small runs on purpose. They gave us enough surface area to find the awkward parts before asking strangers to do it for us.

The things that broke were useful

Closed alpha earned its keep here. A routine operating-system update brought one node back without networking because a component of the cluster network could not start on the updated kernel. Our reboot tooling could not run on that node either, so it retained the cluster-wide reboot lock.

The first fix looked sensible and changed nothing. It turned off in-place updates on the nodes our provisioning tooling manages, and the workers that needed it are created by the autoscaler, which that tooling never touches. We stopped patching nodes in place. Nodes are now replaced from validated images instead. A follow-up audit found an image-rebuild job that had been reporting success while building nothing. Green is a poor signal when the successful path is to skip every input.

Capacity is the other ordinary failure. Our infrastructure provider does not always have a server available when the autoscaler asks for one. The autoscaler now tries the next size up, alerting includes the pool and the reason, and there is a runbook for the operator who gets the alert. With an empty pool, the first cluster waits for a node to arrive.

A few failures only appear when two tenants do the same thing at once. Two clusters with the same name collided on a host-side role name. A concurrent teardown could also lose the record that told the platform to purge a tenant's data. The scale-out driver found both, and deletion now runs through the same endpoints customers use. You can read the public deletion documentation before trying it yourself.

Alerting needed the same scrutiny. Six chronic alerts were re-posting the same unactionable backlog every four hours. They were responsible for most of the noise, rather than the resource alerts we initially suspected. Fixing that reduced alert volume by 92%. Backlog work and incidents now travel through separate alert paths; an alert in the incident channel should be worth somebody's sleep.

We also made isolation a routine assertion instead of a promise in an architecture diagram. Every end-to-end run creates a second real tenant and checks API access, metrics, logs, dashboards, GitOps destinations, host RBAC and the network boundary. The test that proves one tenant cannot read another's metrics runs before each change ships.

What changed in the platform

The image pipeline builds on a cadence, validates the result, canaries a node, then rolls control planes before core nodes. Promotion runs through a staging estate with full end-to-end and load tests before production. That architecture exists to keep an update from becoming a cluster-wide leap of faith. A cluster without high availability can still see a brief API interruption while a node is replaced; choose HA when you create the cluster if that interruption is not acceptable. The high-availability documentation explains that choice.

Signup is open with one tenant per account for now, plus a cap on total tenants that we raise as we go. The point is containment: an ordinary bad day should stay small. Tenant and account deletion are self-service from the console or CLI, and the suite checks for residue after every deletion.

Releases are built, signed and pinned by digest before they are versioned. The cluster will not run an unsigned image, and baseline policies are enforced. The property that matters is that an untrusted image never becomes a running workload.

What is still rough

Alpha is still alpha. Dedicated nodes and backup tooling do not exist yet, there is no SLA, and maintenance can interrupt a single-replica cluster. The alpha service notice has the current limits and the parts of the service that remain experimental.

Start with the first fifteen minutes

Create an account, create a cluster and deploy something small. It takes no invite code and nothing to pay. Then tell us where the path felt heavier than you expected, especially in the first fifteen minutes: a label you expected to find, a default you expected to be set, or a step that made you stop and ask why it existed. That is exactly the feedback alpha is for. Send it to support@kupe.cloud, and check status.kupe.cloud before treating an odd result as your fault.

vCluster custom resource sync, one plugin for every kind | Kupe Cloud

Billy Walker — Mon, 05 Oct 2026 12:12:52 +0000

vCluster runs a full Kubernetes control plane as a pod inside a namespace of a host cluster. We use it to run our tenants' clusters: each one gets cluster-admin inside a real API boundary, without the cost of a dedicated cluster underneath. Anything self-contained inside the virtual cluster works out of the box: Deployments, Services, the operators a tenant installs. The trouble starts when a tenant writes an Argo CD Application that only the central Argo CD on the host can reconcile, an HTTPRoute that needs to attach to a shared Gateway, or an alert rule that belongs on the host ruler. Every multi-tenant platform grows these crossings: objects written on one side of a boundary that a controller on the other side has to act on.

The options for syncing resources across that boundary are thinner than you'd expect. vCluster's open-source syncer handles the core resources well: Pods, Services, Secrets and friends. But the resources we most needed to sync were custom resources, and some of them needed references translated on the way across. vCluster does offer custom-resource syncing and sync patches, but they ship in the Pro image, behind a licence validated against vCluster Platform. That's a reasonable product boundary; it just wasn't a dependency we wanted in the boot and data path of every tenant cluster.

The open-source routes weren't in better shape. Loft's own generic CRD sync plugin pointed in the right direction but has been archived, which leaves the vCluster plugin SDK and a controller of your own.

So that's what we built: an open vCluster generic sync plugin, Apache-2.0, a single plugin process that syncs any resource kind you declare in its configuration. It carries the pieces a real platform needs: a direction and mode per kind, reference translation, label and namespace selectors, and status sync. Every crossing is explicit, reviewable and independent of a licence service.

Resources cross the boundary for different reasons

It's worth naming the actual objects rather than filing them all under “multi-tenant integration”. Each crossing has an owner, a direction and a reason:

Some objects leave the tenant cluster for a host controller: an Application for the central Argo CD, a PrometheusRule for the host Mimir ruler.
Some go the same way but need their status back: an HTTPRoute attaches to the host Gateway, and the tenant needs to see whether it was accepted.
Some only cross on a condition: a dashboard ConfigMap leaves only when it carries the dashboard label.
Some travel the other way: platform-managed Secret objects come back into the tenant cluster.
Some are read-only imports: GatewayClass objects are mirrored so tenants can discover the shared gateway classes.

Those aren't interchangeable operations. A route needs its service references translated; a gateway class must remain host-owned; a repository credential should be admitted only by a narrow label selector. A single broad “sync CRDs” switch would hide all of that policy in code and RBAC.

Our plugin makes the policy a list of syncResources entries. Each entry names an apiVersion, kind, a direction (toHost or fromHost) and an optional selector, plus a mode: sync reconciles the source object to its target, while mirror is a read-only import. Patches translate references and nothing else: rewriteName, rewriteNamespace, rewriteRef, rewriteHostRef, rewriteLabelSelector and none are the complete vocabulary.

That restriction is deliberate. A sync configuration should say which reference is being translated and why. It shouldn't become a second, unreviewed templating language.

The plugin protects the host copy

A resource crossing a cluster boundary isn't a blind copy. The plugin has a few rules that keep source-of-truth decisions visible.

statusSync only operates for a resource with a real status subresource, and status always flows from host to virtual cluster. The plugin strips tenant-written status before it creates the host object. A tenant cannot write an Accepted=True condition into a route or an application and have the platform treat it as host state.

Selectors support labels plus namespace include and exclude rules. Exclusions win. Deletes are provenance-gated: the plugin removes only an object it created and stamped, never an unrelated object that happens to share a name. Trace logging redacts Secret data, and the plugin exports Prometheus metrics on its own endpoint.

The configuration parser is strict as well. An unknown field is an error, so a misspelt key in the sort of YAML that lives for months in a tenant chart fails at load time.

The Argo field we had to leave alone

One lesson came the hard way, from a real failure in our own Argo CD integration. Argo stores a pending sync in the top-level operation field of an Application. Our syncer treated the virtual copy as complete desired state and could remove that field after Argo wrote it but before Argo's worker read it. Auto-sync then never ran.

The fix became hostOwnedFields. These are top-level fields owned by a host controller: the syncer neither copies them from the virtual object nor deletes them when the virtual object lacks them, and host-side changes do not trigger reconciliation. Application.operation is now host-owned by default.

That rule sounds narrow until you run a controller on both sides of a boundary. The object may be shared; every field does not have to be.

A small custom-resource sync

The smallest useful configuration syncs one custom resource from a tenant vCluster to its host cluster. For this walkthrough we'll borrow the test-only Widget CRD from the plugin's own end-to-end suite.

First install the CRD on both sides:

kubectl apply -f https://raw.githubusercontent.com/kupecloud/vcluster-generic-sync-plugin/main/test/testdata/widget-crd.yaml
vcluster connect my-vcluster -- kubectl apply -f https://raw.githubusercontent.com/kupecloud/vcluster-generic-sync-plugin/main/test/testdata/widget-crd.yaml

Then add the plugin and the RBAC required to discover the CRD and reconcile Widget objects:

plugin:
  generic-sync:
    version: v2
    image: ghcr.io/kupecloud/vcluster-generic-sync-plugin:latest # pin a release in production
    imagePullPolicy: IfNotPresent
    rbac:
      role:
        extraRules:
          - apiGroups: ['example.com']
            resources: ['widgets']
            verbs: ['create', 'delete', 'patch', 'update', 'get', 'list', 'watch']
      clusterRole:
        extraRules:
          - apiGroups: ['apiextensions.k8s.io']
            resources: ['customresourcedefinitions']
            verbs: ['get', 'list', 'watch']
    config:
      version: v1
      syncResources:
        - apiVersion: example.com/v1
          kind: Widget
          direction: toHost

rbac:
  clusterRole:
    enabled: true

The two version fields are unrelated: v2 is vCluster's plugin format, while config.version: v1 is the plugin's own configuration schema. The rbac blocks are plain vCluster chart values; the chart folds extraRules into the Role and ClusterRole it creates for the syncer pod.

Create the matching test object through the vCluster connection. The plugin's base configuration guide explains the RBAC shape in more detail. In a real installation, scope those rules to each kind and namespace you intend to sync; * is easier to write and much harder to explain later.

Shared Gateway, tenant routes

Gateway API shows why a generic syncer needs patches. The tenant's HTTPRoute has two kinds of references:

parentRefs point at a Gateway that already exists on the host. Keep those values unchanged.
backendRefs point at Services from the tenant cluster. Rewrite those references to the host-side names created by vCluster.

The plugin's Gateway API example installs the Gateway API CRDs on both clusters and supplies the complete RBAC. The sync configuration does the real work:

plugin:
  generic-sync:
    version: v2
    image: ghcr.io/kupecloud/vcluster-generic-sync-plugin:latest # pin a release in production
    config:
      version: v1
      syncResources:
        - apiVersion: gateway.networking.k8s.io/v1
          kind: HTTPRoute
          direction: toHost
          statusSync: true
          patches:
            - path: spec.parentRefs[*]
              type: none
            - path: spec.rules[*].backendRefs[*]
              type: rewriteRef

type: none is a sharp tool. It lets a route target a shared host Gateway directly, so the host must enforce which vCluster namespaces may attach, which hostnames they may claim, and where the listener sends traffic. The plugin can preserve a reference; it cannot decide whether that reference is safe for your gateway policy.

Seven crossings, scoped permissions

Our production configuration contains seven resource kinds built from the same pieces: a read-only GatewayClass mirror; host-side HTTPRoute, Application, PrometheusRule and dashboard sync; repository-secret sync guarded by Argo labels; and platform-managed Secret import.

The RBAC follows the resource boundary. Application and repository-secret permissions live in namespace Roles rather than a broad ClusterRole, so the plugin in one tenant vCluster cannot list another tenant's credentials. hostNamespace is useful for controllers such as central Argo CD, but it's not a reason to make every host namespace visible.

There's a limit to what “generic” means here. The plugin can sync arbitrary declared kinds, yet it stamps kupe.cloud/* labels and annotations as its provenance contract. That's how it knows which objects it owns, which copies it may delete, and which changes are worth reconciling. The generic part is the configured resource type; the ownership contract stays specific.

Start with one crossing

If you're adding the plugin to your own vCluster setup, start with a single kind: the one that already has an obvious source of truth. Write one syncResources entry for it, grant the narrowest RBAC that lets it work, and watch what lands on the host: the object, its translated references, its status. Once that first crossing behaves, the rest are the same pattern with different nouns.

If something goes wrong, look at ownership before configuration. In our experience the first failure is a field or a reference that both sides think they own, and hostOwnedFields or a patch is usually the fix.

The plugin lives at github.com/kupecloud/vcluster-generic-sync-plugin, with the configuration reference, worked examples and the end-to-end suite the Widget walkthrough came from. If you hit a bug or something that doesn't behave as documented, open an issue and we'll happily help.