POC Install Reference
This page is the full reference for the install.sh script. For the short procedure, see Install a Kloudfuse POC.
Before you start
You need:
-
license.json, the license file from Kloudfuse. Contact Kloudfuse to request one. -
A DNS name for Kloudfuse, such as
kfuse.example.com, and access to its DNS. The install asks you to add anArecord for it. -
For an existing cluster: a kubeconfig context with
cluster-admin, and a node labelled and taintedkf_role=kfusewith at least 16 vCPU and 128 GiB memory. See Existing cluster requirements for everything the cluster needs.
license.json holds a registry key. Keep it private: don’t commit, share, or attach it.
|
Tools
install.sh checks for these tools first and shows a table of what it found. On Windows, run install.sh in WSL.
| Tool | Version |
|---|---|
Matching the cluster |
|
3.8 or later, or 4.x |
|
yq ( |
4.x |
|
any |
|
any |
|
any |
|
any |
|
any |
Terraform (new cluster only) |
1.5 or later (1.x) |
AWS CLI (new cluster on AWS) |
2.x |
|
current |
Azure CLI (new cluster on Azure) |
current |
On an existing cluster, the cloud’s CLI is needed only if your kubeconfig signs in with it.
Sign in to your cloud
-
AWS
-
Google Cloud
-
Azure
aws sso login
To use access keys instead, run aws configure. For an existing cluster, also get its kubeconfig context:
aws eks update-kubeconfig --name <cluster>
On a new cluster, use the same AWS identity for Terraform and install.sh. EKS makes only the identity that created the cluster its admin.
gcloud auth login
gcloud auth application-default login
On the Google sign-in page, check Select all before you click Continue. For an existing cluster, also get its kubeconfig context:
gcloud container clusters get-credentials <cluster>
az login
az account set --subscription <subscription_id>
For an existing cluster, also get its kubeconfig context:
az aks get-credentials --name <cluster> --resource-group <group>
New cluster requirements
Terraform creates a small cluster with two nodes: one for the Kubernetes add-ons and one for Kloudfuse. Before Terraform creates anything, a permission check confirms that your identity has the permissions it needs. POC Terraform Resources lists everything Terraform creates.
-
AWS
-
Google Cloud
-
Azure
| Node | Instance type | Runs |
|---|---|---|
system |
|
Kubernetes add-ons |
kfuse |
|
Kloudfuse |
Requirements
-
vCPU quota of at least 18 for
Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instancesin the target region (Service Quotas > Amazon EC2). -
Two Elastic IPs available in the region.
-
A region whose first availability zone offers
r7i.4xlarge.
| Node | Machine type | Runs |
|---|---|---|
system |
|
Kubernetes add-ons |
kfuse |
|
Kloudfuse |
Requirements
-
A Google Cloud project with billing enabled. Terraform turns on the Compute Engine and Kubernetes Engine APIs.
-
N4_CPUSquota of at least 16 in the region (IAM & Admin > Quotas). -
A zone that offers N4 machines.
| Node | VM size | Runs |
|---|---|---|
system |
|
Kubernetes add-ons |
kfuse |
|
Kloudfuse |
The v5 sizes are used where your subscription allows them, and the v7 sizes otherwise. Some credit and trial subscriptions block v5.
Requirements
-
vCPU quota in the region (Subscriptions > Usage + quotas): at least 16 for
Standard ESv5 Family vCPUsand 2 forStandard DSv5 Family vCPUs. Where the subscription can’t use v5, the same forStandard Esv7 Family vCPUsandStandard Dsv7 Family vCPUs. -
A region with availability zones and Premium SSD v2 disks.
eastus2(the default),westus2, andwesteuropework. To check another region and list its zones:az vm list-skus --location <region> --resource-type disks \ --query "[?name=='PremiumV2_LRS'].locationInfo[].zones" -o tsv
Existing cluster requirements
install.sh creates only what goes inside the cluster: the namespace, Secrets, the kloudfuse-disk storage class, the Gateway API CRDs, cert-manager (for Let’s Encrypt only), and Kloudfuse. The cluster and the resources below must exist first.
A kubeconfig context with cluster-admin, because the install creates CRDs, a storage class, and cluster roles.
A node for Kloudfuse with at least 16 vCPU and 128 GiB memory, for example r7i.4xlarge on AWS, n4-highmem-16 on Google Cloud, or Standard_E16s_v5 on Azure. The node must be Ready, and labelled and tainted so that only Kloudfuse runs on it:
kubectl label nodes <node> kf_role=kfuse
kubectl taint nodes <node> kf_role=kfuse:NoSchedule
-
Move other workloads off the node first, for example with
kubectl drain <node> --ignore-daemonsets, thenkubectl uncordon <node>. DaemonSet pods, such as kube-proxy, the network, and the disk driver, can stay, and so can cert-manager. -
Every node with the
kf_role=kfuselabel must meet these requirements, because Kloudfuse can be scheduled on any of them. -
The cluster’s own add-ons, such as DNS, need at least one other node.
Cloud resources for your cloud:
-
AWS
-
Google Cloud
-
Azure
-
The disk driver
ebs.csi.aws.com, from theaws-ebs-csi-driverEKS add-on. -
An S3 bucket in the cluster’s region that the Kloudfuse node’s IAM role can use:
-
s3:ListBucket,s3:GetBucketLocation, ands3:ListBucketMultipartUploadson the bucket. -
s3:GetObject,s3:PutObject,s3:DeleteObject,s3:AbortMultipartUpload, ands3:ListMultipartUploadPartson its objects.Pods sign in with the node’s role, so the node needs IMDSv2 with a hop limit of 2.
-
-
An Elastic IP for the load balancer, and a public subnet in the Kloudfuse node’s zone for it.
install.shasks for the IP, its allocation ID (eipalloc-…), and the subnet ID (subnet-…).
-
The disk driver
pd.csi.storage.gke.io, which GKE turns on by default. -
A GCS bucket in the cluster’s region, on which the Kloudfuse node’s service account has
roles/storage.objectAdminandroles/storage.legacyBucketReader. Pods sign in as the node’s service account, so the Kloudfuse node pool must use theGCE_METADATAmetadata server, not Workload Identity’sGKE_METADATA. -
A regional static external IP address in the cluster’s region, for the load balancer.
-
The disk driver
disk.csi.azure.com, which AKS turns on by default. The Kloudfuse node must be a premium-capable, zonal VM, because the volumes are Premium SSD v2. -
A storage account (ADLS Gen2) in the cluster’s location, with one container for the deep store and log archive (default
kfuse) and one for RUM (defaultrum). Kloudfuse signs in with the account’s connection string.install.shasks for it, or reads it fromKFUSE_AZURE_STORAGE_CONNECTION_STRING, and keeps it only incustom-values.yamland the cluster. -
A Standard, static public IP in the Kloudfuse node’s zone, for the load balancer. The cluster’s identity needs
Network Contributoron the IP’s resource group.install.shasks for the IP, its name, and its resource group.
Before it asks for the static IP and the storage, install.sh checks that the cluster answers, that the context is a cluster admin, that the disk driver is installed, and that every node labelled kf_role=kfuse is Ready, schedulable, tainted, has at least 16 vCPU and 120 GiB memory (a 128 GiB machine reports a little less), and runs only DaemonSet pods, cert-manager, and, on an upgrade, Kloudfuse. If a check fails, it stops and lists, node by node, what is wrong.
Installation
On a new cluster, the whole install, including creating the cloud resources, takes about 30 minutes. On an existing cluster, it takes about 10–15 minutes.
Create an empty folder and copy license.json to it. Open a terminal in that folder and run:
curl -fsSLO https://install.kloudfuse.io/kfuse-poc/install.sh && bash install.sh
Then:
-
Choose the install type:
1for a new cluster,2for a cluster you already have. -
New cluster: choose the cloud (
aws,gcp, orazure). Existing cluster: press Enter to confirm your current kubeconfig context, or type another one. The cloud is read from the cluster’s nodes. -
install.shdownloads the matching bundle intokfuse-poc-<version>/kfuse-poc-<bundle>/, puts a copy of the bundle’suninstall.shin the current folder, and carries on from the bundle folder. -
Answer the questions. Press Enter to take the value in
[brackets]. You choose the password of theadminlogin here, 12 to 72 characters. Remember it: the cluster keeps only its hash. See Settings for every question. -
New cluster: the script shows the cloud account it is about to install into; answer Yes if it is the right one. Review Terraform’s plans and type
yes. -
Add the DNS record that the script prints, where your domain’s DNS is managed, then press Enter:
<domain> A <static IP>textThe script checks that the name resolves on the public resolvers
1.1.1.1and8.8.8.8. If it doesn’t resolve yet, wait a minute and press Enter again. -
The script installs Kloudfuse and waits until it answers. Once a minute it prints what it is waiting for: the load balancer’s address, the HTTPS answer, or the certificate. Then open
https://<domain>;and log in asadmin.
If a step fails, the script stops and shows the error. Everything that Terraform, kubectl, and Helm printed is in install.log in the bundle folder. Fix the problem, then run ./install.sh again; every step is safe to repeat. See POC Troubleshooting.
To skip the first two questions, add --cloud aws, --cloud gcp, or --cloud azure for a new cluster, or --existing-cluster for a cluster you already have.
|
POC bundles
A bundle is a zip file that holds install.sh, uninstall.sh, and a README.md; the cloud bundles also hold the Terraform files. You don’t have to download one yourself: install.sh downloads the bundle for your choice. The current bundle version is 1.0.2.
| Bundle | For | Download |
|---|---|---|
|
A new EKS cluster on Amazon Web Services |
|
|
A new GKE cluster on Google Cloud |
|
|
A new AKS cluster on Microsoft Azure |
|
|
An EKS, GKE, or AKS cluster you already have (no Terraform) |
Each bundle installs the one Kloudfuse version it was tested with. The bundle version, such as 1.0.2, is not the Kloudfuse version. To install a newer Kloudfuse version, move to a newer bundle.
Settings
install.sh asks for these settings, in groups with a heading each, and keeps the answers in install.env for the next run. Options on the command line win over install.env. Run ./install.sh --help for the full list of options.
| Setting | Option or install.env line |
Description |
|---|---|---|
Cloud account |
AWS: |
New cluster only: where to create it. On Google Cloud, the zone must offer N4 machines. On Azure, the zone must offer the node sizes for your subscription. |
Cluster name |
|
New cluster only: the name of the cluster, and the prefix of the other resource names. Default |
Existing cluster |
All clouds: |
Existing cluster only. See Existing cluster requirements. |
DNS name |
|
The name Kloudfuse is served on over HTTPS, such as |
License |
|
The |
Admin password |
|
The password of the |
HTTPS certificate |
|
Let’s Encrypt (the default) needs the DNS name to resolve publicly to the cluster and port 80 reachable from the internet. Your own certificate is a PEM file with any intermediate certificates, and a key without a password, valid for at least 7 more days. |
SMTP server |
|
Optional mail server for alert emails. The default port is 587; AWS and Google Cloud block port 25. The password is asked, or read from |
Namespace |
|
The Kubernetes namespace for Kloudfuse. Default |
The cloud account settings, the cluster name, and the organization ID from license.json can’t be changed after installation.
Unattended install
--yes asks nothing. It uses install.env and the defaults, and accepts every confirmation. On an existing cluster it uses your current kubeconfig context. A setting with no value stops the script and names the option or install.env line to set. Passwords come from environment variables:
-
KFUSE_ADMIN_PASSWORD, required for a first install. -
KFUSE_SMTP_PASSWORD, when you set an SMTP server.--yesturns on alert emails only wheninstall.envhassmtp=onand thesmtp_lines. -
KFUSE_AZURE_STORAGE_CONNECTION_STRING, on an existing Azure cluster.
Post-install
Keep the bundle folder for as long as the POC runs: it holds every file of your install, and every follow-up action runs from it.
| Action | Run, in the bundle folder |
|---|---|
Carry on after a stop, or reinstall |
|
Change a setting, or the admin password |
|
Add Helm values the install doesn’t set |
Put them in |
Remove Kloudfuse |
|
Check the install
On a new cluster, install.sh keeps the cluster’s credentials in kubeconfig in the bundle folder; your own kubeconfig isn’t changed. To use them:
export KUBECONFIG=$PWD/kubeconfig
kubectl get nodes -L kf_role
kubectl -n kfuse get pods
You see two Ready nodes, with kf_role system and kfuse, and every Kloudfuse pod is Running or Completed. On an existing cluster, run the same kubectl commands with your own kubeconfig context.
The commands on this page use the default namespace, kfuse. If you set another namespace=, use it in place of kfuse in kubectl -n kfuse.
To add a new cluster to your own kubeconfig instead, run this in the bundle folder:
bash -c "$(terraform output -raw kubeconfig_command)"
Install files
Only you can read the files in the bundle folder. One folder holds one install; use another folder for a second install.
| File | Contents |
|---|---|
|
Your settings, without secrets |
|
The Helm values that |
|
Optional. Your own Helm values. |
|
Everything Terraform, kubectl, and Helm printed |
|
New cluster only. The settings Terraform reads. |
|
New cluster only. Terraform’s record of what it created |
|
New cluster only. The cluster’s credentials |
|
Existing cluster only. Tells |
On Azure, custom-values.yaml and its backups hold the storage account key. Don’t commit, share, or attach them.
|
Run the install again
Run ./install.sh in the bundle folder, or the same curl command from the folder you first ran it in. The bundle isn’t downloaded again when its folder exists. The script shows a summary of the earlier install and asks Reinstall with these settings?:
-
Yes reinstalls with the saved settings and keeps the admin and SMTP passwords.
-
No asks every question again, with the saved answers as defaults.
On an existing cluster, the script asks for the kubeconfig context on every run, and stops if it isn’t the context the folder’s install was made in.
If custom-values.yaml differs from what install.sh writes, for example after an edit, the script shows the difference and asks before it replaces the file. The old file is backed up. Put lasting changes in the settings or in extended-values.yaml.
Reset the admin password
The Kloudfuse UI has no password reset. Run ./install.sh, answer No to Reinstall with these settings?, then Yes to Change the admin password?, and type the new password. To do it unattended:
KFUSE_ADMIN_PASSWORD='<new password>' ./install.sh --yes
Install a specific bundle version
Each bundle version has its own address, where <bundle> is aws, gcp, azure, or helm:
https://install.kloudfuse.io/kfuse-poc/<version>/install.sh
https://install.kloudfuse.io/kfuse-poc/<version>/<bundle>.zip
To have the version-free install.sh download a specific version, put the line bundle_version=<version> in a file named install.env beside it before you run it. To download and unzip a bundle yourself:
export VERSION=1.0.2
export BUNDLE=aws
curl -fLO "https://install.kloudfuse.io/kfuse-poc/$VERSION/$BUNDLE.zip" &&
unzip -o -q "$BUNDLE.zip" -d "kfuse-poc-$VERSION" &&
cd "kfuse-poc-$VERSION/kfuse-poc-$BUNDLE" &&
./install.sh
The download site can’t be browsed. A version or bundle that doesn’t exist returns 403 Forbidden, not 404.
Upgrade Kloudfuse with a newer bundle
Your settings, and on a new cluster Terraform’s state, are files in the bundle folder. A new bundle in a new folder starts without them, and on a new cluster would try to create a second cluster with the same names, which fails. Copy your files across before you run anything in the new folder.
Run these commands from the folder you first downloaded into, the one that holds kfuse-poc-<version>/. OLD is the bundle version you installed, NEW is the one to move to, and BUNDLE is your cloud, or helm for an existing cluster:
export OLD=1.0.2
export NEW=1.1.0
export BUNDLE=aws
curl -fLO "https://install.kloudfuse.io/kfuse-poc/$NEW/$BUNDLE.zip"
unzip "$BUNDLE.zip" -d "kfuse-poc-$NEW"
for f in terraform.tfvars terraform.tfstate install.env custom-values.yaml extended-values.yaml; do
# An existing cluster has no Terraform files; extended-values.yaml is optional.
[ ! -f "kfuse-poc-$OLD/kfuse-poc-$BUNDLE/$f" ] ||
cp "kfuse-poc-$OLD/kfuse-poc-$BUNDLE/$f" "kfuse-poc-$NEW/kfuse-poc-$BUNDLE/"
done
cd "kfuse-poc-$NEW/kfuse-poc-$BUNDLE"
./install.sh
Also copy license.json and your certificate files if they are in the old folder. On a new cluster, install.sh shows Terraform’s plan before it changes anything; review it. Helm then upgrades Kloudfuse in place, and the admin password stays unless you change it.
Once the new bundle has changed the install, the old bundle’s install.sh and uninstall.sh refuse to run on it; use the new folder from then on. Keep the old folder until install.sh succeeds in the new one. Afterwards, the version-free install.sh uses the new folder: it picks the newest bundle folder that holds an install.
Uninstall the POC
On a new cluster, sign in to your cloud first. Then run this from the folder you started the install in, or from the bundle folder:
./uninstall.sh
The downloaded install.sh leaves a copy of uninstall.sh in the folder you started in. That copy finds the install in the bundle folder below it and runs the bundle’s own uninstall.sh. If the folder holds more than one install, for example a new-cluster install and an existing-cluster install, it lists their uninstall.sh paths and stops; run the one you want to remove.
uninstall.sh removes only what the install created.
- On a new cluster
-
The script asks whether to keep the storage bucket that holds your Kloudfuse data (press Enter to delete it), and asks you to type the cluster name to confirm. It removes Kloudfuse from the cluster, including its load balancer and disks, then has Terraform delete everything else. It takes about 15–25 minutes. When you keep the bucket, Kloudfuse’s pods stop normally so their last writes reach it, which can add up to about 27 minutes. Delete a kept bucket when you no longer need the data.
Don’t run
terraform destroyon its own: it stops while the Kloudfuse load balancer still uses the static IP. - On an existing cluster
-
After you type the namespace, the script removes Kloudfuse, its namespace, and its GatewayClass. It also removes what
install.shinstalled because it wasn’t there yet: cert-manager and its CRDs, the Gateway API CRDs, and thekloudfuse-diskstorage class, each only when nothing else on the cluster uses it. The cluster, the static IP, the storage and its data, and anything that was on the cluster before the install stay. Kloudfuse’s pods get 30 seconds to stop, so its last few minutes of data may not reach the storage.
Then delete the DNS record you created. If a step fails, the script lists what wasn’t deleted; run ./uninstall.sh again to carry on. The details are in uninstall.log.