POC Install Reference

This page is the full reference for the install.sh script. For the short procedure, see Install a Kloudfuse POC.

Before you start

You need:

  • license.json, the license file from Kloudfuse. Contact Kloudfuse to request one.

  • A DNS name for Kloudfuse, such as kfuse.example.com, and access to its DNS. The install asks you to add an A record for it.

  • For an existing cluster: a kubeconfig context with cluster-admin, and a node labelled and tainted kf_role=kfuse with at least 16 vCPU and 128 GiB memory. See Existing cluster requirements for everything the cluster needs.

license.json holds a registry key. Keep it private: don’t commit, share, or attach it.

Tools

install.sh checks for these tools first and shows a table of what it found. On Windows, run install.sh in WSL.

Tool Version

kubectl

Matching the cluster

Helm

3.8 or later, or 4.x

yq (brew install yq)

4.x

curl

any

unzip

any

htpasswd. Preinstalled on macOS. On Linux, install apache2-utils (Debian, Ubuntu) or httpd-tools (RHEL, Fedora).

any

dig or nslookup. On Linux, install dnsutils (Debian, Ubuntu) or bind-utils (RHEL, Fedora).

any

openssl (only with your own TLS certificate)

any

Terraform (new cluster only)

1.5 or later (1.x)

AWS CLI (new cluster on AWS)

2.x

gcloud and gke-gcloud-auth-plugin (new cluster on Google Cloud)

current

Azure CLI (new cluster on Azure)

current

On an existing cluster, the cloud’s CLI is needed only if your kubeconfig signs in with it.

Sign in to your cloud

aws sso login

To use access keys instead, run aws configure. For an existing cluster, also get its kubeconfig context:

aws eks update-kubeconfig --name <cluster>

On a new cluster, use the same AWS identity for Terraform and install.sh. EKS makes only the identity that created the cluster its admin.

New cluster requirements

Terraform creates a small cluster with two nodes: one for the Kubernetes add-ons and one for Kloudfuse. Before Terraform creates anything, a permission check confirms that your identity has the permissions it needs. POC Terraform Resources lists everything Terraform creates.

Node Instance type Runs

system

m6i.large (2 vCPU, 8 GiB)

Kubernetes add-ons

kfuse

r7i.4xlarge (16 vCPU, 128 GiB)

Kloudfuse

Requirements

  • vCPU quota of at least 18 for Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances in the target region (Service Quotas > Amazon EC2).

  • Two Elastic IPs available in the region.

  • A region whose first availability zone offers r7i.4xlarge.

Existing cluster requirements

install.sh creates only what goes inside the cluster: the namespace, Secrets, the kloudfuse-disk storage class, the Gateway API CRDs, cert-manager (for Let’s Encrypt only), and Kloudfuse. The cluster and the resources below must exist first.

A kubeconfig context with cluster-admin, because the install creates CRDs, a storage class, and cluster roles.

A node for Kloudfuse with at least 16 vCPU and 128 GiB memory, for example r7i.4xlarge on AWS, n4-highmem-16 on Google Cloud, or Standard_E16s_v5 on Azure. The node must be Ready, and labelled and tainted so that only Kloudfuse runs on it:

kubectl label nodes <node> kf_role=kfuse
kubectl taint nodes <node> kf_role=kfuse:NoSchedule
  • Move other workloads off the node first, for example with kubectl drain <node> --ignore-daemonsets, then kubectl uncordon <node>. DaemonSet pods, such as kube-proxy, the network, and the disk driver, can stay, and so can cert-manager.

  • Every node with the kf_role=kfuse label must meet these requirements, because Kloudfuse can be scheduled on any of them.

  • The cluster’s own add-ons, such as DNS, need at least one other node.

Cloud resources for your cloud:

  • The disk driver ebs.csi.aws.com, from the aws-ebs-csi-driver EKS add-on.

  • An S3 bucket in the cluster’s region that the Kloudfuse node’s IAM role can use:

    • s3:ListBucket, s3:GetBucketLocation, and s3:ListBucketMultipartUploads on the bucket.

    • s3:GetObject, s3:PutObject, s3:DeleteObject, s3:AbortMultipartUpload, and s3:ListMultipartUploadParts on its objects.

      Pods sign in with the node’s role, so the node needs IMDSv2 with a hop limit of 2.

  • An Elastic IP for the load balancer, and a public subnet in the Kloudfuse node’s zone for it. install.sh asks for the IP, its allocation ID (eipalloc-…​), and the subnet ID (subnet-…​).

Before it asks for the static IP and the storage, install.sh checks that the cluster answers, that the context is a cluster admin, that the disk driver is installed, and that every node labelled kf_role=kfuse is Ready, schedulable, tainted, has at least 16 vCPU and 120 GiB memory (a 128 GiB machine reports a little less), and runs only DaemonSet pods, cert-manager, and, on an upgrade, Kloudfuse. If a check fails, it stops and lists, node by node, what is wrong.

Installation

On a new cluster, the whole install, including creating the cloud resources, takes about 30 minutes. On an existing cluster, it takes about 10–15 minutes.

Create an empty folder and copy license.json to it. Open a terminal in that folder and run:

curl -fsSLO https://install.kloudfuse.io/kfuse-poc/install.sh && bash install.sh

Then:

  1. Choose the install type: 1 for a new cluster, 2 for a cluster you already have.

  2. New cluster: choose the cloud (aws, gcp, or azure). Existing cluster: press Enter to confirm your current kubeconfig context, or type another one. The cloud is read from the cluster’s nodes.

  3. install.sh downloads the matching bundle into kfuse-poc-<version>/kfuse-poc-<bundle>/, puts a copy of the bundle’s uninstall.sh in the current folder, and carries on from the bundle folder.

  4. Answer the questions. Press Enter to take the value in [brackets]. You choose the password of the admin login here, 12 to 72 characters. Remember it: the cluster keeps only its hash. See Settings for every question.

  5. New cluster: the script shows the cloud account it is about to install into; answer Yes if it is the right one. Review Terraform’s plans and type yes.

  6. Add the DNS record that the script prints, where your domain’s DNS is managed, then press Enter:

    <domain>   A   <static IP>
    text

    The script checks that the name resolves on the public resolvers 1.1.1.1 and 8.8.8.8. If it doesn’t resolve yet, wait a minute and press Enter again.

  7. The script installs Kloudfuse and waits until it answers. Once a minute it prints what it is waiting for: the load balancer’s address, the HTTPS answer, or the certificate. Then open https://<domain>; and log in as admin.

If a step fails, the script stops and shows the error. Everything that Terraform, kubectl, and Helm printed is in install.log in the bundle folder. Fix the problem, then run ./install.sh again; every step is safe to repeat. See POC Troubleshooting.

To skip the first two questions, add --cloud aws, --cloud gcp, or --cloud azure for a new cluster, or --existing-cluster for a cluster you already have.

POC bundles

A bundle is a zip file that holds install.sh, uninstall.sh, and a README.md; the cloud bundles also hold the Terraform files. You don’t have to download one yourself: install.sh downloads the bundle for your choice. The current bundle version is 1.0.2.

Bundle For Download

aws

A new EKS cluster on Amazon Web Services

aws.zip

gcp

A new GKE cluster on Google Cloud

gcp.zip

azure

A new AKS cluster on Microsoft Azure

azure.zip

helm

An EKS, GKE, or AKS cluster you already have (no Terraform)

helm.zip

Each bundle installs the one Kloudfuse version it was tested with. The bundle version, such as 1.0.2, is not the Kloudfuse version. To install a newer Kloudfuse version, move to a newer bundle.

Settings

install.sh asks for these settings, in groups with a heading each, and keeps the answers in install.env for the next run. Options on the command line win over install.env. Run ./install.sh --help for the full list of options.

Setting Option or install.env line Description

Cloud account

AWS: region=
Google Cloud: project_id=, region=, zone=
Azure: subscription_id=, region=, zone=

New cluster only: where to create it. On Google Cloud, the zone must offer N4 machines. On Azure, the zone must offer the node sizes for your subscription.

Cluster name

cluster_name=

New cluster only: the name of the cluster, and the prefix of the other resource names. Default kfuse-poc.

Existing cluster

All clouds: kube_context=, static_ip=
AWS: region=, eip_allocation=, ingress_subnet=, bucket=
Google Cloud: bucket=
Azure: resource_group=, pip_name=, storage_account=, container=, rum_container=

Existing cluster only. See Existing cluster requirements.

DNS name

--domain

The name Kloudfuse is served on over HTTPS, such as kfuse.example.com.

License

--license

The license.json file. A license.json in the folder you run from is used without asking.

Admin password

KFUSE_ADMIN_PASSWORD environment variable, or asked

The password of the admin login, 12 to 72 characters. It is never saved in a file; the cluster keeps only its hash.

HTTPS certificate

--letsencrypt-email, or --tls-cert and --tls-key

Let’s Encrypt (the default) needs the DNS name to resolve publicly to the cluster and port 80 reachable from the internet. Your own certificate is a PEM file with any intermediate certificates, and a key without a password, valid for at least 7 more days.

SMTP server

smtp=on, smtp_host=, smtp_port=, smtp_from=, smtp_name=, smtp_user=

Optional mail server for alert emails. The default port is 587; AWS and Google Cloud block port 25. The password is asked, or read from KFUSE_SMTP_PASSWORD, and is kept only in the cluster.

Namespace

namespace=

The Kubernetes namespace for Kloudfuse. Default kfuse.

The cloud account settings, the cluster name, and the organization ID from license.json can’t be changed after installation.

Unattended install

--yes asks nothing. It uses install.env and the defaults, and accepts every confirmation. On an existing cluster it uses your current kubeconfig context. A setting with no value stops the script and names the option or install.env line to set. Passwords come from environment variables:

  • KFUSE_ADMIN_PASSWORD, required for a first install.

  • KFUSE_SMTP_PASSWORD, when you set an SMTP server. --yes turns on alert emails only when install.env has smtp=on and the smtp_ lines.

  • KFUSE_AZURE_STORAGE_CONNECTION_STRING, on an existing Azure cluster.

Post-install

Keep the bundle folder for as long as the POC runs: it holds every file of your install, and every follow-up action runs from it.

Action Run, in the bundle folder

Carry on after a stop, or reinstall

./install.sh. Every step is safe to repeat.

Change a setting, or the admin password

./install.sh, and answer No to Reinstall with these settings?

Add Helm values the install doesn’t set

Put them in extended-values.yaml, then run ./install.sh

Remove Kloudfuse

./uninstall.sh, here or in the folder you started in. It removes only what the install created.

Check the install

On a new cluster, install.sh keeps the cluster’s credentials in kubeconfig in the bundle folder; your own kubeconfig isn’t changed. To use them:

export KUBECONFIG=$PWD/kubeconfig
kubectl get nodes -L kf_role
kubectl -n kfuse get pods

You see two Ready nodes, with kf_role system and kfuse, and every Kloudfuse pod is Running or Completed. On an existing cluster, run the same kubectl commands with your own kubeconfig context.

The commands on this page use the default namespace, kfuse. If you set another namespace=, use it in place of kfuse in kubectl -n kfuse.

To add a new cluster to your own kubeconfig instead, run this in the bundle folder:

bash -c "$(terraform output -raw kubeconfig_command)"

Install files

Only you can read the files in the bundle folder. One folder holds one install; use another folder for a second install.

File Contents

install.env

Your settings, without secrets

custom-values.yaml

The Helm values that install.sh writes

extended-values.yaml

Optional. Your own Helm values. install.sh passes it to Helm after custom-values.yaml, so its values win, and never changes it.

install.log

Everything Terraform, kubectl, and Helm printed

terraform.tfvars

New cluster only. The settings Terraform reads. install.sh rewrites its own lines on every run and keeps any line you add.

terraform.tfstate

New cluster only. Terraform’s record of what it created

kubeconfig

New cluster only. The cluster’s credentials

HELM_ONLY

Existing cluster only. Tells install.sh to skip Terraform; don’t delete it.

On Azure, custom-values.yaml and its backups hold the storage account key. Don’t commit, share, or attach them.

Run the install again

Run ./install.sh in the bundle folder, or the same curl command from the folder you first ran it in. The bundle isn’t downloaded again when its folder exists. The script shows a summary of the earlier install and asks Reinstall with these settings?:

  • Yes reinstalls with the saved settings and keeps the admin and SMTP passwords.

  • No asks every question again, with the saved answers as defaults.

On an existing cluster, the script asks for the kubeconfig context on every run, and stops if it isn’t the context the folder’s install was made in.

If custom-values.yaml differs from what install.sh writes, for example after an edit, the script shows the difference and asks before it replaces the file. The old file is backed up. Put lasting changes in the settings or in extended-values.yaml.

Reset the admin password

The Kloudfuse UI has no password reset. Run ./install.sh, answer No to Reinstall with these settings?, then Yes to Change the admin password?, and type the new password. To do it unattended:

KFUSE_ADMIN_PASSWORD='<new password>' ./install.sh --yes

Install a specific bundle version

Each bundle version has its own address, where <bundle> is aws, gcp, azure, or helm:

https://install.kloudfuse.io/kfuse-poc/<version>/install.sh
https://install.kloudfuse.io/kfuse-poc/<version>/<bundle>.zip
text

To have the version-free install.sh download a specific version, put the line bundle_version=<version> in a file named install.env beside it before you run it. To download and unzip a bundle yourself:

export VERSION=1.0.2
export BUNDLE=aws
curl -fLO "https://install.kloudfuse.io/kfuse-poc/$VERSION/$BUNDLE.zip" &&
  unzip -o -q "$BUNDLE.zip" -d "kfuse-poc-$VERSION" &&
  cd "kfuse-poc-$VERSION/kfuse-poc-$BUNDLE" &&
  ./install.sh

The download site can’t be browsed. A version or bundle that doesn’t exist returns 403 Forbidden, not 404.

Upgrade Kloudfuse with a newer bundle

Your settings, and on a new cluster Terraform’s state, are files in the bundle folder. A new bundle in a new folder starts without them, and on a new cluster would try to create a second cluster with the same names, which fails. Copy your files across before you run anything in the new folder.

Run these commands from the folder you first downloaded into, the one that holds kfuse-poc-<version>/. OLD is the bundle version you installed, NEW is the one to move to, and BUNDLE is your cloud, or helm for an existing cluster:

export OLD=1.0.2
export NEW=1.1.0
export BUNDLE=aws
curl -fLO "https://install.kloudfuse.io/kfuse-poc/$NEW/$BUNDLE.zip"
unzip "$BUNDLE.zip" -d "kfuse-poc-$NEW"
for f in terraform.tfvars terraform.tfstate install.env custom-values.yaml extended-values.yaml; do
  # An existing cluster has no Terraform files; extended-values.yaml is optional.
  [ ! -f "kfuse-poc-$OLD/kfuse-poc-$BUNDLE/$f" ] ||
    cp "kfuse-poc-$OLD/kfuse-poc-$BUNDLE/$f" "kfuse-poc-$NEW/kfuse-poc-$BUNDLE/"
done
cd "kfuse-poc-$NEW/kfuse-poc-$BUNDLE"
./install.sh

Also copy license.json and your certificate files if they are in the old folder. On a new cluster, install.sh shows Terraform’s plan before it changes anything; review it. Helm then upgrades Kloudfuse in place, and the admin password stays unless you change it.

Once the new bundle has changed the install, the old bundle’s install.sh and uninstall.sh refuse to run on it; use the new folder from then on. Keep the old folder until install.sh succeeds in the new one. Afterwards, the version-free install.sh uses the new folder: it picks the newest bundle folder that holds an install.

Uninstall the POC

On a new cluster, sign in to your cloud first. Then run this from the folder you started the install in, or from the bundle folder:

./uninstall.sh

The downloaded install.sh leaves a copy of uninstall.sh in the folder you started in. That copy finds the install in the bundle folder below it and runs the bundle’s own uninstall.sh. If the folder holds more than one install, for example a new-cluster install and an existing-cluster install, it lists their uninstall.sh paths and stops; run the one you want to remove.

uninstall.sh removes only what the install created.

On a new cluster

The script asks whether to keep the storage bucket that holds your Kloudfuse data (press Enter to delete it), and asks you to type the cluster name to confirm. It removes Kloudfuse from the cluster, including its load balancer and disks, then has Terraform delete everything else. It takes about 15–25 minutes. When you keep the bucket, Kloudfuse’s pods stop normally so their last writes reach it, which can add up to about 27 minutes. Delete a kept bucket when you no longer need the data.

Don’t run terraform destroy on its own: it stops while the Kloudfuse load balancer still uses the static IP.

On an existing cluster

After you type the namespace, the script removes Kloudfuse, its namespace, and its GatewayClass. It also removes what install.sh installed because it wasn’t there yet: cert-manager and its CRDs, the Gateway API CRDs, and the kloudfuse-disk storage class, each only when nothing else on the cluster uses it. The cluster, the static IP, the storage and its data, and anything that was on the cluster before the install stay. Kloudfuse’s pods get 30 seconds to stop, so its last few minutes of data may not reach the storage.

Then delete the DNS record you created. If a step fails, the script lists what wasn’t deleted; run ./uninstall.sh again to carry on. The details are in uninstall.log.