Enable RUM on Kloudfuse for AWS S3

Step 1 Create Session Replay

You can store RUM session recordings in an AWS S3 container. Follow these steps to create an S3 bucket:

  1. Follow instructions in AWS documentation on Creating a bucket.

  2. Save for following tasks this information:

    • S3 accessKey and secretKey; use them to create a secret on the Kloudfuse.

    • S3 bucketName and awsRegion where the bucket exists

Step 1 Create Secret for Kloudfuse Namespace

Run the following command in the cluster that runs Kloudfuse:

Create a Secret for Kloudfuse Namespace
kubectl create secret generic kfuse-rum-s3 --from-literal=accessKey=<accessKey> --from-literal=secretKey='<secretKey>'

Step 1 Configure the custom-values.yaml File

  1. Generate a UUID by running the following command:

    uuidgen | tr 'A-Z' 'a-z'

    You must reference this UUID in the Kloudfuse frontend SDK initialization call.

  2. Add a global.rum section to the custom-values.yaml file:

      rum:
    enabled: true
    applications:
      - name: kf-frontend
        id: 944f6a58-dbc2-45ad-bf93-def505aaff62

    sessionReplayStorage:
      type: s3
      useSecret: true
      secretName: "kfuse-rum-s3"
      s3:
        region: us-west-2 (2)
        bucket: rum-session-replay-playground (3)
    yaml
    1 secretName: Secret created in Step 1 Create Secret for Kloudfuse Namespace.
    2 region: Region specified for session replay storage, in Step 1 Create Session Replay.
    3 bucket: AWS bucket created for session replay storage, in Step 1 Create Session Replay.

  3. List RUM-specific Kafka topics in the custom-values.yaml file.

    Use the configuration for the events stream as a reference for the number of replicas and partitions.

        - name: kf_rum_session_replay_topic
      partitions: 3
      replicationFactor: 2
    - name: kf_rum_views_topic
      partitions: 3
      replicationFactor: 2
    - name: kf_rum_actions_topic
      partitions: 3
      replicationFactor: 2
    - name: kf_rum_resources_topic
      partitions: 3
      replicationFactor: 2
    - name: kf_rum_longtasks_topic
      partitions: 3
      replicationFactor: 2
    - name: kf_rum_errors_topic
      partitions: 3
      replicationFactor: 2
    yaml

  4. Specify the TLS configuration in the tls section of the custom-values.yaml file, and ensure you use a public host.

    tls:
  enabled: true (1)
  host: playground.kloudfuse.io (2)
  email: admin@kloudfuse.com
  clusterIssuer: playground-letsencrypt-prod
    yaml
    1 enabled: true: Enable TLS.
    2 host: playground.kloudfuse.io: This host is public.

  5. Specify the Ingress configuration in the ingress section of the custom-values.yaml file.

    RUM must have a public ingest endpoint where the frontend browser posts data.

    ingress:
  controller:
    service:
      external:
        enabled: true (1)
      externalTrafficPolicy: Local (2)
    yaml
    1 Enable external ingress policy
    2 Enable and specify external traffic policy.

  6. Enable RUM in the ingester section of the custom-values.yaml file:

    ingester:
  config:
    rum:
      enabled: true
      # you can ignore the datadog section below from the PR since
      # this is the default
      datadog:
        proxyToDatadogEnabled: false
    yaml

  7. To ingest frontend logs into Kloudfuse, add parsing rules in the log-parser section of the custom-values.yaml file.

    Include these rules verbatim, along with existing rules.

    logs-parser:
  gcpConfig:
    enabled: true
    subscriptionId: "logs-perf"
    pubsubKey: "<Pub Sub Key>"
  replicaCount: 3
  jvmOpts: "<JVM Options>"
  extraApplicationConfigs: |-
    agent.properties.fluent-bit.enabled = true
  kf_parsing_config:
    config: |-
      - transform:
          args:
            - action: "facet_to_label_map"
            - sourceLabels: "@eventSource"
            - targetLabel: "source"
          conditions:
            - matcher: "#source"
              op: "=="
              value: "aws-cloudtrail-logs-502496443919-709d7802"
      - transform:
          tag: "arn_user_info"
          args:
            - action: facet_to_label_map
              sourceLabels: "@userIdentity_arn"
              # arn:aws:sts::502496443919:assumed-role/DatadogIntegrationRole/DatadogAWSIntegration
              regex: "^.*:.*:.*:.*:.*:(.*)"
              replacement: $1
              targetLabel: user_info
          conditions:
            - matcher: "#agent"
              op: "=="
              value: "kinesis"
            - matcher: "#logGroup"
              op: "=="
              value: "aws-cloudtrail-logs-502496443919-709d7802"
      - transform:
          args:
            - action: replace
              targetLabel: "#user_info"
              sourceLabels: "#user_info"
              replacement: "$1/$1"
              regex: "^(.*)$"
          conditions:
            - matcher: "__kf_tags"
              op: "in"
              value: "arn_user_info"
            - matcher: "#user_info"
              op: "=="
              value: "root"
      - transform:
          args:
            - action: replace
              sourceLabels: "#user_info"
              regex: "^(.*?)/.*?(/.*)?$"
              replacement: $1
              targetLabel: "resource_type"
          conditions:
            - matcher: "__kf_tags"
              op: "in"
              value: "arn_user_info"
      - transform:
          args:
            - action: replace
              sourceLabels: "#user_info"
              regex: "^.*?/(.*?)(/.*)?$"
              replacement: $1
              targetLabel: "role"
          conditions:
            - matcher: "__kf_tags"
              op: "in"
              value: "arn_user_info"
      - transform:
          args:
            - action: replace
              sourceLabels: "#user_info"
              regex: "^.*?/.*?/(.*)$"
              replacement: $1
              targetLabel: "src_user"
          conditions:
            - matcher: "__kf_tags"
              op: "in"
              value: "arn_user_info"
      - transform:
          args:
            - action: "replace"
              sourceLabels: "role"
              targetLabel: "src_user"
          conditions:
            - matcher: "__kf_tags"
              op: "in"
              value: "arn_user_info"
            - matcher: "#src_user"
              op: "=="
              value: ""
      - transform:
          tag: "src_role"
          args:
            - action: "replace"
              sourceLabels: "resource_type"
              targetLabel: "src_role"
          conditions:
            - matcher: "__kf_tags"
              op: "in"
              value: "arn_user_info"
            - matcher: "#role"
              op: "=="
              value: "#src_user"
      - transform:
          tag: "src_role"
          args:
            - action: "replace"
              sourceLabels: "role"
              targetLabel: "src_role"
          conditions:
            - matcher: "__kf_tags"
              op: "in"
              value: "arn_user_info"
            - matcher: "__kf_tags"
              op: "not in"
              value: "src_role"
      - transform:
          tag: "severity"
          args:
            - action: "replace"
              targetLabel: "severity_string"
              replacement: "Info"
          conditions:
            - matcher: "#agent"
              op: "=="
              value: "kinesis"
            - matcher: "#source"
              op: "contains"
              value: "guardduty"
            - matcher: "@detail_severity"
              op: "=="
              value: "0"
      - transform:
          tag: "severity"
          args:
            - action: "replace"
              targetLabel: "severity_string"
              replacement: "Low"
          conditions:
            - matcher: "__kf_tags"
              op: "not in"
              value: "severity"
            - matcher: "#agent"
              op: "=="
              value: "kinesis"
            - matcher: "#source"
              op: "contains"
              value: "guardduty"
            - matcher: "@detail_severity"
              op: "=="
              value: "2"
      - transform:
          tag: "severity"
          args:
            - action: "replace"
              targetLabel: "severity_string"
              replacement: "Medium"
          conditions:
            - matcher: "__kf_tags"
              op: "not in"
              value: "severity"
            - matcher: "#agent"
              op: "=="
              value: "kinesis"
            - matcher: "#source"
              op: "contains"
              value: "guardduty"
            - matcher: "@detail_severity"
              op: "=="
              value: "5"
      - transform:
          tag: "severity"
          args:
            - action: "replace"
              targetLabel: "severity_string"
              replacement: "High"
          conditions:
            - matcher: "__kf_tags"
              op: "not in"
              value: "severity"
            - matcher: "#agent"
              op: "=="
              value: "kinesis"
            - matcher: "#source"
              op: "contains"
              value: "guardduty"
            - matcher: "@detail_severity"
              op: "=="
              value: "8"
      - transform:
          tag: "severity"
          args:
            - action: "replace"
              targetLabel: "severity_string"
              replacement: "Critical"
          conditions:
            - matcher: "__kf_tags"
              op: "not in"
              value: "severity"
            - matcher: "#agent"
              op: "=="
              value: "kinesis"
            - matcher: "#source"
              op: "contains"
              value: "guardduty"
            - matcher: "@detail_severity"
              op: "=="
              value: "9.5"
      - remap:
          args:
            kf_additional_tags:
              - "$.origin"
              - "$.application_id"
              - "$.session_id"
              - "$.view.id"
              - "$.view.url"
              - "$.view.referrer"
              - "$.error.handling"
              - "$.error.kind"
              - "$.error.stack"
              - "$.error.message"
              - "$.http.method"
              - "$.http.status_code"
              - "$.http.url"
          conditions:
            - matcher: "__kf_agent"
              value: "datadog"
              op: "=="
      - transform:
          args:
            - action: "replace"
            - sourceLabels: "#ddsource"
            - targetLabel: "source"
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "error.kind"
            - facetName: "error_kind"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "error.stack"
            - facetName: "error_stack"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "error.message"
            - facetName: "error_message"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "http.method"
            - facetName: "@http_method"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "http.status_code"
            - facetName: "http_status_code"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "http.url"
            - facetName: "http_url"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "session_id"
            - facetName: "sessionid"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "application_id"
            - facetName: "applicationid"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "view.url"
            - facetName: "view_url"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "view.referrer"
            - facetName: "view_referrer"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "view.id"
            - facetName: "view_id"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "error.handling"
            - facetName: "@error_handling"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
      - moveLabelToFacet:
          args:
            - name: "origin"
            - facetName: "event_origin"
            - agentFacet: true
          conditions:
            - matcher: "#__kf_ddrum"
              value: "true"
              op: "=="
    yaml

  8. Enable RUM in the ui section of the custom-values.yaml file:

    ui:
  config:
    rum:
      enabled: true
    yaml

  9. Upgrade the cluster using these changes, and ensure that all pods are active.

Step 1 Instrument the Frontend Application

To set up the Frontend SDK for RUM, see instructions in RUM Setup.

Here is guide to the parameters to configure for SDK initialization:

applicationId

applicationId: '<APPLICATION_ID>'

Must match the application id defined above under the customer yaml global.rum.applications section

clientToken

clientToken: '<CLIENT_TOKEN>'

Use the string dummy as a value if the Kloudfuse installation does not have authenticated ingest enabled. If authenticated ingestion is enabled please follow the additional instructions noted on Step 5: Enable authenticated ingest below.

site

site: '<SITE>'

Use empty string.

proxy

proxy: '<KFUSE_RUM_ENDPOINT>'

Must be of the form https://<customers-kfuse-hostname>/ddrumproxy.

Example: Kloudfuse Playground cluster has the value of https://playground.kloudfuse.io/ddrumproxy.

service

service: '<APPLICATION_NAME>'

Must match the application name, and cannot contain any white space.

Example: kf-frontend

env

env: 'production'

This value depends on the type of deployment: test, production, staging, and so on.

version

version: '1.0.0'

If the frontend application has an identifier, use a string value that represents the version.

Otherwise, use the default value of 1.0.0.

sessionSampleRate

sessionSampleRate: 100

We recommend a small number if the site has many users.

enableSessionRecording

enableSessionRecording: true

We recommend a value of true to use the session capture and replay feature.

enableLogCollection

enableLogCollection: true

We recommend a value of true to get frontend logs into their Kloudfuse cluster.

Step 1 Enable Authenticated Ingestion

(Optional)

By default, this feature is disabled.

When you enable authenticated ingestion on the Kloudfuse platform, you must enable it globally, and add authentication tokens.

Follow these steps:

  1. To enable ingestion authentication, set global authConfig to true in the custom-values.yaml file:

    Enable Ingestion Authentication
    global:
  authConfig:
    enabled: true (1)
    yaml
    1 authConfig: enable

  2. Generate a new authentication token:

    Generate token
    uuidgen | tr -d '-' | tr 'A-Z' 'a-z' | sed 's/^/pub/'

  3. Configure one or more client tokens in the ingester config rum section of the custom-values.yaml file:

    Configure tokens
    ingester:
  config:
    rum:
      enabled: true (1)
      clientTokens: (2)
        - rumauthkey1: <unique authtoken value>
        - rumauthkey2: <unique authtoken value>
    yaml
    1 enabled: Enable RUM ingestion
    2 clientTokens rumauthkey1 and rumauthkey2: Unique authentication tokens for RUM, generated in Generate token.

  4. Match the generated auth tokens to the clientToken value in the frontend RUM SDK configuration:

    kfuseRumSDK.init({
    config: {
        applicationId: '<application id>', (1)
        clientToken: '<unique authtoken value>', (2)
        ...
    }
})
    javascript
    1 applicationId: The identifier of the service that provides the RUM stream.
    2 clientToken: Unique authentication token for RUM, generated in Generate token, matching the declarations in Configure tokens.