Quick Guide for Filebeat Agent for Logs

A lightweight shipper for forwarding and centralizing log data, Filebeat agent installs on your servers, monitors the log files or locations that you specify, collects log events, and forwards them either to ElasticSearch or LogStash for indexing. See Filebeat overview in Elastic documentation.

Install on Helm

  1. Get the IP for the Kloudfuse endpoint.

    If Kloudfuse stack installs with a DNS or VPC Endpoint, use that address instead of the IP address.

    Pick the ingress-external IP.

    kubectl get services | grep -i "ingress"

  2. Create a custom_values.yaml file, or update the one you have to install the Filebeat agent. Ensure that you copy over the existing filebeatConfig code, and avoid overwriting the filebeat.yml content.

    daemonset: # Use deployment, instead of daemonset, if filebeat is configured to run in deployment-mode.
filebeatConfig:
filebeat.yml: |
...
setup.ilm.enabled: false
setup.template.enabled: false
output.elasticsearch:
hosts: ["http://<ingress-ip or DNS>/ingester/api/v1/filebeat"] # use http or https depending on whether Kfuse has https enabled
    yaml

  3. Add the helm repository

    helm repo add elastic https://helm.elastic.co

  4. Using custom_values.yaml file, install Filebeat agent.

    helm upgrade --install filebeat elastic/filebeat -f custom_values.yaml