Privacy Filter🔗

The Privacy Filter is an optional service that redacts personal data (PII) from text before Lightrun stores it. It runs an AI model (openai/privacy-filter) on a GPU and replaces private values with placeholders, e.g. Email <PRIVATE_EMAIL>.

When enabled, the backend sends it the texts of snapshots and logs collected from agents, and stores only the redacted result.

[!NOTE] Privacy Filter is disabled by default and requires RabbitMQ (general.mq.enabled: true). It needs an NVIDIA GPU node.

How it works🔗

                      RabbitMQ
                ┌──────────────────────────┐
Backend ──────► │ privacy-filter-requests  │ ──────► Privacy Filter
Backend ◄────── │ privacy-filter-responses │ ◄──────
                └──────────────────────────┘

Privacy Filter communicates only through two RabbitMQ queues: it reads requests from privacy-filter-requests and writes replies to privacy-filter-responses.

1. The backend publishes the texts to redact to the request queue (in chunks of up to 1000 texts).
2. A Privacy Filter pod consumes them, runs the model and publishes the redacted texts, in the same order, to the response queue. Replies carry the request's correlation_id and an opaque context, so neither side keeps per-request state.
3. The backend consumes the reply and stores the data. A snapshot waits as a pending record, holding no personal data, until all its texts are back.

Privacy Filter has no inbound traffic and no Kubernetes Service: it only connects out to RabbitMQ. Its HTTP port (8786) serves just the /health and /ready probes.

Failure behavior🔗

Nothing is ever stored unredacted.

Enable Privacy Filter🔗

general:
  mq:
    enabled: true
  privacy_filter:
    enabled: true

This deploys the Privacy Filter pod, creates its queues and tells the backend and crons to use it.

[!NOTE] The chart enables the infrastructure. Whether a company's data is actually filtered is controlled in the Lightrun server: the feature must be rolled out and the company must opt in.

Queues🔗

The chart declares the queues; the service only checks that they exist.

Names and limits are under general.mq.privacy_filter in values.yaml.

While Privacy Filter is enabled, RabbitMQ uses larger resources (general.mq.privacy_filter.rabbitmq: 1 CPU, 4Gi memory, 10Gi storage) instead of the regular deployments.rabbitmq values, so bursts of large texts can queue up in the broker.

Configuration🔗

Settings are under deployments.privacy_filter.

deployments:
  privacy_filter:
    replicas: 1
    image:
      repository: lightruncom/privacy-filter
      tag: ""
    resources:
      requests: { cpu: "2", memory: 5Gi }
      limits:   { cpu: "3", memory: 6Gi }
    nodeSelector: {}
    tolerations: []

Other options (extraEnvs, probes, security contexts, PDB, affinity, ...) work as for the other components.

GPU node🔗

Run Privacy Filter on its own GPU node: it runs one container per GPU and uses the whole node. The pod sets NVIDIA_VISIBLE_DEVICES=all and NVIDIA_DRIVER_CAPABILITIES=compute,utility, so on a node with the NVIDIA container runtime no GPU resource request or RuntimeClass is needed. Select the node and tolerate its taint through nodeSelector and tolerations, for example:

deployments:
  privacy_filter:
    nodeSelector:
      workload: privacy-filter
    tolerations:
      - key: workload
        operator: Equal
        value: privacy-filter
        effect: NoSchedule

The default resources suit a dedicated node as small as AWS g6f.2xlarge (8 vCPU, 32 GiB, a quarter of an L4 GPU).

[!IMPORTANT] On a fractional GPU such as g6f, set OPF_GPU_MEMORY_UTILIZATION to "0.75"; the default 0.9 does not fit.

deployments:
  privacy_filter:
    extraEnvs:
      - name: OPF_GPU_MEMORY_UTILIZATION
        value: "0.75"

Tuning🔗

The service reads OPF_* environment variables, set through extraEnvs. Start with the defaults; these are the ones worth changing:

To process more, add replicas: they consume from the same queue, one GPU node each.

Startup🔗

The pod waits for RabbitMQ (wait-for-rabbitmq init container), then loads the model, which is embedded in the image and takes a few minutes. The pod becomes ready once it is consuming from the request queue.