Envoy Gateway Operator

TOC

Understanding Envoy Gateway

Terminology

  • Envoy Gateway is an open-source project for managing Envoy Proxy as a standalone or Kubernetes-based application gateway.
  • Gateway API is a Kubernetes-official collection of custom resources for declaring routing rules and traffic management policies. Envoy Gateway is one implementation of the Gateway API specification.
  • Gateway or gatewayapi/gateway is a CR defined by the Gateway API specification.
  • envoy-gateway instance is Envoy Gateway instance and all its related resources running in a Kubernetes cluster.
  • envoy-gateway-operator is a helm-operator wrapper around Envoy Gateway and use EnvoyGatewayCtl to simplify the deployment and management of envoy-gateway instance.
  • EnvoyProxy is a CR defined by the Envoy Gateway specification, used by envoy-gateway instance to manage envoy-proxy instance.
  • envoy-proxy instance is a running deployment which handles incoming traffic.

Architecture

  1. After installing the envoy-gateway-operator, it watches EnvoyGatewayCtl and deploys an envoy-gateway instance along with a default GatewayClass.

  2. When you create a Gateway that references this GatewayClass and specifies an EnvoyProxy configuration, the envoy-gateway instance deploys an envoy-proxy instance to handle incoming traffic.

  3. HTTP, TCP, or UDP Route resources can then be created and attached to this Gateway to define traffic routing policies.

  4. For more granular configuration, you can create ClientTrafficPolicy or BackendTrafficPolicy resources and attach them to routes to implement advanced traffic management features.

Envoy Gateway and envoy-gateway-operator introduce additional custom resources that, combined with the Gateway API custom resource, comprehensively control all aspects of gateway deployment and routing policy configurations.

Custom resourcesDescription
GatewayClassAn envoy-gateway instance knows which GatewayClasses it manages, and thereby knows which Gateways it should control.
GatewayDefines three essential components:
1. GatewayClass reference - References the GatewayClass that controls this Gateway via .spec.gatewayClassName
2. Listeners configuration - Specifies which ports, hostnames, and TLS certificates to use for traffic handling via .spec.listeners
3. Infrastructure parameters - Configures deployment specifics like replicas, resources by referencing an EnvoyProxy via .spec.infrastructure.parametersRef
HTTP/TCP/UDP RouteDefines four essential components:
1. Multiple matching rule sets - Various conditions to match incoming traffic
2. Backend destinations - Target services for each matching rule set
3. Traffic policies - Per-rule configurations including retries, timeouts, redirects, etc.
4. Gateway listeners attachment - Defined via .spec.parentRefs to connect routes to specific Gateway listeners

For more details about these resources, please refer to the Gateway API Concepts.

Custom resourcesDescription
EnvoyProxyDefines the configuration for envoy-proxy instances, such as replicas, resources, and scheduling constraints

In the recommended deployment pattern, each Gateway uses .spec.infrastructure.parametersRef to reference its own dedicated EnvoyProxy resource.

This one-to-one mapping allows users to control the Gateway's deployment configuration (replicas, resources, scheduling, etc.) by modifying the corresponding EnvoyProxy resource.

Custom resourcesDescription
EnvoyGatewayCtlControls the deployment of a specific envoy-gateway instance.

We recommend deploying a single envoy-gateway instance via EnvoyGatewayCtl per cluster, which is sufficient for typical use cases.

However, the EnvoyGatewayCtl resource allows you to deploy multiple envoy-gateway instance within the same cluster, enabling more flexible control and isolation for advanced scenarios such as different deployment mode

Install Envoy Gateway via Envoy Gateway Operator

Prerequisites

Please ensure that you have read the Understanding Envoy Gateway documentation before proceeding.

Overview

The envoy-gateway-operator is provided to facilitate advanced management of envoy-gateway instance deployments.

The operational architecture functions as follows:

the envoy-gateway-operator continuously monitors EnvoyGatewayCtl and automatically provisions corresponding envoy-gateway instance deployment, service, etc.

Installation

Step 1: Install Envoy Gateway Operator

  1. Navigate to Administrator -> Marketplace -> OperatorHub
  2. Locate the Alauda build of Envoy Gateway under Networking Category and click Install

Step 2: Create a Default EnvoyGatewayCtl

  1. Navigate to Administrator -> Marketplace -> OperatorHub

  2. Locate the Alauda build of Envoy Gateway under Networking Category ,click to Open it.

  3. Navigate to the All Instances tab

  4. Click Create, then in the displayed dialog box, locate the EnvoyGatewayCtl Instance Type section and click Create

  5. The Create EnvoyGatewayCtl page displays the following configuration options:

    FieldDefault ValueDescriptionYAML Path
    Namecpaas-defaultName of the Envoy Gateway instance.metadata.name
    Namespaceenvoy-gateway-operatorNamespace of the Envoy Gateway instance, each ns could only have one envoygatewayctl.metadata.namespace
    Replica1The replica count of the Envoy Gateway deployment.spec.controller.replicaCount
    Deploy Ns ModeControllerNamespaceDetermines whether the gateway deployment is installed in the gateway namespace or the controller namespace, please refer to deployment modespec.config.envoyGateway.provider.kubernetes.deploy.type
    Resources1C1GThe resource requests and limits of the Envoy Gateway deployment.spec.controller.resources

    Recommendation: Use the default values for typical deployments. Click Create to proceed.

Configuration Via YAML

You could also apply EnvoyGatewayCtl via yaml.

Default Config:

apiVersion: envoy-gateway.alauda.io/v1
kind: EnvoyGatewayCtl
metadata:
  name: cpaas-default
  namespace: envoy-gateway-operator
spec:
  config:
    envoyGateway:
      provider:
        kubernetes:
          deploy:
            type: ControllerNamespace
        type: Kubernetes
  deployment:
    resources:
      limits:
        cpu: '1'
        memory: 1024Mi
      requests:
        cpu: 100m
        memory: 256Mi

Each EnvoyGatewayCtl will create a GatewayClass named as $NS-$NAME, for example envoy-gateway-operator-cpaas-default.

Advanced Config Via EnvoyGatewayCtl

EnvoyGatewayCtl is essentially the envoy-gateway Helm chart wrapped with helm-operator. Its spec corresponds to the values of the envoy-gateway chart, allowing you to configure various global-level features of Envoy Gateway in one source.

For detailed information about available configuration options, please refer to envoy-gateway-chart-values.

DANGER

You should only configure the envoy-gateway instance through the EnvoyGatewayCtl resource. Do not directly modify rendered resources such as ConfigMaps, Deployments, as these changes may be lost during upgrades or reconciliation.

Backend Routing

For example you can configure backend routing in the EnvoyGatewayCtl spec:

apiVersion: envoy-gateway.alauda.io/v1
kind: EnvoyGatewayCtl
metadata:
  name: cpaas-default
  namespace: envoy-gateway-operator
spec:
  config:
    envoyGateway:
      extensionApis:
        enableBackend: true # Enable backend routing in EnvoyGatewayCtl not the envoygateway configmap
      provider:
        kubernetes:
          deploy:
            type: ControllerNamespace
        type: Kubernetes
  deployment:
    resources:
      limits:
        cpu: '1'
        memory: 1024Mi
      requests:
        cpu: 100m
        memory: 256Mi