WebSockets

This example walks through some of the ways that Envoy can be configured to proxy WebSockets.

It demonstrates terminating a WebSocket connection with and without TLS, and provides some basic examples of proxying to encrypted and non-encrypted upstream sockets.

Warning

For the sake of simplicity, the examples provided here do not authenticate any client certificates, or validate any of the provided certificates.

When using TLS, you are strongly encouraged to validate all certificates wherever possible.

You should also authenticate clients where you control both sides of the connection, or relevant protocols are available.

Step 1: Create a certificate file for wss

Change directory to examples/websocket in the Envoy repository.

$ pwd
envoy/examples/websocket
$ mkdir -p certs
$ openssl req -batch -new -x509 -nodes -keyout certs/key.pem -out certs/cert.pem
Generating a RSA private key
..................................................................................................................+++++
......+++++
writing new private key to 'certs/key.pem'
-----
$ openssl pkcs12 -export -passout pass: -out certs/output.pkcs12 -inkey certs/key.pem -in certs/cert.pem

Step 2: Build and start the sandbox

This starts three proxies listening on localhost ports 10000-30000.

It also starts two upstream services, one ws and one wss.

The upstream services listen on the internal Docker network on ports 80 and 443 respectively.

The socket servers are very trivial implementations, that simply output [ws] HELO and [wss] HELO in response to any input.

$ docker-compose pull
$ docker-compose up --build -d
$ docker-compose ps

            Name                             Command               State            Ports
---------------------------------------------------------------------------------------------------
websocket_proxy-ws_1                /docker-entrypoint.sh /usr ... Up      0.0.0.0:10000->10000/tcp
websocket_proxy-wss_1               /docker-entrypoint.sh /usr ... Up      0.0.0.0:20000->10000/tcp
websocket_proxy-wss-passthrough_1   /docker-entrypoint.sh /usr ... Up      0.0.0.0:30000->10000/tcp
websocket_service-ws_1              websocat -E ws-listen:0.0. ... Up
websocket_service-wss_1             websocat wss-listen:0.0.0. ... Up

Step 3: Test proxying ws -> ws

The proxy listening on port 10000 terminates the WebSocket connection without TLS and then proxies to an upstream socket, also without TLS.

In order for Envoy to terminate the WebSocket connection, the upgrade_configs in HttpConnectionManager must be set, as can be seen in the provided ws -> ws configuration:

 1static_resources:
 2  listeners:
 3  - address:
 4      socket_address:
 5        address: 0.0.0.0
 6        port_value: 10000
 7    filter_chains:
 8    - filters:
 9      - name: envoy.filters.network.http_connection_manager
10        typed_config:
11          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
12          stat_prefix: ingress_ws_to_ws
13          upgrade_configs:
14          - upgrade_type: websocket
15          route_config:
16            name: local_route
17            virtual_hosts:
18            - name: app
19              domains:
20              - "*"
21              routes:
22              - match:
23                  prefix: "/"
24                route:
25                  cluster: service_ws
26          http_filters:
27          - name: envoy.filters.http.router
28            typed_config:
29              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

You can start an interactive session with the socket as follows:

$ docker run -ti --network=host solsson/websocat ws://localhost:10000
HELO
[ws] HELO
GOODBYE
[ws] HELO

Type Ctrl-c to exit the socket session.

Step 4: Test proxying wss -> wss

The proxy listening on port 20000 terminates the WebSocket connection with TLS and then proxies to an upstream TLS WebSocket.

In addition to the upgrade_configs in HttpConnectionManager, the wss -> wss configuration adds a TLS transport_socket to both the listener and the cluster.

You can start an interactive session with the socket as follows:

$ docker run -ti --network=host solsson/websocat --insecure wss://localhost:20000
HELO
[wss] HELO
GOODBYE
[wss] HELO

Type Ctrl-c to exit the socket session.

Step 5: Test proxying wss passthrough

The proxy listening on port 30000 passes through all TCP traffic to an upstream TLS WebSocket.

The wss passthrough configuration requires no TLS or HTTP setup, and instead uses a simple tcp_proxy.

You can start an interactive session with the socket as follows:

$ docker run -ti --network=host solsson/websocat --insecure wss://localhost:30000
HELO
[wss] HELO
GOODBYE
[wss] HELO

Type Ctrl-c to exit the socket session.

See also

Securing Envoy quick start guide

Outline of key concepts for securing Envoy.

Double proxy sandbox

An example of securing traffic between proxies with validation and mutual authentication using mTLS with non-HTTP traffic.

TLS sandbox

Examples of various TLS termination patterns with Envoy.