Kubernetes WebSocket: Error during WebSocket handshake: Unexpected response code: 403

Problem

笔者从 Vue 中建立 ws 连接访问 Kubernetes WebSocket 获取容器日志，得到此问题：

WebSocket connection to 'ws://xx.xx.xx.xx:8080/api/v1/namespaces/default/pods/test-1/log?follow=true&tailLines=1000&timestamps=true&container=test-1' failed: 
Error during WebSocket handshake: Unexpected response code: 403Error during WebSocket handshake: Unexpected response code: 403

Solution

在和同事讨论以及查阅了一些资料后，发现在连接 ws 连接时缺少了 Kubernetes WebSocket 需要遵循的子协议：

Kubernetes WebSocket 接口遵循 WebSocke t协议，并在此基础上设计了子协议，在编写WebSocket客户端与Kubernetes通信时，必须选择并遵守其中一种协议。该子协议协议并没有官方文档，但我们可以从k8s的代码（ https://github.com/kubernetes/apiserver/blob/master/pkg/util/wsstream/conn.go ）中找到相关说明：

// the channel number (zero indexed) the message was sent on. Messages in both directions should
// prefix their messages with this channel byte. When used for remote execution, the channel numbers
// are by convention defined to match the POSIX file-descriptors assigned to STDIN, STDOUT, and STDERR
// (0, 1, and 2). No other conversion is performed on the raw subprotocol - writes are sent as they
// are received by the server.
//
// Example client session:
//
//    CONNECT http://server.com with subprotocol "channel.k8s.io"
//    WRITE []byte{0, 102, 111, 111, 10} # send "foo
" on channel 0 (STDIN)
//    READ  []byte{1, 10}                # receive "
" on channel 1 (STDOUT)
//    CLOSE
//
const ChannelWebSocketProtocol = "channel.k8s.io"

// The Websocket subprotocol "base64.channel.k8s.io" base64 encodes each message with a character
// indicating the channel number (zero indexed) the message was sent on. Messages in both directions
// should prefix their messages with this channel char. When used for remote execution, the channel
// numbers are by convention defined to match the POSIX file-descriptors assigned to STDIN, STDOUT,
// and STDERR ('0', '1', and '2'). The data received on the server is base64 decoded (and must be
// be valid) and data written by the server to the client is base64 encoded.
//
// Example client session:
//
//    CONNECT http://server.com with subprotocol "base64.channel.k8s.io"
//    WRITE []byte{48, 90, 109, 57, 118, 67, 103, 111, 61} # send "foo
" (base64: "Zm9vCgo=") on channel '0' (STDIN)
//    READ  []byte{49, 67, 103, 61, 61} # receive "
" (base64: "Cg==") on channel '1' (STDOUT)
//    CLOSE
//
const Base64ChannelWebSocketProtocol = "base64.channel.k8s.io"

另外，在一个 k8s client 项目 kubebox 中，并没有使用这两个子协议，而是使用了 binary.k8s.io（(https://github.com/astefanutti/kubebox/blob/master/lib/client.js#L204)[https://github.com/astefanutti/kubebox/blob/master/lib/client.js#L204]），和同事讨论之后，得到一些猜测：

普通的接口用websocket 都是 json 不需要做子协议，log 本质是二进制流，用 binary 子协议；而 exec 是双向用channel。

笔者最终采用了这个子协议解决问题：

initSocket() {
  const WebSocketUrl = "ws://xx.xx.xx.xx:8080/api/v1/namespaces/default/pods/test-1/log?follow=true&tailLines=1000&timestamps=true&container=test-1"
  this.socket = new WebSocket(WebSocketUrl, 'binary.k8s.io')
  ...
}

参考资料：
http://blog.allen-mo.com/2018/04/17/kubernetes_websocket/
https://github.com/astefanutti/kubebox/blob/master/lib/client.js#L204
https://github.com/kubernetes/apiserver/blob/master/pkg/util/wsstream/conn.go