Traefik2.2从坑出发

Traefik2.2安装

---

Traefik 2.2新增的功能如下：

1. 支持了udp
2. traefik2.2 支持使用K/V存储做为动态配置的源，分别是 consul, etcd, Redis, zookeeper
3. 能够使用kubernetes CRD自定义资源定义UDP负载平衡 Ingre***outeUDP。
4. 能够使用 rancher， consul catalog， docker和 marathon中的标签定义UDP的负载平衡
5. 增加了对ingress注解的主持
6. 将TLS存储功能 TLSStores添加到Kubernetes CRD中，使kubernetes用户无需使用配置文件和安装证书即可提供默认证书。
7. 在日志中增加了http的请求方式,是http还是https
8. 因为TLS的配置可能会影响CPU的使用率，因此增加了 TLS version和 TLS cipher使用的指标信息
9. 当前的WRR算法对于权重不平衡端点存在严重的偏差问题，将EDF调度算法用于WeightedRoundRobin， Envoy也是使用了 EOF调度算法
10. 支持请求主体用于流量镜像
11. 增加了 ElasticAPM作为traefik的tracing系统。
12. Traefik的Dashboard增加了UDP的页面
13. Traefik也增加了黑暗主题

下面进行安装过程。

注：我们这里是将traefik部署在ingress-traefik命名空间，如果你需要部署在其他命名空间，需要更改资源清单，如果你是部署在和我同样的命令空间中，你需要创建该命名空间。

创建命名空间：

# kubectl create ns ingress-traefik

1、创建CRD资源

---

Traefik 2.0版本后开始使用CRD来对资源进行管理配置，所以我们需要先创建CRD资源。
traefik-crd.yaml

## Ingre***outeapiVersion: apiextensions.k8s.io/v1beta1kind: CustomResourceDefinitionmetadata:  name: ingre***outes.traefik.containo.usspec:  scope: Namespaced  group: traefik.containo.us  version: v1alpha1  names:    kind: Ingre***oute    plural: ingre***outes    singular: ingre***oute---## Ingre***outeTCPapiVersion: apiextensions.k8s.io/v1beta1kind: CustomResourceDefinitionmetadata:  name: ingre***outetcps.traefik.containo.usspec:  scope: Namespaced  group: traefik.containo.us  version: v1alpha1  names:    kind: Ingre***outeTCP    plural: ingre***outetcps    singular: ingre***outetcp---## MiddlewareapiVersion: apiextensions.k8s.io/v1beta1kind: CustomResourceDefinitionmetadata:  name: middlewares.traefik.containo.usspec:  scope: Namespaced  group: traefik.containo.us  version: v1alpha1  names:    kind: Middleware    plural: middlewares    singular: middleware---apiVersion: apiextensions.k8s.io/v1beta1kind: CustomResourceDefinitionmetadata:  name: tlsoptions.traefik.containo.usspec:  scope: Namespaced  group: traefik.containo.us  version: v1alpha1  names:    kind: TLSOption    plural: tlsoptions    singular: tlsoption---## TraefikServiceapiVersion: apiextensions.k8s.io/v1beta1kind: CustomResourceDefinitionmetadata:  name: traefikservices.traefik.containo.usspec:  scope: Namespaced  group: traefik.containo.us  version: v1alpha1  names:    kind: TraefikService    plural: traefikservices    singular: traefikservice---## TraefikTLSStoreapiVersion: apiextensions.k8s.io/v1beta1kind: CustomResourceDefinitionmetadata:  name: tlsstores.traefik.containo.usspec:  scope: Namespaced  group: traefik.containo.us  version: v1alpha1  names:    kind: TLSStore    plural: tlsstores    singular: tlsstore---## Ingre***outeUDPapiVersion: apiextensions.k8s.io/v1beta1kind: CustomResourceDefinitionmetadata:  name: ingre***outeudps.traefik.containo.us spec:  scope: Namespaced  group: traefik.containo.us  version: v1alpha1  names:    kind: Ingre***outeUDP    plural: ingre***outeudps    singular: ingre***outeudp

创建资源清单：

# kubectl apply -f traefik-crd.yaml# kubectl get crdNAME                                    CREATED ATingre***outes.traefik.containo.us       2019-12-13T05:40:30Zingre***outetcps.traefik.containo.us    2019-12-13T05:40:30Zmiddlewares.traefik.containo.us         2019-12-13T05:40:30Z

2、创建RBAC权限

---

traefik-rbac.yaml

apiVersion: v1kind: ServiceAccountmetadata:  namespace: ingress-traefik   name: traefik-ingress-controller---kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1beta1metadata:  name: traefik-ingress-controllerrules:  - apiGroups: [""]    resources: ["services","endpoints","secrets"]    verbs: ["get","list","watch"]  - apiGroups: ["extensions"]    resources: ["ingresses"]    verbs: ["get","list","watch"]  - apiGroups: ["extensions"]    resources: ["ingresses/status"]    verbs: ["update"]  - apiGroups: ["traefik.containo.us"]    resources: ["middlewares"]    verbs: ["get","list","watch"]  - apiGroups: ["traefik.containo.us"]    resources: ["ingre***outes","traefikservices"]    verbs: ["get","list","watch"]  - apiGroups: ["traefik.containo.us"]    resources: ["ingre***outetcps","ingre***outeudps"]    verbs: ["get","list","watch"]  - apiGroups: ["traefik.containo.us"]    resources: ["tlsoptions","tlsstores"]    verbs: ["get","list","watch"]---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1beta1metadata:  name: traefik-ingress-controllerroleRef:  apiGroup: rbac.authorization.k8s.io  kind: ClusterRole  name: traefik-ingress-controllersubjects:  - kind: ServiceAccount    name: traefik-ingress-controller    namespace: ingress-traefik

创建RBAC资源清单：

# kubectl apply -f traefik-rbac.yaml

3、创建traefik配置文件

---

traefik-config.yaml

kind: ConfigMapapiVersion: v1metadata:  name: traefik-config  namespace: ingress-traefikdata:  traefik.yaml: |-    serversTransport:      insecureSkipVerify: true    api:      insecure: true      dashboard: true      debug: true    metrics:      prometheus: ""    entryPoints:      web:        address: ":80"      websecure:        address: ":443"    providers:      kubernetesCRD: ""    log:      filePath: ""      level: error      format: json    accessLog:      filePath: ""      format: json      bufferingSize: 0      filters:        retryAttempts: true        minDuration: 20      fields:        defaultMode: keep        names:          ClientUsername: drop        headers:          defaultMode: keep          names:            User-Agent: redact            Authorization: drop            Content-Type: keep

创建ConfigMap:

# kubectl apply -f traefik-config.yaml

4、给节点打标签

---

由于是 Kubernetes DeamonSet 这种方式部署 Traefik，所以需要提前给节点设置 Label，这样当程序部署时 Pod 会自动调度到设置 Label 的点上。

节点设置 Label 标签

• 格式：kubectl label nodes [节点名] [key=value]

  kubectl label nodes 172.16.0.33 IngressProxy=true

查看节点标签：

# kubectl get nodes --show-labels NAME          STATUS                     ROLES    AGE   VERSION   LABELS172.16.0.33   Ready,SchedulingDisabled   master   20d   v1.15.0   IngressProxy=true,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/fluentd-ds-ready=true,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=172.16.0.33,kubernetes.io/os=linux,kubernetes.io/role=master172.16.0.52   Ready                      node     20d   v1.15.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/fluentd-ds-ready=true,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=172.16.0.52,kubernetes.io/os=linux,kubernetes.io/role=node

5、部署traefik

---

很多时候我们会采用DS方式部署，并且设置网络为hostNetwork=True，这样方便流量进入。但是在这里我采用的是service的nodeport进行暴露。

（1）、Service的配置清单
traefik-svc.yaml

apiVersion: v1kind: Servicemetadata:  name: traefik  namespace: ingress-traefikspec:  type: NodePort  ports:    - name: web      port: 80    - name: websecure      port: 443    - name: admin      port: 8080  selector:    app: traefik

（2）、DS的配置清单
traefik-ds.yaml

apiVersion: apps/v1kind: DaemonSetmetadata:  name: traefik-ingress-controller  namespace: ingress-traefik  labels:    app: traefikspec:  selector:    matchLabels:      app: traefik  template:    metadata:      name: traefik      labels:        app: traefik    spec:      serviceAccountName: traefik-ingress-controller      terminationGracePeriodSeconds: 1      containers:        - image: registry.cn-hangzhou.aliyuncs.com/rookieops/traefik:v2.2.0          name: traefik-ingress-lb          ports:            - name: web              containerPort: 80            - name: websecure              containerPort: 443            - name: admin              containerPort: 8080          resources:            limits:              cpu: 2000m              memory: 1024Mi            requests:              cpu: 1000m              memory: 1024Mi          securityContext:            capabilities:              drop:                - ALL              add:                - NET_BIND_SERVICE          args:            - --configfile=/config/traefik.yaml          volumeMounts:            - mountPath: "/config"              name: "config"      volumes:        - name: config          configMap:            name: traefik-config       tolerations:              #设置容忍所有污点，防止节点被设置污点        - operator: "Exists"      nodeSelector:             #设置node筛选器，在特定label的节点上启动        IngressProxy: "true"

3) 配置traefik路由规则

apiVersion: traefik.containo.us/v1alpha1kind: Ingre***outemetadata:  name: traefik-dashboard-route  namespace: ingress-traefikspec:  entryPoints:    - web  routes:    - match: Host(`traefik.coolops.cn`)      kind: Rule      services:        - name: traefik          port: 8080

然后创建资源：

# kubectl apply -f traefik-svc.yaml# kubectl apply -f traefik-ds.yaml# kubectl get svc -n ingress-traefik NAME      TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                                     AGEtraefik   NodePort   10.102.225.66   <none>        80:30161/TCP,443:30629/TCP,8080:32359/TCP   31m# kubectl get pod -n ingress-traefik NAME                               READY   STATUS    RESTARTS   AGEtraefik-ingress-controller-whbjm   1/1     Running   0          22m

然后本地配置hosts

10.1.10.128 traefik.coolops.cn

由于我们ingress是通过svc的nodeport暴露的，所以输入以下访问

并且可以看到我配置的HTTP

6、配置SSL

---

这里使用Let's Encrypt 来进行自动化 HTTPS。
修改配置文件，新增如下内容：

    certificatesresolvers:      default:        acme:          tlsChallenge: {}          email: "coolops@163.com"          storage: "acme.json"

更新配置文件。

新增一个https的ingre***oute，如下：

apiVersion: traefik.containo.us/v1alpha1kind: Ingre***outemetadata:  name: traefik-webui-tls  namespace: ingress-traefik spec:  entryPoints:  - websecure  # 注意这里是websecure这个entryPoint，监控443端口  routes:  - match: Host(`traefik.coolops.cn`)    kind: Rule    services:    - name: traefik      port: 8080  tls:    certResolver: default 

查看pod的日志信息，报错如下：

{"level":"error","msg":"Unable to obtain ACME certificate for domains \"traefik.coolops.cn\": cannot get ACME client get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp 172.65.32.248:443: connect: connection refused","providerName":"default.acme","routerName":"ingress-traefik-traefik-webui-tls-fcde00a088d29eefb3a6@kubernetescrd","rule":"Host(`traefik.coolops.cn`)","time":"2020-05-26T02:49:49Z"}

这是因为pod里的网络和https://acme-v02.api.letsencrypt.org/directory 不通造成的。

部署成功后可以使用HTTPS访问了。

上面是自动生成证书，如果有自己的域名证书，那么一切都简单了，你只需要配置一个secret，然后在ingre***oute中引用即可，比如下面来自官方的例子：

apiVersion: v1kind: Secretmetadata:  name: supersecretdata:  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0=  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCi0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0=---apiVersion: traefik.containo.us/v1alpha1kind: Ingre***outemetadata:  name: ingre***outetlsspec:  entryPoints:    - web  routes:  - match: Host(`foo.com`) && PathPrefix(`/bar`)    kind: Rule    services:    - name: whoami      port: 443  tls:    secretName: supersecret

用Let's Encrypt的话，虽然免费，用起来还是有坑，这就需要自己去踩了~~！

完

---