kubectl 源码概览

1802 阅读 0 评论 0 点赞

一、前言

kubectl 是 k8s 的客户端工具，不管是 k8s 的普通使用者还是 k8s 的开发人员，了解 kubectl 的具体实现能从中受益不少。

kubectl 本质是一个调用 k8s api-server 的 restful 工具，同时对于 k8s 开发人员你可以把他当作一个接口文档。在 kubectl 命令后面加上 -v 8 就能把命令执行过程的详细接口调用输出。刚入门学习 k8s 开发大部分对 k8s 接口还不熟悉的情况，这个方法很有用。同时 kubectl 不复杂对于入门学习 go 开发的也很有帮助。

adeMacBook-Pro:~ zhourj$ kubectl get ns -v 8
I1216 09:48:52.018615    1430 loader.go:372] Config loaded from file:  /Users/zhourj/.kube/config
I1216 09:48:52.027458    1430 round_trippers.go:432] GET https://172.18.26.118:6443/api/v1/namespaces?limit=500
I1216 09:48:52.027489    1430 round_trippers.go:438] Request Headers:
I1216 09:48:52.027497    1430 round_trippers.go:442]     Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json;as=Table;v=v1beta1;g=meta.k8s.io,application/json
I1216 09:48:52.027502    1430 round_trippers.go:442]     User-Agent: kubectl/v1.22.5 (darwin/amd64) kubernetes/5c99e2a
I1216 09:48:52.191987    1430 round_trippers.go:457] Response Status: 200 OK in 164 milliseconds
I1216 09:48:52.192013    1430 round_trippers.go:460] Response Headers:
I1216 09:48:52.192021    1430 round_trippers.go:463]     Date: Fri, 16 Dec 2022 01:48:52 GMT
I1216 09:48:52.192026    1430 round_trippers.go:463]     Audit-Id: a41f5321-47f7-453e-90bf-fbdaf9914b1d
I1216 09:48:52.192031    1430 round_trippers.go:463]     Cache-Control: no-cache, private
I1216 09:48:52.192035    1430 round_trippers.go:463]     Content-Type: application/json
I1216 09:48:52.192040    1430 round_trippers.go:463]     X-Kubernetes-Pf-Flowschema-Uid: c59365d7-f4be-4af6-9f86-b228212e6847
I1216 09:48:52.192045    1430 round_trippers.go:463]     X-Kubernetes-Pf-Prioritylevel-Uid: 3d9a0acc-996e-43b2-8acc-4d014ae63594
I1216 09:48:52.238408    1430 request.go:1181] Response Body: {"kind":"Table","apiVersion":"meta.k8s.io/v1","metadata":{"selfLink":"/api/v1/namespaces","resourceVersion":"56560505"},"columnDefinitions":[{"name":"Name","type":"string","format":"name","description":"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names","priority":0},{"name":"Status","type":"string","format":"","description":"The status of the namespace","priority":0},{"name":"Age","type":"string","format":"","description":"CreationTimestamp is a timestamp representing the server time when this object was created. It is not guaranteed to be set in happens-before order across separate operations. Clients may not set this value. It is represented in RFC3339 form and is in UTC.\n\nPopulated by the system. Read-only.  [truncated 7946 chars]

二、kubectl 工程脚手架

kubectl 代码总体来说不复杂，主要包含了2个要素。

- spf13/cobra：有名的命令行框架，docker 也是基于这个实现的。
- go-client: 基于 RESTClient 封装的 k8S 客户端

2.1 spf13/cobra

Cobra基于三个基本概念commands,arguments和flags。其中commands代表行为，arguments代表数值，flags代表对行为的改变。
APPNAME COMMAND ARG --FLAG

// get 是 command，pod 是 argument，-n devops 是 flags
kubectl get pod -n devops

2.2 go-client

直接使用 resetclient 也可以操作 k8s 但需要自己封装一系列的出入参对象，go-client 帮忙把这事做了。ClientSet是一组资源对象客户端的集合，比如操作pod、services等资源的CoreV1Client，负责操作Deploy、DaemonSet等资源的AppsV1Client等。通过这些资源对象客户端提供的操作方法，即可对K8s内置的资源对象进行Create、Update、Get、List、Delete等操作。

package main

import (
 "context"
 "fmt"
 v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 "k8s.io/client-go/kubernetes"
 "k8s.io/client-go/tools/clientcmd"
)

func main() {
    //1.加载配置文件，生成config对象
 config, err := clientcmd.BuildConfigFromFlags("", "F:\\GO\\src\\test\\config")
 if err != nil {
  fmt.Println(err)
 }
    //2.实例化clientset对象
 clientset, err := kubernetes.NewForConfig(config)
 if err != nil {
  panic(err.Error())
 }
    //3.使用clientset去查看kube-system命名空间下的pod列表
 pods, err := clientset.CoreV1().Pods("kube-system").List(context.TODO(), v1.ListOptions{})
 if err != nil {
  panic(err.Error())
 }
    //4.循环打印
 for _, item := range pods.Items {
  fmt.Println(item.Namespace, item.Name)
 }
}

三、框架说明

3.1 代码位置说明

入口代码在 kubernetes 仓库
- https://github.com/kubernetes/kubernetes
- 位于：cmd/kubectl/kubectl.go
主体代码在 kubectl 仓库
- https://github.com/kubernetes/kubectl
- 以依赖包的方式被 kubenetes 引用

3.2 代码编译

// 可以生成你自己编译的 mykubectl 工具
 go build -o bin/mykubectl cmd/kubectl/kubectl.go
 
// 测试
cd bin
./mykubectl get pod -n devops

3.3 代码主线

kubernetes: cmd/kubectl/kubectl.go main 方法

func main() {
    // 创建根命令
 command := cmd.NewDefaultKubectlCommand()
 // 运行命令
 if err := cli.RunNoErrOutput(command); err != nil {
  // Pretty-print the error and exit with an error.
  util.CheckErr(err)
 }
}

kubctl: pkg/cmd NewKubectlCommand 方法，创建 kubectl 命令对象和子命令

func NewKubectlCommand(o KubectlOptions) *cobra.Command {
 warningHandler := rest.NewWarningWriter(o.IOStreams.ErrOut, rest.WarningWriterOptions{Deduplicate: true, Color: term.AllowsColorOutput(o.IOStreams.ErrOut)})
 warningsAsErrors := false
 // Parent command to which all subcommands are added.
 cmds := &cobra.Command{
  Use:   "kubectl",
  Short: i18n.T("kubectl controls the Kubernetes cluster manager"),
  Long: templates.LongDesc(`
      kubectl controls the Kubernetes cluster manager.

      Find more information at:
            https://kubernetes.io/docs/reference/kubectl/`),
  Run: runHelp,
  // 执行命令前的钩子
  PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
   rest.SetDefaultWarningHandler(warningHandler)

   if cmd.Name() == cobra.ShellCompRequestCmd {
    // This is the __complete or __completeNoDesc command which
    // indicates shell completion has been requested.
    plugin.SetupPluginCompletion(cmd, args)
   }

   return initProfiling()
  },
  // 执行命令后的钩子
  PersistentPostRunE: func(*cobra.Command, []string) error {
   if err := flushProfiling(); err != nil {
    return err
   }
   if warningsAsErrors {
    count := warningHandler.WarningCount()
    switch count {
    case 0:
     // no warnings
    case 1:
     return fmt.Errorf("%d warning received", count)
    default:
     return fmt.Errorf("%d warnings received", count)
    }
   }
   return nil
  },
 }
 // From this point and forward we get warnings on flags that contain "_" separators
 // when adding them with hyphen instead of the original name.
 cmds.SetGlobalNormalizationFunc(cliflag.WarnWordSepNormalizeFunc)

 flags := cmds.PersistentFlags()

 addProfilingFlags(flags)

 flags.BoolVar(&warningsAsErrors, "warnings-as-errors", warningsAsErrors, "Treat warnings received from the server as errors and exit with a non-zero exit code")

 kubeConfigFlags := o.ConfigFlags
 if kubeConfigFlags == nil {
  kubeConfigFlags = defaultConfigFlags
 }
 kubeConfigFlags.AddFlags(flags)
 matchVersionKubeConfigFlags := cmdutil.NewMatchVersionFlags(kubeConfigFlags)
 matchVersionKubeConfigFlags.AddFlags(flags)
 // Updates hooks to add kubectl command headers: SIG CLI KEP 859.
 addCmdHeaderHooks(cmds, kubeConfigFlags)

    // 实例化Factory接口，工厂模式
 f := cmdutil.NewFactory(matchVersionKubeConfigFlags)

 // Proxy command is incompatible with CommandHeaderRoundTripper, so
 // clear the WrapConfigFn before running proxy command.
 proxyCmd := proxy.NewCmdProxy(f, o.IOStreams)
 proxyCmd.PreRun = func(cmd *cobra.Command, args []string) {
  kubeConfigFlags.WrapConfigFn = nil
 }

 // Avoid import cycle by setting ValidArgsFunction here instead of in NewCmdGet()
 getCmd := get.NewCmdGet("kubectl", f, o.IOStreams)
 getCmd.ValidArgsFunction = utilcomp.ResourceTypeAndNameCompletionFunc(f)

    // 命令总共分了 7大类，每类里面有对应的子命令。【核心】
 groups := templates.CommandGroups{
  {
   Message: "Basic Commands (Beginner):",
   Commands: []*cobra.Command{
    create.NewCmdCreate(f, o.IOStreams),
    expose.NewCmdExposeService(f, o.IOStreams),
    run.NewCmdRun(f, o.IOStreams),
    set.NewCmdSet(f, o.IOStreams),
   },
  },
  ... 省略，kubectl 总共有7种分类，命令行输入 kubectl 回车后可以看到
 }
 groups.Add(cmds)

 filters := []string{"options"}

 // Hide the "alpha" subcommand if there are no alpha commands in this build.
 alpha := NewCmdAlpha(f, o.IOStreams)
 if !alpha.HasSubCommands() {
  filters = append(filters, alpha.Name())
 }

 templates.ActsAsRootCommand(cmds, filters, groups...)

 utilcomp.SetFactoryForCompletion(f)
 registerCompletionFuncForGlobalFlags(cmds, f)

    // 添加其他命令
 cmds.AddCommand(alpha)
 cmds.AddCommand(cmdconfig.NewCmdConfig(clientcmd.NewDefaultPathOptions(), o.IOStreams))
 cmds.AddCommand(plugin.NewCmdPlugin(o.IOStreams))
 cmds.AddCommand(version.NewCmdVersion(f, o.IOStreams))
 cmds.AddCommand(apiresources.NewCmdAPIVersions(f, o.IOStreams))
 cmds.AddCommand(apiresources.NewCmdAPIResources(f, o.IOStreams))
 cmds.AddCommand(options.NewCmdOptions(o.IOStreams.Out))

 // Stop warning about normalization of flags. That makes it possible to
 // add the klog flags later.
 cmds.SetGlobalNormalizationFunc(cliflag.WordSepNormalizeFunc)
 return cmds
}

到此命令对象都初始化完了，常见的 kubectl get、create、delete 等命令都在上面对应的 group 里面初始化了，可以进一步跟进去查看流程。

上面实例化Factory接口 f := cmdutil.NewFactory(matchVersionKubeConfigFlags)

type Factory interface {
    // 动态客户端
    DynamicClient() (dynamic.Interface, error)

    // ClientSet客户端
    KubernetesClientSet() (*kubernetes.Clientset, error)

    // RESTClient客户端.
    RESTClient() (*restclient.RESTClient, error)

    // 实例化builder，builder用于将命令获取的参数转换成资源对象.
    NewBuilder() *resource.Builder

    // 验证资源对象
    Validator(validate bool) (validation.Schema, error)

}

这边有 3 种与 k8s api 交互的客户端

KubernetesClientSet
只能操作 k8s 内置资源
DynamicClient
不仅可以操作内资资源，还可以操作我们自定义的 crd 资源。但是 ClientSet有类型检查，DynamicClient没有。
RESTClient
KubernetesClientSet、DynamicClient 底层都是基于 RESTClient

小结

本篇内容比较广但又不会很深入，有利于入门理清知识体系。涉及到的知识点包含：go、k8s 开发、kubernetes 源码、kubectl 源码、cobra、go-client。

点赞(0) 打赏

本文分类：golang
本文标签：kubectl
浏览次数：1802 次浏览
发布日期：2022-12-16 15:06:22
本文链接：http://kubeclub.cn/golang/273.html

kubectl 源码概览

一、前言

二、kubectl 工程脚手架

2.1 spf13/cobra

2.2 go-client

三、框架说明

3.1 代码位置说明

3.2 代码编译

3.3 代码主线

小结

评论列表共有 0 条评论

发表评论取消回复

kubectl 源码概览

一、前言

二、kubectl 工程脚手架

2.1 spf13/cobra

2.2 go-client

三、框架说明

3.1 代码位置说明

3.2 代码编译

3.3 代码主线

小结

kubectl 源码概览

kubeconfig 合并工具-kubecm

评论列表 共有 0 条评论

发表评论 取消回复

评论列表共有 0 条评论

发表评论取消回复