{"id":1174,"date":"2021-10-26T09:43:06","date_gmt":"2021-10-26T07:43:06","guid":{"rendered":"https:\/\/loeilduse.fr\/?p=1174"},"modified":"2022-02-19T18:36:56","modified_gmt":"2022-02-19T17:36:56","slug":"add-a-self-signed-certificat-on-tkg-cluster","status":"publish","type":"post","link":"https:\/\/loeilduse.fr\/?p=1174&lang=en","title":{"rendered":"Add a self-signed certificat on TKG cluster"},"content":{"rendered":"

If you want to deploy pods on a Kubernetes cluster that does not know the certificate of the registry containing the images (this is generally the case for labs with self-signed certificates that are not known by an authority), you risk to not be able to deploy your images, let’s see an example by deploying a Kuard image from my private Harbor registry:<\/span><\/p>\n

(Warning, WordPress replaces the two dashes with one: cry:) <\/span><\/p>\n

# kubectl run kuard \u2013image=harbor.cpod-velocity.az-fkd.cloud-garage.net\/library\/kuard-amd64:blue<\/span>
\n# kubectl get pods<\/span>
\nNAME\u00a0 READY STATUS\u00a0\u00a0 RESTARTS\u00a0 AGE<\/span>
\nkuard\u00a0\u00a0\u00a0 0\/1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ImagePullBackOff\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 7s<\/span><\/p>\n

# kubectl describe pods kuard<\/span>
\n\u2026.<\/span>
\nType Reason Age From Message <\/span>
\n\u2014- \u2014\u2014 \u2014- \u2014- \u2014\u2014-<\/span>
\nNormal Scheduled 33s default-scheduler Successfully assigned default\/kuard to test-md-0-5d6756b7fd-b9kwl<\/span>
\nNormal Pulling 19s (x2 over 32s) kubelet Pulling image \u201charbor.cpod-velocity.az-fkd.cloud-garage.net\/library\/kuard-amd64:blue\u201d<\/span>
\nWarning Failed 19s (x2 over 32s) kubelet Failed to pull image \u201charbor.cpod-velocity.az-fkd.cloud-garage.net\/library\/kuard-amd64:blue\u201d: rpc error: code = Unknown desc = failed to pull and unpack image \u201charbor.cpod-velocity.az-fkd.cloud-garage.net\/library\/kuard-amd64:blue\u201d: failed to resolve reference \u201charbor.cpod-velocity.az-fkd.cloud-garage.net\/library\/kuard-amd64:blue\u201d: failed to do request: Head \u201chttps:\/\/harbor.cpod-velocity.az-fkd.cloud-garage.net\/v2\/library\/kuard-amd64\/manifests\/blue\u201d: x509: certificate signed by unknown authority<\/strong><\/span>
\nWarning Failed 19s (x2 over 32s) kubelet Error: ErrImagePull<\/strong><\/span>
\nNormal BackOff 4s (x2 over 31s) kubelet Back-off pulling image \u201charbor.cpod-velocity.az-fkd.cloud-garage.net\/library\/kuard-amd64:blue\u201d<\/strong><\/span>
\nWarning Failed 4s (x2 over 31s) kubelet Error: ImagePullBackOff<\/strong><\/span><\/p>\n

In this example, I tried from my Tanzu Kubernetes Grid (TKG) cluster to deploy the Kuard image which is located on my private Harbor registry hosted on my lab. This self-signed certificate is not recognized by an authority. For this to work anyway, each worker node in my cluster would need to know this certificate. I can copy it to each of them but the principle of TKG is to have a cluster whose life cycle can evolve easily and automatically, add worker nodes, remove them, replace them during the update , …. This would require each time a node is added to include the certificate. <\/span><\/p>\n

Jesse Hu<\/a> wrote a tkg-ytt-overlay-additional-ca-certs \u00b7 GitHub<\/a> procedure that I tested in TKG 1.4. It works great, it makes sense for those who use Kubernetes every day but may seem complicated to others. I will try to clarify it. This consists of obtaining the certificate, encoded in base64, of executing the commands to have the certificate taken into account by all existing and future Kubernetes nodes. <\/span><\/p>\n

Encrypt the certificate with base64 and copy it<\/span><\/span><\/p>\n

# base64 -w0 ca.crt (le r\u00e9sultat a volontairement \u00e9t\u00e9 modifi\u00e9)<\/span>
\nLS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFekNDQWZ1Z0F3SUJBZ0lRREthWmp6NmF4MitZS3RxS0YrUUNKekFOQmdrcWhraUc5dzBCQVFzRkFEQVUKTVJJd0VBWURWUVFERXdsb1lYSmliM0l0WTJFd0hoY05NakV3T0RJek1UVXdNREEwV2hjTk1qSXdPREl6TVRVdwpNREEwV2pBVU1SSXdFQVlEVlFRREV3bG9ZWEppYjNJdFkyRXdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCCkR3QXdnZ0VLQW9JQkFRREl0MEhDUEF0ZWxrSVZhOXZtbjJnQWR4VXNaL3lTd2psNi95SzV0eTZ5OW1PbU5peDgKOHAvVFk2SG9MVHlJNUhtNytUYTJ4RGpvUUpmNllWMURsc3Y3d2E2R2pTUXE0WWxQWG1hUUNvVkp5eno5OVRvUgpDbW80VjhRUnJLbE5WL3NFbStrVXhseGFNZTZOMlA3UjB1MHVCV0Q5NW9Ra3RqWC8rS01uT0ErUlZoWkJ1dEFmCjJXSzhIMmRYeWo4bFV3Vk4rWWJqeW83dkdnNERNZlVHMWtDa0hYYURTMGcvWlhyNU1LRTNDWEo4YUVPZFhMNjMKb3BHTXVMQWp4WUZIdng0SnBMN0lNQ2VZMGFCcjcybUUzcy9SMENCMW1zTU5nYWhXTkhNZjhINktsUy9qUVlnUgpQMjV6WmYyOUZ0VFdnOWhZNVJZQngxalcyeXZERGJsMmFGNXBBZ01CQUFHallUQmZNQTRHQTFVZER3RUIvd1FFCkF3SUNwREFkQmdOVkhTVUVGakFVQmdnckJnRUZCUWNESUt3WUJCUVVIQXdJd0R3WURWUjBUQVFIL0JBVXcKQXdFQi96QWRCZ05WSFE0RUZnUVVVbTBvVUtiSXBHbTcxL3JzUlBReUJJbkZYYmt3RFFZSktvWklodmNOQVFFTApCUUFEZ2dFQkFE cVJBamwzWTNqVk5JK1JUbHBYdmwvdUYxbXNZUTNzTnFwVXR6eVNCVlNDWlpDMkRSSnpwYWZGCjhPdXN5QjBMTTNlS3VTd0t4STgrT0o5OTlhZkdOazRWTnpySVhOQURaZ1BxbnRFSWRucXNReGg4eFBuOVY0T2QKQUtsTVJycVI4R3g4ejdRM2EvN01uR0sra1l3VmorZ3BBNkFGUEJxSVJrU3Jscmo5b2dXVzBqWTFzL2tNU21ydgpaVEFZWTJqcFhBaGZrdzcrVDN4OHYwa0NRai9NREo5L3dNTnhxeVNGMEhzNXd6THVvbVJOM0VEME03eUNhWjg0CmJuOFZTN1VUSjBaWnhBRmx3TWxySlRYWmFpQmNOeDRNdm4wNXN4RG5KZktCdFloSkZwbGRwR3hLMDRUSmNXWm0KU0dDemZhc2FIK2M1NklNT0IvRllMdlJlelh4cE5mMD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo =<\/span><\/p>\n

 <\/p>\n

Find the context of your TKG management cluster and the name of the TKG workload cluster on which you will copy the certificate:<\/span><\/span><\/p>\n

# kubectl get-contexts<\/span><\/p>\n

CURRENT\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 NAME CLUSTER\u00a0 \u00a0 \u00a0 AUTHINFO\u00a0\u00a0\u00a0 \u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0\u00a0 NAMESPACE<\/span>
\n\u2026.<\/span>
\nmy-cluster-admin@my-cluster\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 my-cluster\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 my-cluster-admin<\/span>
\n* test-admin@test\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 test\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <\/strong>test-admin<\/span>
\ntkg-mgmt-vsphere-admin@tkg-mgmt-vsphere<\/strong> \u00a0\u00a0\u00a0 tkg-mgmt-vsphere\u00a0 tkg-mgmt-vsphere-admin<\/span>
\ntool-admin@tool\u00a0<\/span><\/p>\n

Edit the configuration of the controller and worker template to add the content of the certificate and the command to take this certificate into account.<\/span> So each time a node is added, the certificate will be taken into account.<\/span> The content of the file is to be added in the files:<\/span> part and the command in the preKubeadmCommands:<\/span> part as below (I have a configuration based on Photon OS, if you have another OS you must use another command.<\/span> Copied \/ pasted does not work very well it is better to type the words again):<\/span><\/span><\/p>\n

To edit the control plane template : <\/span><\/span>kubectl edit KubeadmControlPlane test-control-plane \u2013context tkg-mgmt-vsphere-admin@tkg-mgmt-vsphere<\/span><\/p>\n

(the syntax just after\u00a0 content shloud be like thist : – content: <base64 certificate><\/span>)<\/em><\/p>\n

\u2026\u2026<\/span><\/p>\n

\u00a0 files:<\/span>
\n\u00a0 \u2013 content: S0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFekNDQWZ1Z0F3SUJBZ0lRREthWmp6NmF4MitZS3RxS0YrUUNKekFOQmdrcWhraUc5dzBCQVFzRkFEQVUKTVJJd0VBWURWUVFERXdsb1lYSmliM0l0WTJFd0hoY05NakV3T0RJek1UVXdNREEwV2hjTk1qSXdPREl6TVRVdwpNREEwV2pBVU1SSXdFQVlEVlFRREV3bG9ZWEppYjNJdFkyRXdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCCkR3QXdnZ0VLQW9JQkFRREl0MEhDUEF0ZWxrSVZhOXZtbjJnQWR4VXNaL3lTd2psNi95SzV0eTZ5OW1PbU5peDgKOHAvVFk2SG9MVHlJNUhtNytUYTJ4RGpvUUpmNllWMURsc3Y3d2E2R2pTUXE0WWxQWG1hUUNvVkp5eno5OVRvUgpDbW80VjhRUnJLbE5WL3NFbStrVXhseGFNZTZOMlA3UjB1MHVCV0Q5NW9Ra3RqWC8rS01uT0ErUlZoWkJ1dEFmCjJXSzhIMmRYeWo4bFV3Vk4rWWJqeW83dkdnNERNZlVHMWtDa0hYYURTMGcvWlhyNU1LRTNDWEo4YUVPZFhMNjMKb3BHTXVMQWp4WUZIdng0SnBMN0lNQ2VZMGFCcjcybUUzcy9SMENCMW1zTU5nYWhXTkhNZjhINktsUy9qUVlnUgpQMjV6WmYyOUZ0VFdnOWhZNVJZQngxalcyeXZERGJsMmFGNXBBZ01CQUFHallUQmZNQTRHQTFVZER3RUIvd1FFCkF3SUNwREFkQmdOVkhTVUVGakFVQmdnckJnRUZCUWNESUt3WUJCUVVIQXdJd0R3WURWUjBUQVFIL0JBVXcKQXdFQi96QWRCZ05WSFE0RUZnUVVVbTBvVUtiSXBHbTcxL3JzUlBReUJJbkZYYmt3RFFZSktvWklodmNOQVFFTApCUUFEZ2dFQkFEcVJBamwzWTNqVk5JK1JUbHBYdmwvdUYxbXNZUTNzTnFwVXR6eVNCVlNDWlpDMkRSSnpwYWZGCjhPdXN5QjBMTTNlS3VTd0t4STgrT0o5OTlhZkdOazRWTnpySVhOQURaZ1BxbnRFSWRucXNReGg4eFBuOVY0T2QKQUtsTVJycVI4R3g4ejdRM2EvN01uR0sra1l3VmorZ3BBNkFGUEJxSVJrU3Jscmo5b2dXVzBqWTFzL2tNU21ydgpaVEFZWTJqcFhBaGZrdzcrVDN4OHYwa0NRai9NREo5L3dNTnhxeVNGMEhzNXd6THVvbVJOM0VEME03eUNhWjg0CmJuOFZTN1VUSjBaWnhBRmx3TWxySlRYWmFpQmNOeDRNdm4wNXN4RG5KZktCdFloSkZwbGRwR3hLMDRUSmNXWm0KU0dDemZhc2FIK2M1NklNT0IvRllMdlJlelh4cE5mMD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=<\/span>
\n\u00a0\u00a0\u00a0 encoding: base64<\/span>
\n\u00a0\u00a0\u00a0 owner: root:root<\/span>
\n\u00a0\u00a0\u00a0 permissions: \u201c0644\u201d<\/span>
\n\u00a0\u00a0\u00a0 path: \/etc\/ssl\/certs\/tkg-custom-ca.pem<\/span><\/p>\n

\u2026\u2026<\/span><\/p>\n

\u00a0 preKubeadmCommands:<\/span><\/p>\n

\u00a0 \u2013 \u2018<\/span>! which rehash_ca_certificates.sh 2>\/dev\/null || rehash_ca_certificates.sh\u2018<\/span><\/span><\/span><\/p>\n

\u2026\u2026\u2026.<\/span><\/p>\n

It is taken into account immediately and a new controller is deployed to replace the old one.<\/span> (or several depending on the plan chosen at the time of creation).<\/span> Wait while the new controller is deploying and replaces the old one.<\/span> The same for the worker: kubectl edit KubeadmConfigTemplate test-md-0 \u2013context tkg-mgmt-vsphere-admin@tkg-mgmt-vsphere.<\/span><\/span> The consideration for workers is not immediate, you have to run the following command:<\/span><\/span><\/p>\n

# kubectl patch machinedeployment test-md-0 –type merge -p “{\\”spec\\”:{\\”template\\”:{\\”metadata\\”:{\\”annotations\\”:{\\”date\\”:\\”`date +’%s’`\\”}}}}}” –context tkg-mgmt-vsphere-admin@tkg-mgmt
\npatched Wait a bit for the worker nodes to be replaced by new ones then we can retry deploying a new kuard image<\/span> <\/span><\/span><\/p>\n

# kubectl get node NAME STATUS ROLES AGE VERSION test-control-plane-gsmqz Ready control-plane,master 34m v1.21.2+vmware.1 test-md-0-698857566f-8pvt7 Ready <none> 118s v1.21.2+vmware.1<\/span> Now we can redeploy the Kuard image to check that it will run<\/span> <\/span> # kubectl run kuard –image=harbor.cpod-velocity.az-fkd.cloud-garage.net\/library\/kuard-amd64:blue pod\/kuard created<\/span><\/p>\n

# kg pods<\/span>
\nNAME READY STATUS RESTARTS AGE<\/span>
\nkuard 1\/1 Running 0 17s<\/span><\/p>\n

This procedure is valid for taking certificates into account if the cluster is already deployed.<\/span> if it has not yet been created, it is preferable to have the certificate taken into account from the start using the procedure described in the installation documentation.<\/span> <\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

If you want to deploy pods on a Kubernetes cluster that does not know the certificate of the registry containing the images (this is generally the case for labs with self-signed certificates that are not known by an authority), you risk to not be able to deploy your images, let’s see an example by deploying<\/p><\/div>\n

Lire la suite<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[401],"tags":[],"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/loeilduse.fr\/index.php?rest_route=\/wp\/v2\/posts\/1174"}],"collection":[{"href":"https:\/\/loeilduse.fr\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/loeilduse.fr\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/loeilduse.fr\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/loeilduse.fr\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1174"}],"version-history":[{"count":10,"href":"https:\/\/loeilduse.fr\/index.php?rest_route=\/wp\/v2\/posts\/1174\/revisions"}],"predecessor-version":[{"id":1189,"href":"https:\/\/loeilduse.fr\/index.php?rest_route=\/wp\/v2\/posts\/1174\/revisions\/1189"}],"wp:attachment":[{"href":"https:\/\/loeilduse.fr\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1174"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/loeilduse.fr\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1174"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/loeilduse.fr\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1174"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}