2023-03-21 | Olaf Bohlen | |
2023-03-21 | Olaf Bohlen | |
2023-03-21 | Olaf Bohlen |
Containerfile | ●●●●● patch | view | raw | blame | history | |
README.md | ●●●●● patch | view | raw | blame | history | |
README.org | ●●●●● patch | view | raw | blame | history |
Containerfile
@@ -1,4 +1,5 @@ FROM docker.eenfach.de/olbohlen/kshbase:latest FROM quay.io/olbohlen/kshbase:latest #FROM docker.eenfach.de/olbohlen/kshbase:latest LABEL author="Olaf Bohlen <olbohlen@eenfach.de>" COPY recipe-processor.ksh /usr/local/bin/recipe-processor.ksh README.md
New file @@ -0,0 +1,305 @@ # Table of Contents 1. [I need a cookie recipe database!](#org651bf34) 2. [Preparing the cookierecipes CRD](#org58b8bb6) 3. [Storing some sample recipes](#orgbb687b0) 4. [Creating RBAC resources](#org2d91ada) 5. [Storing some sample recipes (hopefully this time!!)](#org69aa53f) 6. [Now can we do anything with our recipes?](#orgbd83100) 7. [I'm an operator with my pocket calculator](#org3bb5d31) 8. [What do we need?](#orgcebea86) 9. [Let's review the Containerfile](#org6c2e7f3) 10. [Have a look at the Controller](#org3165f3c) 11. [Now let's also have a look at the deployment](#org6ec9a40) 12. [The ServiceAccount](#orgcba3d6d) 13. [Building the stuff together](#orga6fa80a) 14. [Deploying the Operator](#org22de8a4) 15. [Test the Operator](#org1cbedf9) 16. [Test for updated recipes](#org40457b2) <a id="org651bf34"></a> # I need a cookie recipe database! …and because it makes total sense, we are going to abuse the K8s API for it. - thankfully we can extend K8s with Custom Resource Definitions (CRDs) - but how does it work? - `CustomResourceDefinitions` are themselves a `Resource`, based on a `ResourceDefinition` $ oc api-resources | egrep "(NAME|CustomResourceDefinition)" NAME SHORTNAMES APIVERSION NAMESPACED KIND customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition $ oc explain crds KIND: CustomResourceDefinition VERSION: apiextensions.k8s.io/v1 DESCRIPTION: CustomResourceDefinition represents a resource that should be exposed on the API server. Its name MUST be in the format <.spec.name>.<.spec.group>. [...] <a id="org58b8bb6"></a> # Preparing the cookierecipes CRD Let's create the CRD from <https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-crd.yaml> $ oc new-project kitchen Now using project "kitchen" on server "https://api.crc.testing:6443". $ oc create -f cookie-crd.yaml Error from server (Forbidden): error when creating "cookie-crd.yaml": customresourcedefinitions.apiextensions.k8s.io is forbidden: User "developer" cannot create resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope $ oc login -u kubeadmin $ oc create -f cookie-crd.yaml customresourcedefinition.apiextensions.k8s.io/cookierecipes.de.eenfach.olbohlen created $ oc login -u developer Now the cluster knows about the CRD and we could store recipes! <a id="orgbb687b0"></a> # Storing some sample recipes We try to store sample cookie recipes…but: $ oc create -f sample-cookie.yaml Error from server (Forbidden): error when creating "sample-cookie.yaml": cookierecipes.de.eenfach.olbohlen is forbidden: User "developer" cannot create resource "cookierecipes" in API group "de.eenfach.olbohlen" in the namespace "kitchen" Error from server (Forbidden): error when creating "sample-cookie.yaml": cookierecipes.de.eenfach.olbohlen is forbidden: User "developer" cannot create resource "cookierecipes" in API group "de.eenfach.olbohlen" in the namespace "kitchen" We need to set up some RBAC resources first: - a `ClusterRole` that allows viewing recipes - a `ClusterRole` that allows editing recipes - and a `ClusterRoleBinding` that allows that for authenticated users <a id="org2d91ada"></a> # Creating RBAC resources Apply the RBAC definitions from: <https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-rbac.yaml> $ oc login -u kubeadmin $ oc create -f cookie-rbac.yaml clusterrole.rbac.authorization.k8s.io/cookierecipe-edit created clusterrole.rbac.authorization.k8s.io/cookierecipe-view created clusterrolebinding.rbac.authorization.k8s.io/cookierecipe-edit created The `ClusterRoleBinding` "cookierecipe-edit" allows `system:authenticated:oauth` group members to edit `cookierecipes`. `system:authenticated:oauth` contains all users that logged in via the OAuth service (via an `IdentityProvider`). <a id="org69aa53f"></a> # Storing some sample recipes (hopefully this time!!) Now we should be able to create the sample recipes: $ oc login -u developer $ oc create -f sample-cookie.yaml cookierecipe.de.eenfach.olbohlen/vintage-chocolate-chip created cookierecipe.de.eenfach.olbohlen/double-dipped-shortbread created $ oc get cookierecipe NAME AGE double-dipped-shortbread 17s vintage-chocolate-chip 17s There is no functionality here - we just stored the recipes in the etcd via the K8s API. <a id="orgbd83100"></a> # Now can we do anything with our recipes? Of course we can **oc get -o yaml** for example on them and filter: $ oc get cookierecipe vintage-chocolate-chip -o yaml | yq -y .spec.ingredients[0] amount: 150 name: salted butter remarks: softened unit: grams This is handy, as we can extract exactly the data which we need at a time. But…it's a lot of manual work… <a id="org3bb5d31"></a> # I'm an operator with my pocket calculator Operators were introduced as "Kubernetes Native Applications" and that actually means nothing. Operators are in the end just `Pods`. These Pods run one or more containers, but one container should run a `Controller` that can interprete your `CustomResources`. So let's write a CookieRecipe Operator. In shell-script… :) Of course this Operator is not compatible with the `OperatorLifecycyleManager` (`OLM`), so we have to install it manually. <a id="orgcebea86"></a> # What do we need? We need: - a ContainerImage - and therefore probably a **Containerfile** - Controller code Then we are going to build the Operator ContainerImage and push it to a Registry. <a id="org6c2e7f3"></a> # Let's review the Containerfile The Containerfile is here: <https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/Containerfile> the base image is a "kshbase" image, which itself is based upon ubi9 containing also a ksh93 and an oc client. <a id="org3165f3c"></a> # Have a look at the Controller The controller is written in KornShell 93 (ksh93), which is mostly bash compatible :) The code is here: <https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/recipe-processor.ksh> <a id="org6ec9a40"></a> # Now let's also have a look at the deployment Note: this deployment does not use an `ImageStream`, so it would work also on native k8s <https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-operator-deployment.yaml> This deployment requires a `ServiceAccount` called "cookieprocessor", this `ServiceAccount` provides a Token to authenticate against the API (which we use in the controller script). <a id="orgcba3d6d"></a> # The ServiceAccount We need a `ServiceAccount`, but that alone will not help. The `ServiceAccount` is NOT member of `system:authenticated:oauth`, so it can't read `cookierecipes` based on the `ClusterRoleBinding` we created earlier. For that reason we also create a `RoleBinding` (namespaced!) that allows reading recipes: <https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookieprocessor-sa.yaml> <a id="orga6fa80a"></a> # Building the stuff together $ oc create -f cookieprocessor-sa.yaml serviceaccount/cookieprocessor created rolebinding.rbac.authorization.k8s.io/cookierecipe-view created The registry docker.eenfach.de requires login credentials, so we need to set up a secret and link it. First login to the registry with **podman login**, then pick the resulting auth.json: $ podman login -u olbohlen docker.eenfach.de Password: Login Succeeded! $ oc create secret generic docker-eenfach-de \ > --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json \ > --type kubernetes.io/dockerconfigjson secret/docker-eenfach-de created $ oc secrets link cookieprocessor docker-eenfach-de --for pull <a id="org22de8a4"></a> # Deploying the Operator Now that we have everything in place, we will just deploy the Operator Pod: $ oc create -f cookie-operator-deployment.yaml deployment.apps/recipe-processor created $ oc get pod NAME READY STATUS RESTARTS AGE recipe-processor-7f9969697b-qt9lv 1/1 Running 0 17s $ oc logs -f recipe-processor-7f9969697b-qt9lv New recipe found: double-dipped-shortbread -------------------------------------------------------------------------- Pre: we heat up the oven to 180 degrees Celsius Fetching ingredients from recipe: ---------------------------------- Fetching 200grams of salted butter (softened) [...] The Operator will process both sample recipes. <a id="org1cbedf9"></a> # Test the Operator We should test if the Operator notices new recipes, so let's create a third recipe from <https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/oaty-hazelnut-cookies.yaml> $ oc create -f oaty-hazelnut-cookies.yaml cookierecipe.de.eenfach.olbohlen/oaty-hazelnut created After a few seconds, we should see in the Operator log: New recipe found: oaty-hazelnut -------------------------------------------------------------------------- Pre: we heat up the oven to 180 degrees Celsius [...] <a id="org40457b2"></a> # Test for updated recipes But what if we update a resource? Keep the **oc logs -f** on the Operator Pod open, and in another terminal let's patch a recipe. $ oc patch cookierecipes double-dipped-shortbread --type merge \ > -p '{"spec":{"temperature":172}}' cookierecipe.de.eenfach.olbohlen/double-dipped-shortbread patched And again in the log you should see New recipe found: double-dipped-shortbread -------------------------------------------------------------------------- Pre: we heat up the oven to 172 degrees Celsius Fetching ingredients from recipe: ---------------------------------- [...] README.org
New file @@ -0,0 +1,257 @@ * I need a cookie recipe database! ...and because it makes total sense, we are going to abuse the K8s API for it. - thankfully we can extend K8s with Custom Resource Definitions (CRDs) - but how does it work? - =CustomResourceDefinitions= are themselves a =Resource=, based on a =ResourceDefinition= #+begin_example $ oc api-resources | egrep "(NAME|CustomResourceDefinition)" NAME SHORTNAMES APIVERSION NAMESPACED KIND customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition $ oc explain crds KIND: CustomResourceDefinition VERSION: apiextensions.k8s.io/v1 DESCRIPTION: CustomResourceDefinition represents a resource that should be exposed on the API server. Its name MUST be in the format <.spec.name>.<.spec.group>. [...] #+end_example * Preparing the cookierecipes CRD Let's create the CRD from https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-crd.yaml #+begin_example $ oc new-project kitchen Now using project "kitchen" on server "https://api.crc.testing:6443". $ oc create -f cookie-crd.yaml Error from server (Forbidden): error when creating "cookie-crd.yaml": customresourcedefinitions.apiextensions.k8s.io is forbidden: User "developer" cannot create resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope $ oc login -u kubeadmin $ oc create -f cookie-crd.yaml customresourcedefinition.apiextensions.k8s.io/cookierecipes.de.eenfach.olbohlen created $ oc login -u developer #+end_example Now the cluster knows about the CRD and we could store recipes! * Storing some sample recipes We try to store sample cookie recipes...but: #+begin_example $ oc create -f sample-cookie.yaml Error from server (Forbidden): error when creating "sample-cookie.yaml": cookierecipes.de.eenfach.olbohlen is forbidden: User "developer" cannot create resource "cookierecipes" in API group "de.eenfach.olbohlen" in the namespace "kitchen" Error from server (Forbidden): error when creating "sample-cookie.yaml": cookierecipes.de.eenfach.olbohlen is forbidden: User "developer" cannot create resource "cookierecipes" in API group "de.eenfach.olbohlen" in the namespace "kitchen" #+end_example We need to set up some RBAC resources first: - a =ClusterRole= that allows viewing recipes - a =ClusterRole= that allows editing recipes - and a =ClusterRoleBinding= that allows that for authenticated users * Creating RBAC resources Apply the RBAC definitions from: https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-rbac.yaml #+begin_example $ oc login -u kubeadmin $ oc create -f cookie-rbac.yaml clusterrole.rbac.authorization.k8s.io/cookierecipe-edit created clusterrole.rbac.authorization.k8s.io/cookierecipe-view created clusterrolebinding.rbac.authorization.k8s.io/cookierecipe-edit created #+end_example The =ClusterRoleBinding= "cookierecipe-edit" allows =system:authenticated:oauth= group members to edit =cookierecipes=. =system:authenticated:oauth= contains all users that logged in via the OAuth service (via an =IdentityProvider=). * Storing some sample recipes (hopefully this time!!) Now we should be able to create the sample recipes: #+begin_example $ oc login -u developer $ oc create -f sample-cookie.yaml cookierecipe.de.eenfach.olbohlen/vintage-chocolate-chip created cookierecipe.de.eenfach.olbohlen/double-dipped-shortbread created $ oc get cookierecipe NAME AGE double-dipped-shortbread 17s vintage-chocolate-chip 17s #+end_example There is no functionality here - we just stored the recipes in the etcd via the K8s API. * Now can we do anything with our recipes? Of course we can *oc get -o yaml* for example on them and filter: #+begin_example $ oc get cookierecipe vintage-chocolate-chip -o yaml | yq -y .spec.ingredients[0] amount: 150 name: salted butter remarks: softened unit: grams #+end_example This is handy, as we can extract exactly the data which we need at a time. But...it's a lot of manual work... * I'm an operator with my pocket calculator Operators were introduced as "Kubernetes Native Applications" and that actually means nothing. Operators are in the end just =Pods=. These Pods run one or more containers, but one container should run a =Controller= that can interprete your =CustomResources=. So let's write a CookieRecipe Operator. In shell-script... :) Of course this Operator is not compatible with the =OperatorLifecycyleManager= (=OLM=), so we have to install it manually. * What do we need? We need: - a ContainerImage - and therefore probably a *Containerfile* - Controller code Then we are going to build the Operator ContainerImage and push it to a Registry. * Let's review the Containerfile The Containerfile is here: https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/Containerfile the base image is a "kshbase" image, which itself is based upon ubi9 containing also a ksh93 and an oc client. * Have a look at the Controller The controller is written in KornShell 93 (ksh93), which is mostly bash compatible :) The code is here: https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/recipe-processor.ksh * Now let's also have a look at the deployment Note: this deployment does not use an =ImageStream=, so it would work also on native k8s https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-operator-deployment.yaml This deployment requires a =ServiceAccount= called "cookieprocessor", this =ServiceAccount= provides a Token to authenticate against the API (which we use in the controller script). * The ServiceAccount We need a =ServiceAccount=, but that alone will not help. The =ServiceAccount= is NOT member of =system:authenticated:oauth=, so it can't read =cookierecipes= based on the =ClusterRoleBinding= we created earlier. For that reason we also create a =RoleBinding= (namespaced!) that allows reading recipes: https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookieprocessor-sa.yaml * Building the stuff together #+begin_example $ oc create -f cookieprocessor-sa.yaml serviceaccount/cookieprocessor created rolebinding.rbac.authorization.k8s.io/cookierecipe-view created #+end_example The registry docker.eenfach.de requires login credentials, so we need to set up a secret and link it. First login to the registry with *podman login*, then pick the resulting auth.json: #+begin_example $ podman login -u olbohlen docker.eenfach.de Password: Login Succeeded! $ oc create secret generic docker-eenfach-de \ > --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json \ > --type kubernetes.io/dockerconfigjson secret/docker-eenfach-de created $ oc secrets link cookieprocessor docker-eenfach-de --for pull #+end_example * Deploying the Operator Now that we have everything in place, we will just deploy the Operator Pod: #+begin_example $ oc create -f cookie-operator-deployment.yaml deployment.apps/recipe-processor created $ oc get pod NAME READY STATUS RESTARTS AGE recipe-processor-7f9969697b-qt9lv 1/1 Running 0 17s $ oc logs -f recipe-processor-7f9969697b-qt9lv New recipe found: double-dipped-shortbread -------------------------------------------------------------------------- Pre: we heat up the oven to 180 degrees Celsius Fetching ingredients from recipe: ---------------------------------- Fetching 200grams of salted butter (softened) [...] #+end_example The Operator will process both sample recipes. * Test the Operator We should test if the Operator notices new recipes, so let's create a third recipe from https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/oaty-hazelnut-cookies.yaml #+begin_example $ oc create -f oaty-hazelnut-cookies.yaml cookierecipe.de.eenfach.olbohlen/oaty-hazelnut created #+end_example After a few seconds, we should see in the Operator log: #+begin_example New recipe found: oaty-hazelnut -------------------------------------------------------------------------- Pre: we heat up the oven to 180 degrees Celsius [...] #+end_example * Test for updated recipes But what if we update a resource? Keep the *oc logs -f* on the Operator Pod open, and in another terminal let's patch a recipe. #+begin_example $ oc patch cookierecipes double-dipped-shortbread --type merge \ > -p '{"spec":{"temperature":172}}' cookierecipe.de.eenfach.olbohlen/double-dipped-shortbread patched #+end_example And again in the log you should see #+begin_example New recipe found: double-dipped-shortbread -------------------------------------------------------------------------- Pre: we heat up the oven to 172 degrees Celsius Fetching ingredients from recipe: ---------------------------------- [...] #+end_example