| commit | author | age | ||
| 0cfd14 | 1 | * I need a cookie recipe database! |
| OB | 2 | |
| 3 | ...and because it makes total sense, we are going to abuse the K8s API for it. | |
| 4 | ||
| 5 | - thankfully we can extend K8s with Custom Resource Definitions (CRDs) | |
| 6 | - but how does it work? | |
| 7 | - =CustomResourceDefinitions= are themselves a =Resource=, based on a =ResourceDefinition= | |
| 8 | ||
| 9 | #+begin_example | |
| 10 | $ oc api-resources | egrep "(NAME|CustomResourceDefinition)" | |
| 11 | NAME SHORTNAMES APIVERSION NAMESPACED KIND | |
| 12 | customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition | |
| 13 | $ oc explain crds | |
| 14 | KIND: CustomResourceDefinition | |
| 15 | VERSION: apiextensions.k8s.io/v1 | |
| 16 | ||
| 17 | DESCRIPTION: | |
| 18 | CustomResourceDefinition represents a resource that should be exposed on | |
| 19 | the API server. Its name MUST be in the format <.spec.name>.<.spec.group>. | |
| 20 | [...] | |
| 21 | #+end_example | |
| 22 | ||
| 23 | * Preparing the cookierecipes CRD | |
| 24 | ||
| 25 | Let's create the CRD from https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-crd.yaml | |
| 26 | ||
| 27 | #+begin_example | |
| 28 | $ oc new-project kitchen | |
| 29 | Now using project "kitchen" on server "https://api.crc.testing:6443". | |
| 30 | $ oc create -f cookie-crd.yaml | |
| 31 | Error from server (Forbidden): error when creating "cookie-crd.yaml": | |
| 32 | customresourcedefinitions.apiextensions.k8s.io is forbidden: User "developer" | |
| 33 | cannot create resource "customresourcedefinitions" in API group "apiextensions.k8s.io" | |
| 34 | at the cluster scope | |
| 35 | ||
| 36 | $ oc login -u kubeadmin | |
| 37 | $ oc create -f cookie-crd.yaml | |
| 38 | customresourcedefinition.apiextensions.k8s.io/cookierecipes.de.eenfach.olbohlen created | |
| 39 | $ oc login -u developer | |
| 40 | #+end_example | |
| 41 | ||
| 42 | Now the cluster knows about the CRD and we could store recipes! | |
| 43 | ||
| 44 | * Storing some sample recipes | |
| 45 | ||
| 46 | We try to store sample cookie recipes...but: | |
| 47 | ||
| 48 | #+begin_example | |
| 49 | $ oc create -f sample-cookie.yaml | |
| 50 | Error from server (Forbidden): error when creating "sample-cookie.yaml": | |
| 51 | cookierecipes.de.eenfach.olbohlen is forbidden: User "developer" cannot create | |
| 52 | resource "cookierecipes" in API group "de.eenfach.olbohlen" in the namespace "kitchen" | |
| 53 | Error from server (Forbidden): error when creating "sample-cookie.yaml": | |
| 54 | cookierecipes.de.eenfach.olbohlen is forbidden: User "developer" cannot create | |
| 55 | resource "cookierecipes" in API group "de.eenfach.olbohlen" in the namespace "kitchen" | |
| 56 | #+end_example | |
| 57 | ||
| 58 | We need to set up some RBAC resources first: | |
| 59 | - a =ClusterRole= that allows viewing recipes | |
| 60 | - a =ClusterRole= that allows editing recipes | |
| 61 | - and a =ClusterRoleBinding= that allows that for authenticated users | |
| 62 | ||
| 63 | * Creating RBAC resources | |
| 64 | ||
| 65 | Apply the RBAC definitions from: https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-rbac.yaml | |
| 66 | #+begin_example | |
| 67 | $ oc login -u kubeadmin | |
| 68 | $ oc create -f cookie-rbac.yaml | |
| 69 | clusterrole.rbac.authorization.k8s.io/cookierecipe-edit created | |
| 70 | clusterrole.rbac.authorization.k8s.io/cookierecipe-view created | |
| 71 | clusterrolebinding.rbac.authorization.k8s.io/cookierecipe-edit created | |
| 72 | #+end_example | |
| 73 | ||
| 74 | The =ClusterRoleBinding= "cookierecipe-edit" allows =system:authenticated:oauth= | |
| 75 | group members to edit =cookierecipes=. | |
| 76 | =system:authenticated:oauth= contains all users that logged in via the OAuth | |
| 77 | service (via an =IdentityProvider=). | |
| 78 | ||
| 79 | * Storing some sample recipes (hopefully this time!!) | |
| 80 | ||
| 81 | Now we should be able to create the sample recipes: | |
| 82 | ||
| 83 | #+begin_example | |
| 84 | $ oc login -u developer | |
| 85 | $ oc create -f sample-cookie.yaml | |
| 86 | cookierecipe.de.eenfach.olbohlen/vintage-chocolate-chip created | |
| 87 | cookierecipe.de.eenfach.olbohlen/double-dipped-shortbread created | |
| 88 | $ oc get cookierecipe | |
| 89 | NAME AGE | |
| 90 | double-dipped-shortbread 17s | |
| 91 | vintage-chocolate-chip 17s | |
| 92 | #+end_example | |
| 93 | ||
| 94 | There is no functionality here - we just stored the recipes in the etcd via the K8s API. | |
| 95 | ||
| 96 | * Now can we do anything with our recipes? | |
| 97 | Of course we can *oc get -o yaml* for example on them and filter: | |
| 98 | ||
| 99 | #+begin_example | |
| 100 | $ oc get cookierecipe vintage-chocolate-chip -o yaml | yq -y .spec.ingredients[0] | |
| 101 | amount: 150 | |
| 102 | name: salted butter | |
| 103 | remarks: softened | |
| 104 | unit: grams | |
| 105 | #+end_example | |
| 106 | ||
| 107 | This is handy, as we can extract exactly the data which we need at a time. | |
| 108 | ||
| 109 | But...it's a lot of manual work... | |
| 110 | ||
| 111 | * I'm an operator with my pocket calculator | |
| 112 | ||
| 113 | Operators were introduced as "Kubernetes Native Applications" and that actually | |
| 114 | means nothing. Operators are in the end just =Pods=. | |
| 115 | ||
| 116 | These Pods run one or more containers, but one container should run a =Controller= | |
| 117 | that can interprete your =CustomResources=. | |
| 118 | ||
| 119 | So let's write a CookieRecipe Operator. In shell-script... :) | |
| 120 | ||
| 121 | Of course this Operator is not compatible with the =OperatorLifecycyleManager= (=OLM=), | |
| 122 | so we have to install it manually. | |
| 123 | ||
| 124 | * What do we need? | |
| 125 | ||
| 126 | We need: | |
| 127 | - a ContainerImage | |
| 128 | - and therefore probably a *Containerfile* | |
| 129 | - Controller code | |
| 130 | ||
| 131 | Then we are going to build the Operator ContainerImage and push it to a Registry. | |
| 132 | ||
| 133 | * Let's review the Containerfile | |
| 134 | ||
| 135 | The Containerfile is here: | |
| 136 | ||
| 137 | https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/Containerfile | |
| 138 | ||
| 139 | the base image is a "kshbase" image, which itself is based upon ubi9 containing also a ksh93 | |
| 140 | and an oc client. | |
| 141 | ||
| 142 | * Have a look at the Controller | |
| 143 | ||
| 144 | The controller is written in KornShell 93 (ksh93), which is mostly bash compatible :) | |
| 145 | ||
| 146 | The code is here: | |
| 147 | ||
| 148 | https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/recipe-processor.ksh | |
| 149 | ||
| 150 | * Now let's also have a look at the deployment | |
| 151 | ||
| 152 | Note: this deployment does not use an =ImageStream=, so it would work also on native k8s | |
| 153 | ||
| 154 | https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-operator-deployment.yaml | |
| 155 | ||
| 156 | This deployment requires a =ServiceAccount= called "cookieprocessor", this =ServiceAccount= provides | |
| 157 | a Token to authenticate against the API (which we use in the controller script). | |
| 158 | ||
| 159 | * The ServiceAccount | |
| 160 | ||
| 161 | We need a =ServiceAccount=, but that alone will not help. The =ServiceAccount= is NOT member | |
| 162 | of =system:authenticated:oauth=, so it can't read =cookierecipes= based on the =ClusterRoleBinding= we created earlier. | |
| 163 | For that reason we also create a =RoleBinding= (namespaced!) that allows reading recipes: | |
| 164 | ||
| 165 | https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookieprocessor-sa.yaml | |
| 166 | ||
| 167 | * Building the stuff together | |
| 168 | ||
| 169 | #+begin_example | |
| 170 | $ oc create -f cookieprocessor-sa.yaml | |
| 171 | serviceaccount/cookieprocessor created | |
| 172 | rolebinding.rbac.authorization.k8s.io/cookierecipe-view created | |
| 173 | #+end_example | |
| 174 | ||
| 175 | The registry docker.eenfach.de requires login credentials, so we need to set up a secret and link it. | |
| 176 | First login to the registry with *podman login*, then pick the resulting auth.json: | |
| 177 | ||
| 178 | #+begin_example | |
| 179 | $ podman login -u olbohlen docker.eenfach.de | |
| 180 | Password: | |
| 181 | Login Succeeded! | |
| 182 | $ oc create secret generic docker-eenfach-de \ | |
| 183 | > --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json \ | |
| 184 | > --type kubernetes.io/dockerconfigjson | |
| 185 | secret/docker-eenfach-de created | |
| 186 | $ oc secrets link cookieprocessor docker-eenfach-de --for pull | |
| 187 | #+end_example | |
| 188 | ||
| 189 | * Deploying the Operator | |
| 190 | ||
| 191 | Now that we have everything in place, we will just deploy the Operator Pod: | |
| 192 | ||
| 193 | #+begin_example | |
| 194 | $ oc create -f cookie-operator-deployment.yaml | |
| 195 | deployment.apps/recipe-processor created | |
| 196 | $ oc get pod | |
| 197 | NAME READY STATUS RESTARTS AGE | |
| 198 | recipe-processor-7f9969697b-qt9lv 1/1 Running 0 17s | |
| 199 | $ oc logs -f recipe-processor-7f9969697b-qt9lv | |
| 200 | ||
| 201 | ||
| 202 | New recipe found: double-dipped-shortbread | |
| 203 | -------------------------------------------------------------------------- | |
| 204 | ||
| 205 | Pre: we heat up the oven to 180 degrees Celsius | |
| 206 | ||
| 207 | Fetching ingredients from recipe: | |
| 208 | ---------------------------------- | |
| 209 | Fetching 200grams of salted butter (softened) | |
| 210 | [...] | |
| 211 | #+end_example | |
| 212 | ||
| 213 | The Operator will process both sample recipes. | |
| 214 | ||
| 215 | * Test the Operator | |
| 216 | ||
| 217 | We should test if the Operator notices new recipes, so let's create a third | |
| 218 | recipe from | |
| 219 | https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/oaty-hazelnut-cookies.yaml | |
| 220 | ||
| 221 | #+begin_example | |
| 222 | $ oc create -f oaty-hazelnut-cookies.yaml | |
| 223 | cookierecipe.de.eenfach.olbohlen/oaty-hazelnut created | |
| 224 | #+end_example | |
| 225 | ||
| 226 | After a few seconds, we should see in the Operator log: | |
| 227 | #+begin_example | |
| 228 | New recipe found: oaty-hazelnut | |
| 229 | -------------------------------------------------------------------------- | |
| 230 | ||
| 231 | Pre: we heat up the oven to 180 degrees Celsius | |
| 232 | [...] | |
| 233 | #+end_example | |
| 234 | ||
| 235 | * Test for updated recipes | |
| 236 | ||
| 237 | But what if we update a resource? | |
| 238 | Keep the *oc logs -f* on the Operator Pod open, and in another terminal let's patch a recipe. | |
| 239 | ||
| 240 | #+begin_example | |
| 241 | $ oc patch cookierecipes double-dipped-shortbread --type merge \ | |
| 242 | > -p '{"spec":{"temperature":172}}' | |
| 243 | cookierecipe.de.eenfach.olbohlen/double-dipped-shortbread patched | |
| 244 | #+end_example | |
| 245 | ||
| 246 | And again in the log you should see | |
| 247 | ||
| 248 | #+begin_example | |
| 249 | New recipe found: double-dipped-shortbread | |
| 250 | -------------------------------------------------------------------------- | |
| 251 | ||
| 252 | Pre: we heat up the oven to 172 degrees Celsius | |
| 253 | ||
| 254 | Fetching ingredients from recipe: | |
| 255 | ---------------------------------- | |
| 256 | [...] | |
| 257 | #+end_example | |
