commit | author | age
|
0cfd14
|
1 |
* I need a cookie recipe database! |
OB |
2 |
|
|
3 |
...and because it makes total sense, we are going to abuse the K8s API for it. |
|
4 |
|
|
5 |
- thankfully we can extend K8s with Custom Resource Definitions (CRDs) |
|
6 |
- but how does it work? |
|
7 |
- =CustomResourceDefinitions= are themselves a =Resource=, based on a =ResourceDefinition= |
|
8 |
|
|
9 |
#+begin_example |
|
10 |
$ oc api-resources | egrep "(NAME|CustomResourceDefinition)" |
|
11 |
NAME SHORTNAMES APIVERSION NAMESPACED KIND |
|
12 |
customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition |
|
13 |
$ oc explain crds |
|
14 |
KIND: CustomResourceDefinition |
|
15 |
VERSION: apiextensions.k8s.io/v1 |
|
16 |
|
|
17 |
DESCRIPTION: |
|
18 |
CustomResourceDefinition represents a resource that should be exposed on |
|
19 |
the API server. Its name MUST be in the format <.spec.name>.<.spec.group>. |
|
20 |
[...] |
|
21 |
#+end_example |
|
22 |
|
|
23 |
* Preparing the cookierecipes CRD |
|
24 |
|
|
25 |
Let's create the CRD from https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-crd.yaml |
|
26 |
|
|
27 |
#+begin_example |
|
28 |
$ oc new-project kitchen |
|
29 |
Now using project "kitchen" on server "https://api.crc.testing:6443". |
|
30 |
$ oc create -f cookie-crd.yaml |
|
31 |
Error from server (Forbidden): error when creating "cookie-crd.yaml": |
|
32 |
customresourcedefinitions.apiextensions.k8s.io is forbidden: User "developer" |
|
33 |
cannot create resource "customresourcedefinitions" in API group "apiextensions.k8s.io" |
|
34 |
at the cluster scope |
|
35 |
|
|
36 |
$ oc login -u kubeadmin |
|
37 |
$ oc create -f cookie-crd.yaml |
|
38 |
customresourcedefinition.apiextensions.k8s.io/cookierecipes.de.eenfach.olbohlen created |
|
39 |
$ oc login -u developer |
|
40 |
#+end_example |
|
41 |
|
|
42 |
Now the cluster knows about the CRD and we could store recipes! |
|
43 |
|
|
44 |
* Storing some sample recipes |
|
45 |
|
|
46 |
We try to store sample cookie recipes...but: |
|
47 |
|
|
48 |
#+begin_example |
|
49 |
$ oc create -f sample-cookie.yaml |
|
50 |
Error from server (Forbidden): error when creating "sample-cookie.yaml": |
|
51 |
cookierecipes.de.eenfach.olbohlen is forbidden: User "developer" cannot create |
|
52 |
resource "cookierecipes" in API group "de.eenfach.olbohlen" in the namespace "kitchen" |
|
53 |
Error from server (Forbidden): error when creating "sample-cookie.yaml": |
|
54 |
cookierecipes.de.eenfach.olbohlen is forbidden: User "developer" cannot create |
|
55 |
resource "cookierecipes" in API group "de.eenfach.olbohlen" in the namespace "kitchen" |
|
56 |
#+end_example |
|
57 |
|
|
58 |
We need to set up some RBAC resources first: |
|
59 |
- a =ClusterRole= that allows viewing recipes |
|
60 |
- a =ClusterRole= that allows editing recipes |
|
61 |
- and a =ClusterRoleBinding= that allows that for authenticated users |
|
62 |
|
|
63 |
* Creating RBAC resources |
|
64 |
|
|
65 |
Apply the RBAC definitions from: https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-rbac.yaml |
|
66 |
#+begin_example |
|
67 |
$ oc login -u kubeadmin |
|
68 |
$ oc create -f cookie-rbac.yaml |
|
69 |
clusterrole.rbac.authorization.k8s.io/cookierecipe-edit created |
|
70 |
clusterrole.rbac.authorization.k8s.io/cookierecipe-view created |
|
71 |
clusterrolebinding.rbac.authorization.k8s.io/cookierecipe-edit created |
|
72 |
#+end_example |
|
73 |
|
|
74 |
The =ClusterRoleBinding= "cookierecipe-edit" allows =system:authenticated:oauth= |
|
75 |
group members to edit =cookierecipes=. |
|
76 |
=system:authenticated:oauth= contains all users that logged in via the OAuth |
|
77 |
service (via an =IdentityProvider=). |
|
78 |
|
|
79 |
* Storing some sample recipes (hopefully this time!!) |
|
80 |
|
|
81 |
Now we should be able to create the sample recipes: |
|
82 |
|
|
83 |
#+begin_example |
|
84 |
$ oc login -u developer |
|
85 |
$ oc create -f sample-cookie.yaml |
|
86 |
cookierecipe.de.eenfach.olbohlen/vintage-chocolate-chip created |
|
87 |
cookierecipe.de.eenfach.olbohlen/double-dipped-shortbread created |
|
88 |
$ oc get cookierecipe |
|
89 |
NAME AGE |
|
90 |
double-dipped-shortbread 17s |
|
91 |
vintage-chocolate-chip 17s |
|
92 |
#+end_example |
|
93 |
|
|
94 |
There is no functionality here - we just stored the recipes in the etcd via the K8s API. |
|
95 |
|
|
96 |
* Now can we do anything with our recipes? |
|
97 |
Of course we can *oc get -o yaml* for example on them and filter: |
|
98 |
|
|
99 |
#+begin_example |
|
100 |
$ oc get cookierecipe vintage-chocolate-chip -o yaml | yq -y .spec.ingredients[0] |
|
101 |
amount: 150 |
|
102 |
name: salted butter |
|
103 |
remarks: softened |
|
104 |
unit: grams |
|
105 |
#+end_example |
|
106 |
|
|
107 |
This is handy, as we can extract exactly the data which we need at a time. |
|
108 |
|
|
109 |
But...it's a lot of manual work... |
|
110 |
|
|
111 |
* I'm an operator with my pocket calculator |
|
112 |
|
|
113 |
Operators were introduced as "Kubernetes Native Applications" and that actually |
|
114 |
means nothing. Operators are in the end just =Pods=. |
|
115 |
|
|
116 |
These Pods run one or more containers, but one container should run a =Controller= |
|
117 |
that can interprete your =CustomResources=. |
|
118 |
|
|
119 |
So let's write a CookieRecipe Operator. In shell-script... :) |
|
120 |
|
|
121 |
Of course this Operator is not compatible with the =OperatorLifecycyleManager= (=OLM=), |
|
122 |
so we have to install it manually. |
|
123 |
|
|
124 |
* What do we need? |
|
125 |
|
|
126 |
We need: |
|
127 |
- a ContainerImage |
|
128 |
- and therefore probably a *Containerfile* |
|
129 |
- Controller code |
|
130 |
|
|
131 |
Then we are going to build the Operator ContainerImage and push it to a Registry. |
|
132 |
|
|
133 |
* Let's review the Containerfile |
|
134 |
|
|
135 |
The Containerfile is here: |
|
136 |
|
|
137 |
https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/Containerfile |
|
138 |
|
|
139 |
the base image is a "kshbase" image, which itself is based upon ubi9 containing also a ksh93 |
|
140 |
and an oc client. |
|
141 |
|
|
142 |
* Have a look at the Controller |
|
143 |
|
|
144 |
The controller is written in KornShell 93 (ksh93), which is mostly bash compatible :) |
|
145 |
|
|
146 |
The code is here: |
|
147 |
|
|
148 |
https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/recipe-processor.ksh |
|
149 |
|
|
150 |
* Now let's also have a look at the deployment |
|
151 |
|
|
152 |
Note: this deployment does not use an =ImageStream=, so it would work also on native k8s |
|
153 |
|
|
154 |
https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-operator-deployment.yaml |
|
155 |
|
|
156 |
This deployment requires a =ServiceAccount= called "cookieprocessor", this =ServiceAccount= provides |
|
157 |
a Token to authenticate against the API (which we use in the controller script). |
|
158 |
|
|
159 |
* The ServiceAccount |
|
160 |
|
|
161 |
We need a =ServiceAccount=, but that alone will not help. The =ServiceAccount= is NOT member |
|
162 |
of =system:authenticated:oauth=, so it can't read =cookierecipes= based on the =ClusterRoleBinding= we created earlier. |
|
163 |
For that reason we also create a =RoleBinding= (namespaced!) that allows reading recipes: |
|
164 |
|
|
165 |
https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookieprocessor-sa.yaml |
|
166 |
|
|
167 |
* Building the stuff together |
|
168 |
|
|
169 |
#+begin_example |
|
170 |
$ oc create -f cookieprocessor-sa.yaml |
|
171 |
serviceaccount/cookieprocessor created |
|
172 |
rolebinding.rbac.authorization.k8s.io/cookierecipe-view created |
|
173 |
#+end_example |
|
174 |
|
|
175 |
The registry docker.eenfach.de requires login credentials, so we need to set up a secret and link it. |
|
176 |
First login to the registry with *podman login*, then pick the resulting auth.json: |
|
177 |
|
|
178 |
#+begin_example |
|
179 |
$ podman login -u olbohlen docker.eenfach.de |
|
180 |
Password: |
|
181 |
Login Succeeded! |
|
182 |
$ oc create secret generic docker-eenfach-de \ |
|
183 |
> --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json \ |
|
184 |
> --type kubernetes.io/dockerconfigjson |
|
185 |
secret/docker-eenfach-de created |
|
186 |
$ oc secrets link cookieprocessor docker-eenfach-de --for pull |
|
187 |
#+end_example |
|
188 |
|
|
189 |
* Deploying the Operator |
|
190 |
|
|
191 |
Now that we have everything in place, we will just deploy the Operator Pod: |
|
192 |
|
|
193 |
#+begin_example |
|
194 |
$ oc create -f cookie-operator-deployment.yaml |
|
195 |
deployment.apps/recipe-processor created |
|
196 |
$ oc get pod |
|
197 |
NAME READY STATUS RESTARTS AGE |
|
198 |
recipe-processor-7f9969697b-qt9lv 1/1 Running 0 17s |
|
199 |
$ oc logs -f recipe-processor-7f9969697b-qt9lv |
|
200 |
|
|
201 |
|
|
202 |
New recipe found: double-dipped-shortbread |
|
203 |
-------------------------------------------------------------------------- |
|
204 |
|
|
205 |
Pre: we heat up the oven to 180 degrees Celsius |
|
206 |
|
|
207 |
Fetching ingredients from recipe: |
|
208 |
---------------------------------- |
|
209 |
Fetching 200grams of salted butter (softened) |
|
210 |
[...] |
|
211 |
#+end_example |
|
212 |
|
|
213 |
The Operator will process both sample recipes. |
|
214 |
|
|
215 |
* Test the Operator |
|
216 |
|
|
217 |
We should test if the Operator notices new recipes, so let's create a third |
|
218 |
recipe from |
|
219 |
https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/oaty-hazelnut-cookies.yaml |
|
220 |
|
|
221 |
#+begin_example |
|
222 |
$ oc create -f oaty-hazelnut-cookies.yaml |
|
223 |
cookierecipe.de.eenfach.olbohlen/oaty-hazelnut created |
|
224 |
#+end_example |
|
225 |
|
|
226 |
After a few seconds, we should see in the Operator log: |
|
227 |
#+begin_example |
|
228 |
New recipe found: oaty-hazelnut |
|
229 |
-------------------------------------------------------------------------- |
|
230 |
|
|
231 |
Pre: we heat up the oven to 180 degrees Celsius |
|
232 |
[...] |
|
233 |
#+end_example |
|
234 |
|
|
235 |
* Test for updated recipes |
|
236 |
|
|
237 |
But what if we update a resource? |
|
238 |
Keep the *oc logs -f* on the Operator Pod open, and in another terminal let's patch a recipe. |
|
239 |
|
|
240 |
#+begin_example |
|
241 |
$ oc patch cookierecipes double-dipped-shortbread --type merge \ |
|
242 |
> -p '{"spec":{"temperature":172}}' |
|
243 |
cookierecipe.de.eenfach.olbohlen/double-dipped-shortbread patched |
|
244 |
#+end_example |
|
245 |
|
|
246 |
And again in the log you should see |
|
247 |
|
|
248 |
#+begin_example |
|
249 |
New recipe found: double-dipped-shortbread |
|
250 |
-------------------------------------------------------------------------- |
|
251 |
|
|
252 |
Pre: we heat up the oven to 172 degrees Celsius |
|
253 |
|
|
254 |
Fetching ingredients from recipe: |
|
255 |
---------------------------------- |
|
256 |
[...] |
|
257 |
#+end_example |