| commit | author | age | ||
| ee916e | 1 | #+TODO: TODO(t) NEXT(n) WAITING(w) SOMEDAY(s) DELEGATED(g) PROJ(p) PLANNED(l) | DONE(d) FORWARDED(f) CANCELLED(c) |
| OB | 2 | #+startup: beamer |
| 3 | #+LaTeX_CLASS: beamer | |
| 4 | #+LaTeX_CLASS_OPTIONS: [a4paper] | |
| 5 | #+LaTeX_CLASS_OPTIONS: [captions=tableheading] | |
| 6 | #+LATEX_HEADER: \usetheme{Warsaw} \usepackage{courier} | |
| 7 | #+LATEX_HEADER: \usepackage{textpos} | |
| 8 | #+LATEX_HEADER: \RequirePackage{fancyvrb} | |
| 9 | #+LATEX_HEADER: \DefineVerbatimEnvironment{verbatim}{Verbatim}{fontsize=\tiny} | |
| c14f64 | 10 | # +LATEX_HEADER: \setbeamercolor{title}{fg=green} |
| OB | 11 | # +LATEX_HEADER: \setbeamercolor{structure}{fg=black} |
| 12 | # +LATEX_HEADER: \setbeamercolor{section in head/foot}{fg=green} | |
| 13 | # +LATEX_HEADER: \setbeamercolor{subsection in head/foot}{fg=green} | |
| 14 | # +LATEX_HEADER: \setbeamercolor{item}{fg=green} | |
| ee916e | 15 | #+LATEX_HEADER: \setbeamerfont{frametitle}{family=\ttfamily} |
| OB | 16 | # logo |
| c14f64 | 17 | # +LATEX_HEADER: \addtobeamertemplate{frametitle}{}{ \begin{textblock*}{100mm}(0.85\textwidth,-0.8cm) \includegraphics[height=0.7cm,width=2cm]{niit-logo.png} \end{textblock*}} |
| OB | 18 | #+OPTIONS: toc:nil ^:nil |
| ee916e | 19 | #+LANGUAGE: en |
| c14f64 | 20 | #+TITLE: Diving into OpenShift |
| ee916e | 21 | |
| OB | 22 | |
| 23 | ||
| c14f64 | 24 | * Helpful tools on the way |
| OB | 25 | - *jq(1)* is a JSON query tool for the command line, a must have |
| 26 | - *yq* is a python script that does YAML queries | |
| 27 | - yes, there's also *xq* for XML but that's for the JAVA folks ;) | |
| 28 | - *jo* to produce JSON output on the shell | |
| 29 | ||
| 30 | How to get? | |
| 31 | - jq: it's very likely shipped as a package with your distribution | |
| 32 | - yq: install via =pip install --user yq= | |
| 33 | - xq: install via =pip install --user xq= | |
| 34 | - jo: get the sources from https://github.com/jpmens/jo | |
| 35 | ||
| 36 | * jq | |
| 37 | The most basic usage is filter down a JSON for a specific attributes value. | |
| 38 | #+begin_src js | |
| 39 | { | |
| 40 | "kind": "ServiceAccount", | |
| 41 | "apiVersion": "v1", | |
| 42 | "metadata": { | |
| 43 | "name": "foobar", | |
| 44 | "creationTimestamp": null | |
| 45 | } | |
| 46 | } | |
| 47 | #+end_src | |
| 48 | ||
| 49 | To get the name of the ServiceAccount you would run | |
| 50 | ||
| 51 | =jq .metadata.name= | |
| 52 | ||
| 53 | If you have a list, and want to see all items, use [], for example: | |
| 54 | ||
| 55 | =jq .items[]= | |
| 56 | ||
| 57 | If you want to get a specific list entry by index, run | |
| 58 | ||
| 59 | =jq .items[5]= to get the sixth entry | |
| 60 | ||
| 61 | * yq | |
| 62 | *yq* works very similar in the basic usage as *jq*, however per default it outputs - confusing enough - JSON (because it's a wrapper for jq). | |
| 63 | If you use the =-y= option, it will output YAML. | |
| 64 | ||
| 65 | I find this very handy for reading data from =ConfigMaps=: | |
| 66 | ||
| 67 | #+begin_example | |
| 68 | $ oc get cm foobar -o yaml | yq -y .data | |
| 69 | key: value | |
| 70 | text: hello world | |
| 71 | #+end_example | |
| 72 | ||
| 73 | * jo | |
| 74 | *jo* is a very special tool that can be very interesting if you have shell commands producing output that | |
| 75 | should be converted into JSON. More examples can be found here: https://jpmens.net/2016/03/05/a-shell-command-to-create-json-jo/ | |
| 76 | ||
| 77 | #+begin_example | |
| 78 | $ ps -fo uid,pid,ppid,stime,tty,time,args | nawk '$NF~/ksh/ { | |
| 79 | > printf("uid=%s pid=%s ppid=%s stime=%s tty=%s cputime=%s comm=%s\n" | |
| 80 | > , $1, $2, $3, $4, $5, $6, $7)}' | while read line; do jo ${line} | |
| 81 | > done | jo -pa | |
| 82 | [ | |
| 83 | { | |
| 84 | "uid": 4100, | |
| 85 | "pid": 4880, | |
| 86 | "ppid": 4001, | |
| 87 | "stime": "Jan_01", | |
| 88 | "tty": "pts/2", | |
| 89 | "cputime": "00:00", | |
| 90 | "comm": "-ksh" | |
| 91 | }, | |
| 92 | { | |
| 93 | "uid": 4100, | |
| 94 | "pid": 20599, | |
| 95 | "ppid": 4880, | |
| 96 | "stime": "23:46:19", | |
| 97 | "tty": "pts/2", | |
| 98 | "cputime": "00:00", | |
| 99 | "comm": "-ksh" | |
| 100 | } | |
| 101 | ] | |
| 102 | #+end_example | |
| 103 | ||
| 104 | * oc get -o jsonpath | |
| 105 | If you use *oc* or *kubectl* to retreive JSON data, you can also filter down with JSONPATH | |
| 106 | instead of using *jq*. The basic syntax is simple, imagine you want the hostname of a route, | |
| 107 | which is in the =host= key under the =spec=: | |
| 108 | ||
| 109 | #+begin_example | |
| 110 | $ oc get -o jsonpath='{"http://"}{.spec.host}{"\n"}' route hi | |
| 111 | http://hi-olbohlen-demo.apps-crc.testing | |
| 112 | #+end_example | |
| 113 | ||
| 114 | And because we are clever, we will create a shell alias from it: | |
| 115 | #+begin_example | |
| 116 | $ alias gr='oc get -o jsonpath="{\"http://\"}{.spec.host}{\"\n\"}" route ' | |
| 117 | [crc@lol cookie-operator]$ gr hi | |
| 118 | http://hi-olbohlen-demo.apps-crc.testing | |
| 119 | $ curl $(gr hi) | |
| 120 | <html> | |
| 121 | <head> | |
| 122 | <title>hello php</title> | |
| 123 | [...] | |
| 124 | #+end_example | |
| 125 | ||
| 126 | * Declarative versus Imperative | |
| 127 | ||
| 128 | It's important to understand that Kubernetes works declarative. | |
| 129 | What does that mean? | |
| 130 | ||
| 131 | Traditionally we work imperative with computers, we tell them what and how to execute. | |
| 132 | The problem here is, that the result is undefined, there are so many side-cases to catch. | |
| 133 | ||
| 134 | If we work declarative, we declare our will to the computer and rely on algorithms that | |
| 135 | shall fulfill our will. Of course this still can fail, but ideally the code results in "failed" or "completed". | |
| 136 | ||
| 137 | * Controller Scheme | |
| 138 | #+begin_src ditaa :file ocp-workshop1.png :cmdline -E -s 0.8 | |
| 139 | ||
| 140 | +---------------------------+ +-----------------------+ +-------------------------+ | |
| 141 | |{s} | | Kube API Server | | Kube Controller Manager | | |
| 142 | | etcd | | | | | | |
| 143 | | | <---> |etcd connector | <---> | watches API for updates | | |
| 144 | |/kubernetes.io/pods/ns/pod | | | | if update: run code | | |
| 145 | | | | REST Connector | | | | |
| 146 | +---------------------------+ +-----------------------+ +-------------------------+ | |
| 147 | ^ | |
| 148 | | HTTP(S)/REST | |
| 149 | | (JSON) | |
| 150 | v | |
| 151 | +-----------------------+ | |
| 152 | | REST Client | | |
| 153 | | | | |
| 154 | | oc/kubectl | | |
| 155 | | (converts JSON to | | |
| 156 | | output format) | | |
| 157 | | | | |
| 158 | +-----------------------+ | |
| 159 | #+end_src | |
| 160 | ||
| 161 | * Syntactic sugar | |
| 162 | OpenShift's *oc* client, but also *kubectl* have various sub-commands to make the start with Kubernetes easier. | |
| 163 | Especially in the beginning the idea to declare everything in JSON or YAML doesn't sound convincing. | |
| 164 | ||
| 165 | But if you look under the hood, commands like *oc new-app* or *kubectl create* will send declarations. | |
| 166 | ||
| 167 | You can validate that by doing a dry-run: | |
| 168 | ||
| 169 | #+begin_example | |
| 170 | $ oc create --dry-run=client -o yaml sa foobar | |
| 171 | apiVersion: v1 | |
| 172 | kind: ServiceAccount | |
| 173 | metadata: | |
| 174 | creationTimestamp: null | |
| 175 | name: foobar | |
| 176 | #+end_example | |
| 177 | ||
| 178 | * Dry-Runs | |
| 179 | There are two ways of dry-runs: =client= and =server= | |
| 180 | ||
| 181 | If we do a client dry-run, nothing gets sent to the kubernetes API, so we basically only test | |
| 182 | if we can convert the local YAML into JSON. Illegal values, missing required keys or immutable values cannot be detected. | |
| 183 | ||
| 184 | if you however use a =server= dry-run, your declaration gets sent to the API, but it will not get persisted into the ETCD. | |
| 185 | ||
| 186 | #+begin_example | |
| 187 | $ oc create --dry-run=server -o yaml sa foobar | |
| 188 | apiVersion: v1 | |
| 189 | kind: ServiceAccount | |
| 190 | metadata: | |
| 191 | creationTimestamp: "2023-01-02T20:33:23Z" | |
| 192 | name: foobar | |
| 193 | namespace: kitchen | |
| 194 | uid: 3bfc1a4c-9a87-4bfa-9dd7-c0941a98ec1f | |
| 195 | $ oc get sa foobar | |
| 196 | Error from server (NotFound): serviceaccounts "foobar" not found | |
| 197 | #+end_example | |
| 198 | ||
| 199 | As you can see, you get an =uid= and a =creationTimestamp= back. However, it will not actually be created. | |
| 200 | ||
| 201 | * Client Side debugging | |
| 202 | Turn on the debugging in *oc* / *kubectl* with =--loglevel=x=, the higher the value, the more verbose. | |
| 203 | If you want to see a lot of details, =--loglevel=9= is very helpful. | |
| 204 | ||
| 205 | For example if you want to validate how server side dry-runs work, run: | |
| 206 | #+begin_example | |
| 207 | $ oc --loglevel=9 create --dry-run=server -o yaml sa foobar | |
| 208 | I0102 21:39:10.590303 1993958 loader.go:372] Config loaded from file: /home/crc/.kube/config | |
| 209 | [...] | |
| 210 | I0102 21:39:10.818643 1993958 request.go:1073] Request Body: {"kind":"ServiceAccount", | |
| 211 | "apiVersion":"v1","metadata":{"name":"foobar","creationTimestamp":null}} | |
| 212 | I0102 21:39:10.818703 1993958 round_trippers.go:466] curl -v -XPOST | |
| 213 | -H "Content-Type: application/json" -H "Accept: application/json, */*" | |
| 214 | -H "User-Agent: oc/4.11.0 (linux/amd64) kubernetes/1928ac4" | |
| 215 | -H "Authorization: Bearer <masked>" 'https://api.crc.testing:6443/api/v1/namespaces/kitchen/ | |
| 216 | serviceaccounts?dryRun=All&fieldManager=kubectl-create&fieldValidation=Ignore' | |
| 217 | I0102 21:39:10.822375 1993958 round_trippers.go:553] POST https://api.crc.testing:6443/api/ | |
| 218 | v1/namespaces/kitchen/serviceaccounts?dryRun=All&fieldManager=kubectl-create& | |
| 219 | fieldValidation=Ignore 201 Created in 3 milliseconds | |
| 220 | [...] | |
| 221 | #+end_example | |
| 222 | ||
| 223 | Very handy is that with =--loglevel=9=, *oc* will also print a *curl* command line that could be used to | |
| 224 | simulate that call. | |
| 225 | ||
| 226 | * Use curl to modify K8s resources | |
| 227 | ||
| 228 | Using the log output from the previous slide, we can now try to create a =ServiceAccount= via *curl*. | |
| 229 | We need of course the JSON declaration...thankfull we can generate that with a client dry-run: | |
| 230 | ||
| 231 | #+begin_example | |
| 232 | $ oc create --dry-run=client -o json sa foobar >/tmp/sa-foobar.json | |
| 233 | $ curl -k -s -XPOST -d "@/tmp/sa-foobar.json" -H "Content-Type: application/json" \ | |
| 234 | > -H "Accept: application/json, */*" -H "User-Agent: oc/4.11.0 (linux/amd64) kubernetes/1928ac4" \ | |
| 235 | > -H "Authorization: Bearer $(oc whoami -t)" \ | |
| 236 | > 'https://api.crc.testing:6443/api/v1/namespaces/api-access/serviceaccounts?fieldManager=\ | |
| 237 | > kubectl-create&fieldValidation=Ignore' | |
| 238 | $ oc get sa foobar | |
| 239 | NAME SECRETS AGE | |
| 240 | foobar 1 11s | |
| 241 | #+end_example | |
| 242 | ||
| 243 | And how to delete the =ServiceAccount=? Simple: | |
| 244 | #+begin_example | |
| 245 | $ curl -k -XDELETE -H "Content-Type: application/json" -H "Accept: application/json, */*" \ | |
| 246 | > -H "User-Agent: oc/4.11.0 (linux/amd64) kubernetes/1928ac4" \ | |
| 247 | > -H "Authorization: Bearer $(oc whoami -t)" \ | |
| 248 | > 'https://api.crc.testing:6443/api/v1/namespaces/api-access/serviceaccounts/foobar' | |
| 249 | {"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"foobar", | |
| 250 | "namespace":"api-access","uid":"94a1c0b4-2177-41b7-bbe2-0ff5e3932785","resourceVersion":"1258915", | |
| 251 | "creationTimestamp":"2023-01-02T20:57:35Z"},"secrets":[{"name":"foobar-dockercfg-brshk"}], | |
| 252 | "imagePullSecrets":[{"name":"foobar-dockercfg-brshk"}]} | |
| 253 | $ oc get sa foobar | |
| 254 | Error from server (NotFound): serviceaccounts "foobar" not found | |
| 255 | #+end_example | |
| 256 | ||
| 257 | * Weird effects with the K8s API | |
| 258 | ||
| 259 | - the API will ignore unkown keys, so you could put a "=Olaf: was here=" key-value pair anywhere you want, the API will happily ignore it! | |
| 260 | - that means, if you *oc edit* a resource and damage the name of a key or dictionary which is not mandatory, it will throw away those definitions (=OAuth/cluster.spec.IdentityProviders=) | |
| 261 | - You can create dependent resources on non-existing objects, i.e. you can give non-existing users privileges: ETCD does not support referential integrity, so no forein key constraints | |
| 262 | - your declaration will be processed after your client terminated the REST connection, so always check the results! | |
| 263 | ||
| 264 | * Short Refresh on RBAC | |
| 265 | Kubernetes supports Role Based Access Controls (RBAC) | |
| 266 | ||
| 267 | We have five different resources here: | |
| 268 | - =Users= | |
| 269 | - =Groups= | |
| 270 | - =ServiceAccounts= | |
| 271 | - =Roles= / =ClusterRoles= | |
| 272 | - =RoleBindings= / =ClusterRoleBindings= | |
| 273 | ||
| 274 | * Users and Groups | |
| 275 | =Users= are a resource that have an access token, to talk to the external API. | |
| 276 | ||
| 277 | =Groups= bundle =Users=, as you would expect. | |
| 278 | ||
| 279 | There are also system Users (they contain colons..) which are *hard coded* and there is no way to list them via the API. | |
| 280 | The same is true for system Groups! | |
| 281 | ||
| 282 | System Users cannot authenticate via the external API. | |
| 283 | ||
| 284 | * ServiceAccounts | |
| 285 | If =Pods= would run under the User who created the =Pod=, others probably couldn't operate on them. | |
| 286 | For that reason we need =ServiceAccounts=, they bundle permissions (=Roles=, =SCCs=), specific secrets (for pulling Images, etc) | |
| 287 | and they have an API Access Token that we can use within the =Pod= to talk to the Kubernetes API. | |
| 288 | ||
| 289 | * Roles / ClusterRoles | |
| 290 | Where is the difference between =Roles= and =ClusterRoles=? | |
| 291 | ||
| 292 | It just says if the resource itself is scoped in a =Namespace= or if it's available for the whole Cluster. | |
| 293 | If you want to define a Role for all Namespaces in your Cluster, you create a =ClusterRole=. | |
| 294 | If you need a local Role just for a specific Namespace, you create a =Role=. | |
| 295 | ||
| 296 | * How does a ClusterRole look like? | |
| 297 | A =Role= contains a list of =Rules=, specifying what is allowed if you own that =Role=. | |
| 298 | The most simple one is the =cluster-admin= =ClusterRole=. | |
| 299 | ||
| 300 | #+begin_example | |
| 301 | $ oc get clusterrole cluster-admin -o yaml | |
| 302 | apiVersion: rbac.authorization.k8s.io/v1 | |
| 303 | kind: ClusterRole | |
| 304 | metadata: | |
| 305 | annotations: | |
| 306 | rbac.authorization.kubernetes.io/autoupdate: "true" | |
| 307 | creationTimestamp: "2022-12-06T10:10:16Z" | |
| 308 | labels: | |
| 309 | kubernetes.io/bootstrapping: rbac-defaults | |
| 310 | name: cluster-admin | |
| 311 | resourceVersion: "75" | |
| 312 | uid: d6fd1146-5bc7-4815-8170-3905ae1e2856 | |
| 313 | rules: | |
| 314 | - apiGroups: | |
| 315 | - '*' | |
| 316 | resources: | |
| 317 | - '*' | |
| 318 | verbs: | |
| 319 | - '*' | |
| 320 | - nonResourceURLs: | |
| 321 | - '*' | |
| 322 | verbs: | |
| 323 | - '*' | |
| 324 | #+end_example | |
| 325 | So you can issue any command verb on any resource in any apiGroup. (iddqd) | |
| 326 | ||
| 327 | * And what is a RoleBinding? | |
| 328 | #+begin_src ditaa :file rbac.png :cmdline -E -s 0.8 | |
| 329 | +------+ +------+ | |
| 330 | | User | | Role | | |
| 331 | | |--------+ +--------------------+ +-----| | | |
| 332 | +------+ +------->| RoleBinding | <---+ +------+ | |
| 333 | | | | |
| 334 | +----------------+ +--->| | | |
| 335 | | ServiceAccount |--+ +--------------------+ | |
| 336 | | | ^ | |
| 337 | +----------------+ | | |
| 338 | | | |
| 339 | +-------+ | | |
| 340 | | Group |----------------------+ | |
| 341 | | | | |
| 342 | +-------+ | |
| 343 | #+end_src | |
| 344 | ||
| 345 | #+begin_example | |
| 346 | $ oc get rolebinding admin -o yaml | |
| 347 | apiVersion: rbac.authorization.k8s.io/v1 | |
| 348 | kind: RoleBinding | |
| 349 | metadata: | |
| 350 | creationTimestamp: "2023-01-02T20:55:05Z" | |
| 351 | name: admin | |
| 352 | namespace: api-access | |
| 353 | resourceVersion: "1258364" | |
| 354 | uid: a59f1f83-86b1-47eb-97ea-985f13d03102 | |
| 355 | roleRef: | |
| 356 | apiGroup: rbac.authorization.k8s.io | |
| 357 | kind: ClusterRole | |
| 358 | name: admin | |
| 359 | subjects: | |
| 360 | - apiGroup: rbac.authorization.k8s.io | |
| 361 | kind: User | |
| 362 | name: kubeadmin | |
| 363 | #+end_example | |
| 364 | ||
| 365 | * And a ClusterRoleBinding? | |
| 366 | ||
| 367 | Well, a =ClusterRoleBinding= binds on a Cluster-scope, so the permissions take effect cluster-wide. | |
| 368 | Note here that there is no =metadata.namespace:= attribute | |
| 369 | #+begin_example | |
| 370 | $ oc get clusterrolebinding cluster-admin -o yaml | |
| 371 | apiVersion: rbac.authorization.k8s.io/v1 | |
| 372 | kind: ClusterRoleBinding | |
| 373 | metadata: | |
| 374 | annotations: | |
| 375 | rbac.authorization.kubernetes.io/autoupdate: "true" | |
| 376 | creationTimestamp: "2022-12-06T10:10:17Z" | |
| 377 | labels: | |
| 378 | kubernetes.io/bootstrapping: rbac-defaults | |
| 379 | name: cluster-admin | |
| 380 | resourceVersion: "159" | |
| 381 | uid: b971b2ac-e3a8-4db9-ba91-f017ec0a7a5f | |
| 382 | roleRef: | |
| 383 | apiGroup: rbac.authorization.k8s.io | |
| 384 | kind: ClusterRole | |
| 385 | name: cluster-admin | |
| 386 | subjects: | |
| 387 | - apiGroup: rbac.authorization.k8s.io | |
| 388 | kind: Group | |
| 389 | name: system:masters | |
| 390 | #+end_example | |
| 391 | ||
| 392 | * I need a cookie receipt database! | |
| 393 | ||
| 394 | ...and because it makes total sense, we are going to abuse the K8s API for it. | |
| 395 | ||
| 396 | - thankfully we can extend K8s with Custom Resource Definitions (CRDs) | |
| 397 | - but how does it work? | |
| 398 | - =CustomResourceDefinitions= are themselves a =Resource=, based on a =ResourceDefinition= | |
| 399 | ||
| 400 | #+begin_example | |
| 401 | $ oc api-resources | egrep "(NAME|CustomResourceDefinition)" | |
| 402 | NAME SHORTNAMES APIVERSION NAMESPACED KIND | |
| 403 | customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition | |
| 404 | $ oc explain crds | |
| 405 | KIND: CustomResourceDefinition | |
| 406 | VERSION: apiextensions.k8s.io/v1 | |
| 407 | ||
| 408 | DESCRIPTION: | |
| 409 | CustomResourceDefinition represents a resource that should be exposed on | |
| 410 | the API server. Its name MUST be in the format <.spec.name>.<.spec.group>. | |
| 411 | [...] | |
| 412 | #+end_example | |
| 413 | ||
| 414 | * Preparing the cookiereceipts CRD | |
| 415 | ||
| 416 | Let's create the CRD from https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-crd.yaml | |
| 417 | ||
| 418 | #+begin_example | |
| 419 | $ oc new-project kitchen | |
| 420 | Now using project "kitchen" on server "https://api.crc.testing:6443". | |
| 421 | $ oc create -f cookie-crd.yaml | |
| 422 | Error from server (Forbidden): error when creating "cookie-crd.yaml": | |
| 423 | customresourcedefinitions.apiextensions.k8s.io is forbidden: User "developer" | |
| 424 | cannot create resource "customresourcedefinitions" in API group "apiextensions.k8s.io" | |
| 425 | at the cluster scope | |
| 426 | ||
| 427 | $ oc login -u kubeadmin | |
| 428 | $ oc create -f cookie-crd.yaml | |
| 429 | customresourcedefinition.apiextensions.k8s.io/cookiereceipts.de.eenfach.olbohlen created | |
| 430 | $ oc login -u developer | |
| 431 | #+end_example | |
| 432 | ||
| 433 | Now the cluster knows about the CRD and we could store receipts! | |
| 434 | ||
| 435 | * Storing some sample receipts | |
| 436 | ||
| 437 | We try to store sample cookie receipts...but: | |
| 438 | ||
| 439 | #+begin_example | |
| 440 | $ oc create -f sample-cookie.yaml | |
| 441 | Error from server (Forbidden): error when creating "sample-cookie.yaml": | |
| 442 | cookiereceipts.de.eenfach.olbohlen is forbidden: User "developer" cannot create | |
| 443 | resource "cookiereceipts" in API group "de.eenfach.olbohlen" in the namespace "kitchen" | |
| 444 | Error from server (Forbidden): error when creating "sample-cookie.yaml": | |
| 445 | cookiereceipts.de.eenfach.olbohlen is forbidden: User "developer" cannot create | |
| 446 | resource "cookiereceipts" in API group "de.eenfach.olbohlen" in the namespace "kitchen" | |
| 447 | #+end_example | |
| 448 | ||
| 449 | We need to set up some RBAC resources first: | |
| 450 | - a =ClusterRole= that allows viewing receipts | |
| 451 | - a =ClusterRole= that allows editing receipts | |
| 452 | - and a =ClusterRoleBinding= that allows that for authenticated users | |
| 453 | ||
| 454 | * Creating RBAC resources | |
| 455 | ||
| 456 | Apply the RBAC definitions from: https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-rbac.yaml | |
| 457 | #+begin_example | |
| 458 | $ oc login -u kubeadmin | |
| 459 | $ oc create -f cookie-rbac.yaml | |
| 460 | clusterrole.rbac.authorization.k8s.io/cookiereceipt-edit created | |
| 461 | clusterrole.rbac.authorization.k8s.io/cookiereceipt-view created | |
| 462 | clusterrolebinding.rbac.authorization.k8s.io/cookiereceipt-edit created | |
| 463 | #+end_example | |
| 464 | ||
| 465 | The =ClusterRoleBinding= "cookiereceipt-edit" allows =system:authenticated:oauth= | |
| 466 | group members to edit =cookiereceipts=. | |
| 467 | =system:authenticated:oauth= contains all users that logged in via the OAuth | |
| 468 | service (via an =IdentityProvider=). | |
| 469 | ||
| 470 | * Storing some sample receipts (hopefully this time!!) | |
| 471 | ||
| 472 | Now we should be able to create the sample receipts: | |
| 473 | ||
| 474 | #+begin_example | |
| 475 | $ oc login -u developer | |
| 476 | $ oc create -f sample-cookie.yaml | |
| 477 | cookiereceipt.de.eenfach.olbohlen/vintage-chocolate-chip created | |
| 478 | cookiereceipt.de.eenfach.olbohlen/double-dipped-shortbread created | |
| 479 | $ oc get cookiereceipt | |
| 480 | NAME AGE | |
| 481 | double-dipped-shortbread 17s | |
| 482 | vintage-chocolate-chip 17s | |
| 483 | #+end_example | |
| 484 | ||
| 485 | There is no functionality here - we just stored the receipts in the etcd via the K8s API. | |
| 486 | ||
| 487 | * Can we validate that in the ETCD? | |
| 488 | ||
| 489 | Yes, we can use *etcdctl* to look into the db | |
| 490 | ||
| 491 | #+begin_example | |
| 492 | $ oc login -u kubeadmin | |
| 493 | $ oc rsh -n openshift-etcd -c etcdctl \ | |
| 494 | > etcd-crc-pbwlw-master-0 etcdctl get / --prefix --keys-only | grep -i cookie | |
| 495 | /kubernetes.io/apiextensions.k8s.io/customresourcedefinitions/cookiereceipts.de.eenfach.olbohlen | |
| 496 | /kubernetes.io/apiserver.openshift.io/apirequestcounts/cookiereceipts.v1.de.eenfach.olbohlen | |
| 497 | /kubernetes.io/clusterrolebindings/cookiereceipt-edit | |
| 498 | /kubernetes.io/clusterroles/cookiereceipt-edit | |
| 499 | /kubernetes.io/clusterroles/cookiereceipt-view | |
| 500 | /kubernetes.io/de.eenfach.olbohlen/cookiereceipts/kitchen/double-dipped-shortbread | |
| 501 | /kubernetes.io/de.eenfach.olbohlen/cookiereceipts/kitchen/vintage-chocolate-chip | |
| 502 | /kubernetes.io/secrets/openshift-machine-config-operator/cookie-secret | |
| 503 | #+end_example | |
| 504 | ||
| 505 | The =etcd pod= in the =openshift-etcd= project has a sidecar container called =etcdctl= | |
| 506 | which contains the *etcdctl* utility and the correct environment. | |
| 507 | The data hierarchy starts with =/= and with =--keys-only= we don't get the values. | |
| 508 | ||
| 509 | * Let's look into a receipt | |
| 510 | ||
| 511 | Check if we can *etcdctl get* on a known receipt: | |
| 512 | #+begin_example | |
| 513 | $ oc rsh -n openshift-etcd -c etcdctl etcd-crc-pbwlw-master-0 etcdctl \ | |
| 514 | > get /kubernetes.io/de.eenfach.olbohlen/cookiereceipts/kitchen/vintage-chocolate-chip | |
| 515 | /kubernetes.io/de.eenfach.olbohlen/cookiereceipts/kitchen/vintage-chocolate-chip | |
| 516 | {"apiVersion":"de.eenfach.olbohlen/v1","kind":"CookieReceipt","metadata":{ | |
| 517 | "creationTimestamp":"2023-01-02T15:18:49Z","generation":1,"managedFields": | |
| 518 | [{"apiVersion":"de.eenfach.olbohlen/v1","fieldsType":"FieldsV1", | |
| 519 | ||
| 520 | [...] | |
| 521 | #+end_example | |
| 522 | With *etcdctl put*, we could also update the entry and *etcdctl watch* allows us to | |
| 523 | see updates to reosurces. | |
| 524 | ||
| 525 | #+begin_example | |
| 526 | $ oc rsh -n openshift-etcd -c etcdctl etcd-crc-pbwlw-master-0 etcdctl watch \ | |
| 527 | > --prefix /kubernetes.io/de.eenfach.olbohlen/cookiereceipts | |
| 528 | PUT | |
| 529 | /kubernetes.io/de.eenfach.olbohlen/cookiereceipts/kitchen/double-dipped-shortbread | |
| 530 | {"apiVersion":"de.eenfach.olbohlen/v1","kind":"CookieReceipt","metadata":{"annotations": | |
| 531 | {"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\": | |
| 532 | \"de.eenfach.olbohlen/v1\",\"kind\":\"CookieReceipt\",\"metadata\":{ | |
| 533 | [...] | |
| 534 | #+end_example | |
| 535 | (run an *oc* apply/replace/create/delete in another terminal) | |
| 536 | ||
| 537 | * Now can we do anything with our receipts? | |
| 538 | Of course we can *oc get -o yaml* for example on them and filter: | |
| 539 | ||
| 540 | #+begin_example | |
| 541 | $ oc get cookiereceipt vintage-chocolate-chip -o yaml | yq -y .spec.ingredients[0] | |
| 542 | amount: 150 | |
| 543 | name: salted butter | |
| 544 | remarks: softened | |
| 545 | unit: grams | |
| 546 | #+end_example | |
| 547 | ||
| 548 | This is handy, as we can extract exactly the data which we need at a time. | |
| 549 | ||
| 550 | But...it's a lot of manual work... | |
| 551 | ||
| 552 | * I'm an operator with my pocket calculator | |
| 553 | ||
| 554 | Operators were introduced as "Kubernetes Native Applications" and that actually | |
| 555 | means nothing. Operators are in the end just =Pods=. | |
| 556 | ||
| 557 | These Pods run one or more containers, but one container should run a =Controller= | |
| 558 | that can interprete your =CustomResources=. | |
| 559 | ||
| 560 | So let's write a CookieReceipt Operator. In shell-script... :) | |
| 561 | ||
| 562 | Of course this Operator is not compatible with the =OperatorLifecycyleManager= (=OLM=), | |
| 563 | so we have to install it manually. | |
| 564 | ||
| 565 | * What do we need? | |
| 566 | ||
| 567 | We need: | |
| 568 | - a ContainerImage | |
| 569 | - and therefore probably a *Containerfile* | |
| 570 | - Controller code | |
| 571 | ||
| 572 | Then we are going to build the Operator ContainerImage and push it to a Registry. | |
| 573 | ||
| 574 | * Let's review the Containerfile | |
| 575 | ||
| 576 | The Containerfile is here: | |
| 577 | ||
| 578 | https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/Containerfile | |
| 579 | ||
| 580 | the base image is a "kshbase" image, which itself is based upon ubi9 containing also a ksh93 | |
| 581 | and an oc client. | |
| 582 | ||
| 583 | * Have a look at the Controller | |
| 584 | ||
| 585 | The controller is written in KornShell 93 (ksh93), which is mostly bash compatible :) | |
| 586 | ||
| 587 | The code is here: | |
| 588 | ||
| 589 | https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/receipt-processor.ksh | |
| 590 | ||
| 591 | * Now let's also have a look at the deployment | |
| 592 | ||
| 593 | Note: this deployment does not use an =ImageStream=, so it would work also on native k8s | |
| 594 | ||
| 595 | https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-operator-deployment.yaml | |
| 596 | ||
| 597 | This deployment requires a =ServiceAccount= called "cookieprocessor", this =ServiceAccount= provides | |
| 598 | a Token to authenticate against the API (which we use in the controller script). | |
| 599 | ||
| 600 | * The ServiceAccount | |
| 601 | ||
| 602 | We need a =ServiceAccount=, but that alone will not help. The =ServiceAccount= is NOT member | |
| 603 | of =system:authenticated:oauth=, so it can't read =cookiereceipts= based on the =ClusterRoleBinding= we created earlier. | |
| 604 | For that reason we also create a =RoleBinding= (namespaced!) that allows reading receipts: | |
| 605 | ||
| 606 | https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookieprocessor-sa.yaml | |
| 607 | ||
| 608 | * Building the stuff together | |
| 609 | ||
| 610 | #+begin_example | |
| 611 | $ oc create -f cookieprocessor-sa.yaml | |
| 612 | serviceaccount/cookieprocessor created | |
| 613 | rolebinding.rbac.authorization.k8s.io/cookiereceipt-view created | |
| 614 | #+end_example | |
| 615 | ||
| 616 | The registry docker.eenfach.de requires login credentials, so we need to set up a secret and link it. | |
| 617 | First login to the registry with *podman login*, then pick the resulting auth.json: | |
| 618 | ||
| 619 | #+begin_example | |
| 620 | $ podman login -u olbohlen docker.eenfach.de | |
| 621 | Password: | |
| 622 | Login Succeeded! | |
| 623 | $ oc create secret generic docker-eenfach-de \ | |
| 624 | > --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json \ | |
| 625 | > --type kubernetes.io/dockerconfigjson | |
| 626 | secret/docker-eenfach-de created | |
| 627 | $ oc secrets link cookieprocessor docker-eenfach-de --for pull | |
| 628 | #+end_example | |
| 629 | ||
| 630 | * Deploying the Operator | |
| 631 | ||
| 632 | Now that we have everything in place, we will just deploy the Operator Pod: | |
| 633 | ||
| 634 | #+begin_example | |
| 635 | $ oc create -f cookie-operator-deployment.yaml | |
| 636 | deployment.apps/receipt-processor created | |
| 637 | $ oc get pod | |
| 638 | NAME READY STATUS RESTARTS AGE | |
| 639 | receipt-processor-7f9969697b-qt9lv 1/1 Running 0 17s | |
| 640 | $ oc logs -f receipt-processor-7f9969697b-qt9lv | |
| ee916e | 641 | |
| OB | 642 | |
| c14f64 | 643 | New receipt found: double-dipped-shortbread |
| OB | 644 | -------------------------------------------------------------------------- |
| 645 | ||
| 646 | Pre: we heat up the oven to 180 degrees Celsius | |
| 647 | ||
| 648 | Fetching ingredients from receipt: | |
| 649 | ---------------------------------- | |
| 650 | Fetching 200grams of salted butter (softened) | |
| 651 | [...] | |
| 652 | #+end_example | |
| 653 | ||
| 654 | The Operator will process both sample receipts. | |
| 655 | ||
| 656 | * Test the Operator | |
| 657 | ||
| 658 | We should test if the Operator notices new receipts, so let's create a third | |
| 659 | receipt from | |
| 660 | https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/oaty-hazelnut-cookies.yaml | |
| 661 | ||
| 662 | #+begin_example | |
| 663 | $ oc create -f oaty-hazelnut-cookies.yaml | |
| 664 | cookiereceipt.de.eenfach.olbohlen/oaty-hazelnut created | |
| 665 | #+end_example | |
| 666 | ||
| 667 | After a few seconds, we should see in the Operator log: | |
| 668 | #+begin_example | |
| 669 | New receipt found: oaty-hazelnut | |
| 670 | -------------------------------------------------------------------------- | |
| 671 | ||
| 672 | Pre: we heat up the oven to 180 degrees Celsius | |
| 673 | [...] | |
| 674 | #+end_example | |
| 675 | ||
| 676 | * Test for updated receipts | |
| 677 | ||
| 678 | But what if we update a resource? | |
| 679 | Keep the *oc logs -f* on the Operator Pod open, and in another terminal let's patch a receipt. | |
| 680 | ||
| 681 | #+begin_example | |
| 682 | $ oc patch cookiereceipts double-dipped-shortbread --type merge \ | |
| 683 | > -p '{"spec":{"temperature":172}}' | |
| 684 | cookiereceipt.de.eenfach.olbohlen/double-dipped-shortbread patched | |
| 685 | #+end_example | |
| 686 | ||
| 687 | And again in the log you should see | |
| 688 | ||
| 689 | #+begin_example | |
| 690 | New receipt found: double-dipped-shortbread | |
| 691 | -------------------------------------------------------------------------- | |
| 692 | ||
| 693 | Pre: we heat up the oven to 172 degrees Celsius | |
| 694 | ||
| 695 | Fetching ingredients from receipt: | |
| 696 | ---------------------------------- | |
| 697 | [...] | |
| 698 | #+end_example | |
| ee916e | 699 | |
| OB | 700 | * Thank You |
| 701 | ||
| 702 | Thank you for your attention.\\ | |
| 703 | \\ | |
| 704 | Do you have any questions?\\ | |
| 705 | \\ | |
| 706 | Feel free to ask now or contact me later: | |
| 707 | ||
| 708 | [[mailto:olaf.bohlen@niit.com][olaf.bohlen@niit.com]] | |
