…and because it makes total sense, we are going to abuse the K8s API for it.
CustomResourceDefinitions are themselves a Resource, based on a ResourceDefinition
$ oc api-resources | egrep "(NAME|CustomResourceDefinition)"
NAME SHORTNAMES APIVERSION NAMESPACED KIND
customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition
$ oc explain crds
KIND: CustomResourceDefinition
VERSION: apiextensions.k8s.io/v1
DESCRIPTION:
CustomResourceDefinition represents a resource that should be exposed on
the API server. Its name MUST be in the format <.spec.name>.<.spec.group>.
[...]
Let's create the CRD from https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-crd.yaml
$ oc new-project kitchen
Now using project "kitchen" on server "https://api.crc.testing:6443".
$ oc create -f cookie-crd.yaml
Error from server (Forbidden): error when creating "cookie-crd.yaml":
customresourcedefinitions.apiextensions.k8s.io is forbidden: User "developer"
cannot create resource "customresourcedefinitions" in API group "apiextensions.k8s.io"
at the cluster scope
$ oc login -u kubeadmin
$ oc create -f cookie-crd.yaml
customresourcedefinition.apiextensions.k8s.io/cookierecipes.de.eenfach.olbohlen created
$ oc login -u developer
Now the cluster knows about the CRD and we could store recipes!
We try to store sample cookie recipes…but:
$ oc create -f sample-cookie.yaml
Error from server (Forbidden): error when creating "sample-cookie.yaml":
cookierecipes.de.eenfach.olbohlen is forbidden: User "developer" cannot create
resource "cookierecipes" in API group "de.eenfach.olbohlen" in the namespace "kitchen"
Error from server (Forbidden): error when creating "sample-cookie.yaml":
cookierecipes.de.eenfach.olbohlen is forbidden: User "developer" cannot create
resource "cookierecipes" in API group "de.eenfach.olbohlen" in the namespace "kitchen"
We need to set up some RBAC resources first:
ClusterRole that allows viewing recipesClusterRole that allows editing recipesClusterRoleBinding that allows that for authenticated usersApply the RBAC definitions from: https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookie-rbac.yaml
$ oc login -u kubeadmin
$ oc create -f cookie-rbac.yaml
clusterrole.rbac.authorization.k8s.io/cookierecipe-edit created
clusterrole.rbac.authorization.k8s.io/cookierecipe-view created
clusterrolebinding.rbac.authorization.k8s.io/cookierecipe-edit created
The ClusterRoleBinding "cookierecipe-edit" allows system:authenticated:oauth
group members to edit cookierecipes.system:authenticated:oauth contains all users that logged in via the OAuth
service (via an IdentityProvider).
Now we should be able to create the sample recipes:
$ oc login -u developer
$ oc create -f sample-cookie.yaml
cookierecipe.de.eenfach.olbohlen/vintage-chocolate-chip created
cookierecipe.de.eenfach.olbohlen/double-dipped-shortbread created
$ oc get cookierecipe
NAME AGE
double-dipped-shortbread 17s
vintage-chocolate-chip 17s
There is no functionality here - we just stored the recipes in the etcd via the K8s API.
Of course we can oc get -o yaml for example on them and filter:
$ oc get cookierecipe vintage-chocolate-chip -o yaml | yq -y .spec.ingredients[0]
amount: 150
name: salted butter
remarks: softened
unit: grams
This is handy, as we can extract exactly the data which we need at a time.
But…it's a lot of manual work…
Operators were introduced as "Kubernetes Native Applications" and that actually
means nothing. Operators are in the end just Pods.
These Pods run one or more containers, but one container should run a Controller
that can interprete your CustomResources.
So let's write a CookieRecipe Operator. In shell-script… :)
Of course this Operator is not compatible with the OperatorLifecycyleManager (OLM),
so we have to install it manually.
We need:
Then we are going to build the Operator ContainerImage and push it to a Registry.
The Containerfile is here:
https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/Containerfile
the base image is a "kshbase" image, which itself is based upon ubi9 containing also a ksh93
and an oc client.
The controller is written in KornShell 93 (ksh93), which is mostly bash compatible :)
The code is here:
https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/recipe-processor.ksh
Note: this deployment does not use an ImageStream, so it would work also on native k8s
This deployment requires a ServiceAccount called "cookieprocessor", this ServiceAccount provides
a Token to authenticate against the API (which we use in the controller script).
We need a ServiceAccount, but that alone will not help. The ServiceAccount is NOT member
of system:authenticated:oauth, so it can't read cookierecipes based on the ClusterRoleBinding we created earlier.
For that reason we also create a RoleBinding (namespaced!) that allows reading recipes:
https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/cookieprocessor-sa.yaml
$ oc create -f cookieprocessor-sa.yaml
serviceaccount/cookieprocessor created
rolebinding.rbac.authorization.k8s.io/cookierecipe-view created
The registry docker.eenfach.de requires login credentials, so we need to set up a secret and link it.
First login to the registry with podman login, then pick the resulting auth.json:
$ podman login -u olbohlen docker.eenfach.de
Password:
Login Succeeded!
$ oc create secret generic docker-eenfach-de \
> --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json \
> --type kubernetes.io/dockerconfigjson
secret/docker-eenfach-de created
$ oc secrets link cookieprocessor docker-eenfach-de --for pull
Now that we have everything in place, we will just deploy the Operator Pod:
$ oc create -f cookie-operator-deployment.yaml
deployment.apps/recipe-processor created
$ oc get pod
NAME READY STATUS RESTARTS AGE
recipe-processor-7f9969697b-qt9lv 1/1 Running 0 17s
$ oc logs -f recipe-processor-7f9969697b-qt9lv
New recipe found: double-dipped-shortbread
--------------------------------------------------------------------------
Pre: we heat up the oven to 180 degrees Celsius
Fetching ingredients from recipe:
----------------------------------
Fetching 200grams of salted butter (softened)
[...]
The Operator will process both sample recipes.
We should test if the Operator notices new recipes, so let's create a third
recipe from
https://www.eenfach.de/gitblit/blob/~olbohlen!cookie-operator.git/master/oaty-hazelnut-cookies.yaml
$ oc create -f oaty-hazelnut-cookies.yaml
cookierecipe.de.eenfach.olbohlen/oaty-hazelnut created
After a few seconds, we should see in the Operator log:
New recipe found: oaty-hazelnut
--------------------------------------------------------------------------
Pre: we heat up the oven to 180 degrees Celsius
[...]
But what if we update a resource?
Keep the oc logs -f on the Operator Pod open, and in another terminal let's patch a recipe.
$ oc patch cookierecipes double-dipped-shortbread --type merge \
> -p '{"spec":{"temperature":172}}'
cookierecipe.de.eenfach.olbohlen/double-dipped-shortbread patched
And again in the log you should see
New recipe found: double-dipped-shortbread
--------------------------------------------------------------------------
Pre: we heat up the oven to 172 degrees Celsius
Fetching ingredients from recipe:
----------------------------------
[...]
