We are delighted to announce the new Strimzi 0.11.1 release with some new features!
JBOD storage support for Kafka brokers
Other than supporting the ephemeral and persistent storage, Strimzi now supports JBOD (Just a Bunch of Disks) storage for Kafka brokers (but not for Zookeeper nodes).
The JBOD data storage configuration provides a way for Kafka brokers to make use of multiple disks. JBOD is one approach to providing increased data storage for Kafka brokers. It can also improve performance when there is an independent I/O path for each disk. A JBOD configuration is described by one or more volumes, each of which can be either ephemeral or persistent.
To use JBOD with Strimzi, the storage
type must be set to the new
volumes property allows you to describe the disks that make up your JBOD storage array or configuration.
Here’s an example snippet with a JBOD configuration.
apiVersion: kafka.strimzi.io/v1alpha1 kind: Kafka metadata: name: my-cluster spec: kafka: ... storage: type: jbod volumes: - id: 0 type: persistent-claim size: 100Gi deleteClaim: false - id: 1 type: persistent-claim size: 100Gi deleteClaim: false
The Cluster Operator will configure the
log.dirs Kafka parameter accordingly, in order to store messages logs on the different volumes.
Improved configuration for off-cluster access
This improvement was mostly driven and contributed by the community! A lot of developers asked about having more flexibility and extensibility on the listeners configuration for exposing the Kafka cluster outside of the Kubernetes/OpenShift cluster. Thanks to djotanov for opening a pull request about part of these new features.
When configuring the
listeners.external section of the
Kafka resource, it’s now possible to leverage a new
overrides property for a more powerful configuration.
For example, by default, using a
route listener, the route hostnames are automatically assigned by OpenShift.
Now, you can override the assigned route hostnamess, specifying the names you want to use.
Here’s an example snippet for overriding route hosts.
apiVersion: kafka.strimzi.io/v1alpha1 kind: Kafka metadata: name: my-cluster spec: kafka: ... listeners: external: type: route authentication: type: tls overrides: bootstrap: host: bootstrap.myrouter.com brokers: - broker: 0 host: broker-0.myrouter.com - broker: 1 host: broker-1.myrouter.com - broker: 2 host: broker-2.myrouter.com
nodeport listener is now also more configurable.
By default, the port numbers used for the bootstrap and broker services are automatically assigned by OpenShift or Kubernetes.
This new Strimzi release lets you can override the assigned node ports by specifying the requested port numbers.
Here’s an example snippet for overriding node ports.
apiVersion: kafka.strimzi.io/v1alpha1 kind: Kafka metadata: name: my-cluster spec: kafka: ... listeners: external: type: nodeport tls: true authentication: type: tls overrides: bootstrap: nodePort: 32100 brokers: - broker: 0 nodePort: 32000 - broker: 1 nodePort: 32001 - broker: 2 nodePort: 32002
Finally, a really interesting new feature on listeners is customization of the Kafka brokers’ advertised addresses. By default, Strimzi tries to automatically determine the hostnames and ports that your Kafka cluster advertises to its clients. This is not sufficient in all situations, because the infrastructure on which Strimzi is running might not provide the right hostname or port through which Kafka can be accessed. Now, you can customize the advertised hostname and port. Strimzi will then automatically configure the advertised address in the Kafka brokers and add it to the broker certificates so it can be used for TLS hostname verification.
Here’s an example snippet for overriding advertised addresses.
apiVersion: kafka.strimzi.io/v1alpha1 kind: Kafka metadata: name: my-cluster spec: kafka: ... listeners: external: type: route authentication: type: tls overrides: brokers: - broker: 0 advertisedHost: example.hostname.0 advertisedPort: 12340 - broker: 1 advertisedHost: example.hostname.1 advertisedPort: 12341 - broker: 2 advertisedHost: example.hostname.2 advertisedPort: 12342
You can find many more overrides option in the official documentation in the Kafka broker listeners chapter.
Other than providing the server component for scraping metrics from your applications, the Prometheus project also provides an alerting system through the alert manager component. It is possible to declare alerting rules on the Prometheus server in order to be notified about specific conditions in the metrics. When an alert condition is evaluated as true, Prometheus sends alert data to the alert manager which then sends notifications out. Notifications can be sent via methods such as email, Slack, PagerDuty and HipChat.
The new Strimzi release provides the Kubernetes/OpenShift resources for deploying the alert manager as well as a few examples of alerting rules for Kafka and Zookeeper metrics and related notifications via Slack.
Here’s an example snippet with an alert related to the under replicated partitions metric.
- alert: UnderReplicatedPartitions expr: kafka_server_replicamanager_underreplicatedpartitions > 0 for: 10s labels: severity: warning annotations: summary: 'Kafka under replicated partitions' description: 'There are under replicated partitions on '
… and many more
Other features were included in this new release, the most important ones:
- Support for the Cluster Operator to watch all namespaces for the supported resources.
- Allow users to configure the default ImagePullPolicy related to the containers images download.
- Operator Lifecycle Manager integration which allowed to have Strimzi as part of the OperatorHub.io. You can get more information on this post
Of course, bug fixes are there as well!
This release represents another milestone for this open source project. You can refer to the release change log to get more information.
What are you waiting for? Engage with the community and help us to improve the Strimzi project for running your Kafka cluster on Kubernetes/OpenShift!