• Creating Strimzi documentation assemblies

    Good documentation tells you exactly how to do that thing you want to do as clearly as possible. And it’s a beautiful thing. Some say it’s a rare thing. Actually, there is a lot of great documentation around. But great documentation tends to be invisible, quietly doing its job without being noticed. You step through that entirely accurate and perfectly written procedure and move on. Bad documentation, on the other hand, is far easier to notice. And once noticed, it is consequently ignored.

    We want the Strimzi documentation to be the great (invisible) kind. The kind you want to return to. And we want you to help us make it that way.

  • Deploying Debezium using the new KafkaConnector resource

    For too long our Kafka Connect story hasn’t been quite as “Kubernetes-native” as it could have been. We had a KafkaConnect resource to configure a Kafka Connect cluster but you still had to use the Kafka Connect REST API to actually create a connector within it. While this wasn’t especially difficult using something like curl, it stood out because everything else could be done using kubectl and it meant that connectors didn’t fit into our Kubernetes-native vision. With the help of a contribution from the community, Strimzi now supports a KafkaConnector custom resource and the rest of this blog post is going to explain how to use it using Debezium as an example. As if that wasn’t enough, there’s some awesome ASCII art to help explain how it all fits together. So read on to find out about this new Kubernetes-native way of managing connectors.

  • Using Strimzi with Amazon Network load balancers

    Strimzi supports accessing Apache Kafka using load balancers. We already have a full blog post dedicated to load balancers. But when running Strimzi on Amazon AWS, it will by default use the so called Classic Load Balancers. A lot of users seem to be asking about using the newer Network Load Balancers with Strimzi and Kafka. So in this blog post we will look at whether and how Strimzi could work with Network load balancers.

  • A look back at 2019

    The last year has been a busy one for Strimzi with many highlights. We wanted to reflect on and celebrate a few of them.

  • Strimzi at KubeCon+CloudNativeCon in San Diego

    This week, most of the Cloud Native and Kubernetes world meets in San Diego for the KubeCon+CloudNativeCon conference organized by Cloud Native Computing Foundation (CNCF). As a CNCF Sandbox project, Strimzi will, of course, be represented there as well. Read more to learn how to meet Strimzi maintainers and see Strimzi demos.

  • Exposing your Apache Kafka cluster through the HTTP bridge

    In the previous blog post about HTTP Bridge, we introduced the new Strimzi HTTP bridge component. Using the bridge, it is possible to interact with an Apache Kafka cluster through the HTTP/1.1 protocol instead of the native Kafka protocol. We already covered how simple it is to deploy the bridge through the new KafkaBridge custom resource and via the Strimzi Cluster Operator. We also covered all the REST endpoints that the bridge provides as an API for consumers and producers. In this blog post we are going to describe how to expose the bridge outside of the Kubernetes (or OpenShift) cluster, where it is running alongside the Kafka cluster. We will also show real examples using curl commands.

  • Kafka authentication using OAuth 2.0

    In Strimzi 0.14.0 we have added an additional authentication option to the standard set supported by Kafka brokers. Your Kafka clients can now use OAuth 2.0 token-based authentication when establishing a session to a Kafka broker. With this kind of authentication Kafka clients and brokers talk to a central OAuth 2.0 compliant authorization server. Clients use the authorization server to obtain access tokens, or are configured with access tokens issued by the server. Brokers communicate with authorization server to validate the tokens presented by clients, thus confirming their identities. In this model, the administrator doesn’t create users in Kafka. Rather, users are typically managed through identity providers exposed over LDAP. Strimzi client components like Kafka Bridge, Kafka Connect, and Kafka Mirror Maker can be configured with OAuth 2.0 authentication through custom resource definitions.

  • Improving Prometheus metrics

    In our previous blog post we focused on tracing which was one of the new features in our 0.14.0 release. But tracing is not the only improvement to the monitoring capabilities we did in 0.14.0. We also have some significant improvements to Prometheus monitoring. Strimzi has supported Prometheus for Kafka metrics almost since the beginning. But in 0.14.0 we have made some major improvements by adding support for the Kafka Exporter tool. Kafka Exporter adds some additional metrics that are missing in Kafka brokers. Learn more about them in this blog post.

  • Strimzi, Apache Kafka and Tracing

    Distributed tracing is a technique used to profile and monitor applications which span more than one process. It is especially useful for distributed architectures - for example based on microservices. Tracing helps to track performance bottlenecks and pinpoint where failures occur. Strimzi 0.14.0 allows you to enable tracing in some of the components which are part of Apache Kafka - in particular in Kafka Connect and Kafka Mirror Maker. This blog post will show how to use tracing not just within the Strimzi project but also in Kafka clients generally.

  • What's new in Strimzi 0.14.0

    We are delighted to announce the availability of Strimzi 0.14.0. This is our first release as a CNCF Sandbox project and it is a big release with lot of new features. This blog post will give you an overview of the new features and in the following days more detailed blog posts will be dedicated to each of them.

  • Using Strimzi with Knative

    Knative is a platform for running Serverless workloads on Kubernetes clusters. It provides middleware abstractions and APIs that ease the development and deployment lifecycle in order to allow developers to focus on application logic. For example, Knative deals with the networking and exposing of applications, tracking of versions and scaling (including to zero). Further, it offers declarative ways to bind applications to events and data streams including those based on Apache Kafka.

  • Strimzi Apache Kafka Operator joins the CNCF

    We are excited to announce that Strimzi has joined the Cloud Native Computing Foundation (CNCF) as a Sandbox Project. The CNCF oversees many projects that provide critical infrastructure in cloud based environments and we are thrilled to be part of that community. From the core Kubernetes project to exciting early stage projects such as CloudEvents we feel Strimzi is in esteemed company.

  • Hacking Strimzi for Cruise Control

    Cruise Control is a highly customizable software tool for simplifying the monitoring and balancing of workloads across a Kafka cluster. The flexibility of Cruise Control makes it easy to customize for different environments and use cases. This makes it possible to extend Cruise Control to work with Strimzi in an Kubernetes environment with a reasonable amount of effort. This post will walk through how to hook Cruise Control up to a Strimzi deployed Kafka cluster.

  • Bridging the HTTP protocol to Apache Kafka

    Apache Kafka uses a custom protocol on top of TCP/IP for communication between applications and the cluster. There are many client implementations for different programming languages, from Java to Golang, from Python to C# and many more. These libraries simplify development because they abstract the Apache Kafka protocol on the wire; every client implementation provides a simple API for sending and receiving messages, or performing administrative operations on the cluster. However, there are scenarios where it is not possible to use the clients, or indeed the native protocol. Communicating with an Apache Kafka cluster using a standard protocol like HTTP/1.1 eases development these scenarios. This blog post will introduce the new HTTP - Apache Kafka bridge that’s available as part of the Strimzi 0.12 release

  • Persistent storage improvements

    A few months ago I wrote a blog post about how you can manually increase the size of the persistent volumes you use for Kafka or Zookeeper storage. I promised that one day it would be supported directly in Strimzi. And that’s what happened in the Strimzi 0.12 release. In this post I’m going to tell you more about the improvements to persistent storage in the 0.12 release.

  • Accessing Kafka: Part 5 - Ingress

    In the fifth and last part of this blog post series we will look at exposing Kafka using Kubernetes Ingress. This post will explain how to use Ingress controllers on Kubernetes, how Ingress compares with OpenShift Routes and how it can be used with Strimzi and Kafka. Off-cluster access using Kubernetes Ingress is available only from Strimzi 0.12.0.

  • Accessing Kafka: Part 4 - Load Balancers

    In the fourth part of this blog post series we will look at exposing Kafka using load balancers. This post will explain how to use load balancers in public cloud environments and how they can be used with Kafka.

  • Accessing Kafka: Part 3 - OpenShift Routes

    In the third part of this blog post series we will look at exposing Kafka using OpenShift Routes. This post will explain how routes work and how they can be used with Kafka. Routes are available only on OpenShift. But if you are a Kubernetes user, don’t be sad, one of the next parts will be about using Kubernetes Ingress which is similar to OpenShift routes.

  • Accessing Kafka: Part 2 - Node ports

    In the second part of this blog post series we will look at exposing Kafka using node ports. This post will explain how node ports work and how can they be used with Kafka. It will also cover the different configuration options which are available to users and cover the main pros and cons of using node ports.

  • Accessing Kafka: Part 1 - Introduction

    Scalability is one of the flagship features of Apache Kafka. It is achieved by partitioning the data and distributing them across multiple brokers. Such data sharding has also a big impact on how Kafka clients connect to the brokers. This is especially visible when Kafka is running within a platform like Kubernetes but is accessed from outside of that platform. This blog-series will explain how Kafka and its clients work and how Strimzi deals with it to make it accessible for clients running outside of Kubernetes.

  • Strimzi 0.11.1 released!

    We are delighted to announce the new Strimzi 0.11.1 release with some new features!

  • Deploy Apache Kafka in few seconds with Strimzi and Operator Framework and Operator Lifecycle Manager

    Operator Framework is an open source toolkit for writing and managing Kubernetes Operators. It consists of several components and one of them is the Operator Lifecycle Manager (OLM). OLM is an Operator for Operators. It provides a declarative way for installing and managing operators. It can make installation and management of Operators on Kubernetes much easier. And Strimzi is part of it.

  • Resizing persistent volumes with Strimzi

    Do you know the feeling when you run out of disk space? When you want to save some new data to your disk but there is not enough space? It has probably happened to most of us at least once in our life. So, what should you do when this happens to your Kafka cluster?

  • Strimzi 0.9.0 released!

    We are delighted to announce the new Strimzi 0.9.0 release with some new features!

  • Using the Vault PKI secrets engine to set up a CA to use with the Strimzi Kafka Operator

    Securing Kafka can be difficult. Securing Kafka on Kubernetes can also be diffult. But using Strimzi’s Kafka Operator makes it easy!

    Below you will find a guide on how to use the Vault PKI secrets engine to generate an intermediate CA to use with the Strimzi Kafka Operator.

  • Using Helm to deploy the Strimzi Kafka Operator

    The Kubernetes Helm project is the leading way to package, configure, and deploy Kubernetes resources. Let’s see how it’s possible to use it in order to deploy an Apache Kafka cluster on Kubernetes through the Strimzi project.

  • Strimzi 0.8.0 released!

    We are delighted to announce the new Strimzi 0.8.0 release with many awesome new features!

  • Strimzi 0.7.0 released!

    We are delighted to announce the new Strimzi 0.7.0 release with many awesome new features!

  • Strimzi 0.6.0 released!

    We are delighted to announce the new Strimzi 0.6.0 release with many awesome new features!

  • Running Kafka on dedicated Kubernetes nodes

    When running applications on OpenShift or Kubernetes, different pods get colocated on the same nodes. That is very useful, because it helps to decrease costs by better utilizing the available hardware. But it can also cause some issues. One of which is the noisy neighbour phenomenon. This is a situation when multiple applications which are intensively utilizing the same resource - for example network bandwidth or disk I/O - are scheduled on the same node. How do you avoid such problems when running Apache Kafka on OpenShift or Kubernetes with Strimzi? And how do you make sure your Apache Kafka delivers the best performance?

  • Strimzi 0.5.0 released!

    We are delighted to announce the new Strimzi 0.5.0 release with many awesome new features!

  • Deploying Kafka on Kubernetes with Local storage using Strimzi

    When deploying new Kafka cluster, one of the first decisions you have to make is the type of storage you want to use. Even before deciding on such storage aspects such as the type of filesystem which should be used, you have to decide a more basic question: should you run Kafka with local storage or with network attached storage?

  • Running Strimzi on OpenShift Online

    OpenShift Online is a managed OpenShift-as-a-Service offering from Red Hat. It has a Starter tier and a Pro tier. The Starter tier is for free, but offers only very limited resources. In the Pro tier, you can buy additional resources. Can it be used to run Apache Kafka using the Strimzi project? Of course it can! But OpenShift Online has some limitations. So we will need to do some modifications to the default Strimzi installation files to get it up and running.

  • Strimzi 0.4.0 released!

    We are really happy to announce the new Strimzi 0.4.0 release with a lot of fixes and interesting new features!

  • Strimzi at Red Hat Summit

    If your plan is to attend the Red Hat Summit 2018 in San Francisco (May 8-10) then you have to know that the Strimzi project will be there as well !

    The offered content is really awesome because there will be one session and a dedicated lab focused on it.

  • Apache Kafka on Kubernetes and OpenShift: Barnabas is dead ... long live Strimzi!

    this is a re-publication of the following blog post

    Almost one year and half ago, I started my journey about running Apache Kafka on Kubernetes and OpenShift. At that time, these containers orchestration platforms were focused on “stateless” (micro)services so there wasn’t a real support for a technology like Apache Kafka which is “stateful” by definition.