Announcing Eventing RabbitMQ moving to GA¶

Author: Gab Satchi, Senior Software Engineer @ VMware

Date: 2022-07-26

We’re excited to announce that Eventing RabbitMQ is now Generally Available (GA). With this milestone, the APIs provided will remain stable in future iterations. A quick start can be found here.

Key Features¶

Use any RabbitMQ instance¶

Eventing RabbitMQ works well with RabbitMQ clusters that have been provisioned by using the RabbitMQ Cluster Kubernetes Operator, but can also be configured to use RabbitMQ instances provisioned by other means. The RabbitMQ Broker and Source both contain a rabbitmqClusterReference field, which can either reference a RabbitMQ cluster that was created by using the Operator, or a Secret containing the credentials for a RabbitMQ instance. Full documentation can be found here.

Parallelism and event ordering¶

RabbitMQ by default uses a first-in-first-out (FIFO) ordering mode for events. However, there can be use cases where ordering isn't required and higher throughput may be preferred. Both the RabbitMQ Broker and Source contain a parallelism annotation that can be configured to process more than one message in parallel. An example outlining both FIFO and high throughput modes can be found here.

Quorum queues¶

Quorum queues are a fault tolerant, replicated queue type that are used in multi-node setups. RabbitMQ uses quorum queues by default. The queue type can be configured for RabbitMQ Brokers, and any Triggers subscribed to a Broker will inherit the queue type. Full documentation can be found here

Metrics¶

Eventing RabbitMQ mirrors a subset of the eventing metrics found here. Due to the single-tenant nature, Broker - Filter metrics aren't applicable.

Architecture¶

Eventing RabbitMQ follows a single tenant model where dedicated resources are created for each Broker, Trigger, and Source object. While there is some overhead to this approach, there are also benefits: - Pods can be autoscaled to handle the expected traffic for a given Broker, Trigger, or Source. - User's existing policies and roles can be used for role based access control (RBAC). - Increased transparency into what is running on the cluster.

Eventing RabbitMQ broker architecture

Creating a Broker automatically creates an ingress pod that can receive CloudEvents, and the RabbitMQ resources needed to route CloudEvents through the RabbitMQ system. Triggers create dispatcher pods that consume messages from RabbitMQ, and send CloudEvents to an addressable resource.

Eventing RabbitMQ source architecture

RabbitMQSource objects can retrieve messages from a queue, convert them to CloudEvents, and send these CloudEvents to addressable consumers.

Performance¶

A suite of performance tests are run with each release to ensure adequate throughput and acceptable latency. The results of the latest run can be found here.

Adopters¶

Eventing RabbitMQ is currently in use by VMware Event Broker Appliance (VEBA), Cloud Native Runtimes (CNR).

Feedback¶

We will appreciate community feedback on trying this new release and reporting back any issues or queries that you might have. New issues can be created here.