Confluent Kafka 2.3.0-4.0.0e

DC/OS Confluent Kafka is an automated service that makes it easy to deploy and manage Confluent Kafka on Mesosphere DC/OS, eliminating nearly all of the complexity traditionally associated with managing a Kafka cluster. Confluent Kafka is a distributed high-throughput publish-subscribe messaging system with strong ordering guarantees. Kafka clusters are highly available, fault tolerant, and very durable. DC/OS Confluent Kafka gives you direct access to the Confluent Kafka API so that existing producers and consumers can interoperate. You can configure and install DC/OS Confluent Kafka in moments. Multiple Confluent Kafka clusters can be installed on DC/OS and managed independently, so you can offer Confluent Kafka as a managed service to your organization.

Benefits

DC/OS Confluent Kafka offers the following benefits of a semi-managed service:

  • Easy installation
  • Multiple Confluent Kafka clusters
  • Elastic scaling of brokers
  • Replication and graceful shutdown for high availability
  • Confluent Kafka cluster and broker monitoring

Features

DC/OS Confluent Kafka provides the following features:

  • Single-command installation for rapid provisioning
  • Multiple clusters for multiple tenancy with DC/OS
  • High availability runtime configuration and software updates
  • Storage volumes for enhanced data durability, known as Mesos Dynamic Reservations and Persistent Volumes
  • Integration with syslog-compatible logging services for diagnostics and troubleshooting
  • Integration with statsd-compatible metrics services for capacity and performance monitoring

Related Services

Getting Started

To start a basic test cluster with three brokers, run the following command on the DC/OS CLI.…Read More

Configuration

The default DC/OS Confluent Kafka installation provides reasonable defaults for trying out the service, but may not be sufficient for production use. You may require a different configuration depending on the context of the deployment.…Read More

Operations

The DC/OS Confluent Kafka service provides a robust API (accessible by HTTP or DC/OS CLI) for managing, repairing, and monitoring the service. Here, only the CLI version is presented for conciseness, but see the API Reference for HTTP instructions.…Read More

Updates

Enterprise DC/OS 1.10 introduced a convenient command line option that allows for easier updates to a service’s configuration and version, as well as allowing users to inspect the status of an update, to pause and resume updates, and to restart or complete steps if necessary.…Read More

Security

The DC/OS Confluent Kafka service supports Kafka’s native transport encryption, authentication, and authorization mechanisms. The service provides automation and orchestration to simplify the usage of these important features.…Read More

Uninstall

If you are using DC/OS 1.10 or later and the installed service has a version later than 2.0.0-x, then uninstalling the service is simple.…Read More

Troubleshooting

After a configuration change, the service may enter an unhealthy state. This commonly occurs when an invalid configuration change was made by the user. Certain configuration values may not be changed, or may not be decreased. To verify whether this is the case, check the service’s deploy plan for any errors.…Read More

Advanced

This section describes some advanced features of the DC/OS Confluent Kafka service.…Read More

API Reference

The DC/OS Confluent Kafka Service implements a REST API that may be accessed from outside the cluster. The parameter referenced below indicates the base URL of the DC/OS cluster on which the DC/OS Confluent Kafka Service is deployed.…Read More

Limitations

Out-of-band configuration modifications are not supported. The service’s core responsibility is to deploy and maintain the service with a specified configuration. In order to do this, the service assumes that it has ownership of task configuration. If an end-user makes modifications to individual tasks through out-of-band configuration operations, the service will override those modifications at a later time. For example:…Read More

Support Policy

DC/OS and certified package version support policy are described in detail here.…Read More

Release Notes

Discover the new features, updates, and known limitations in this release of the Confluent Kafka Service…Read More