Apache Kafka

A distributed, high-performance, low-latency event streaming platform. Ideal for data pipelines, messaging, and real-time processing.

Version:

4.2.0

back to Marketplace

VM Requisites

Resource Minimum
CPU 2 vCPU
RAM 4 GB
Disk 40 GB
Operating System Ubuntu 22.04 / 24.04

Access Port

Port Protocol Usage
30909 TCP Kafka broker (producers and consumers)

How To Access

Kafka does not have a web interface. Interaction is directly between producers and consumers using the broker's port.

Connection string

<IP_DE_LA_VM>:30909

Verify connectivity with kafka-topics

# List existing topics
kafka-topics.sh --bootstrap-server <VM_IP>:30909 --list

# Create a topic
kafka-topics.sh --bootstrap-server <VM_IP>:30909 \ 
--create --topic my-topic --partitions 3 --replication-factor 1

# Describe a topic
kafka-topics.sh --bootstrap-server <VM_IP>:30909 \ 
--describe --topic my-topic

Example with producer and consumer

# Send messages (producer)
kafka-console-producer.sh --bootstrap-server <VM_IP>:30909 --topic my-topic

# Receive messages (consumer)
kafka-console-consumer.sh --bootstrap-server <VM_IP>:30909

-topic my-topic --from-beginning

Connection string for applications

# Java / Spring Boot
bootstrap-servers=<IP_DE_LA_VM>:30909

# Python (kafka-python)
bootstrap_servers=['<IP_DE_LA_VM>:30909']

# Node.js (kafkajs)
brokers: ['<IP_DE_LA_VM>:30909']

Verify that Kafka is active

# View the Kafka pod
`kubectl get pods -A | grep kafka`

# View the installation log
`tail -f /var/log/cuemby/bootstrap.log`

# Verify that the port is listening
`ss -tlnp | grep 30909`

Expected output:

NAME                     READY   STATUS    RESTARTS
kafka-xxxxxxxxx-xxxxx    1/1     Running   0        ← Running ✓

Configuration Parameters

Parameter Default Description
KAFKA_CLUSTER_ID auto-generated Unique cluster identifier in KRaft mode (without ZooKeeper). Cuemby generates it automatically.
KAFKA_DATA_SIZE 20Gi Persistent volume size for event logs.
KAFKA_HEAP_OPTS -Xmx1g -Xms1g JVM heap options. Adjust according to available RAM (recommended: 25–50% of total RAM).

ℹ️ This installation uses KRaft mode (without ZooKeeper). It does not require a separate ZooKeeper ensemble.

First Steps (quick start)

# 1. Create a topic with 3 partitions
kafka-topics.sh --bootstrap-server <VM_IP>:30909
--create --topic events --partitions 3 --replication-factor 1

# 2. Verify that the topic was created
kafka-topics.sh --bootstrap-server <VM_IP>:30909 --list

# 3. Produce a test message
echo "hello world" | kafka-console-producer.sh
--bootstrap-server <VM_IP>:30909 --topic events

# 4. Consume the message
kafka-console-consumer.sh --bootstrap-server <VM_IP>:30909
--topic events --from-beginning --max-messages 1

💡 Tip: Adjust KAFKA_HEAP_OPTS if you see OutOfMemoryError errors. With 8 GB of RAM, a value like -Xmx3g or -Xms3g provides enough headroom for light production workloads.

Quick Troubleshooting

Problem Probable cause Solution
Connection refused on port 30909 Kafka still beginning Wait ~3–5 min and check tail -f /var/log/cuemby/bootstrap.log.
LEADER_NOT_AVAILABLE when creating topic Newly started broker Wait a few seconds and try again. It's temporary.
OutOfMemoryError in logs Insufficient Heap JVM Increase KAFKA_HEAP_OPTS to -Xmx3g -Xms3g and restart the pod.
Pod in CrashLoopBackOff Insufficient RAM or disk space Verify that the VM has at least 8 GB of RAM and 50 GB of disk space.

Cuemby Cloud

IaaS

About Cuemby Cloud

Cuemby Cloud is enterprise-grade cloud infrastructure managed from a single console, built for stronger security, predictable operations, and easy scaling across regions.

Datacenters regions available in Colombia, Ecuador, and Chile

Backed by Tier III / Tier IV data center locations

Zero Network Ingress and Egress Fees

24/7 local expert support

Ready to Modernize Your Enterprise?

Future challenges demand future-ready solutions.
Let’s get started!

© Copyright | Cuemby® 2025