Kafka

At Daugherty, we offer cutting-edge capabilities and work continuosuly with the newest technologies to meet the needs of our clients.

Apache Kafka is an open-source, stream-processing software platform used to build real-time data pipelines and streaming apps. In data warehousing, batch ETLs tend to get tangled and experience lags in data delivery. Apache Kafka provides a unified, high-throughput, low-latency platform for handling real-time data.

At Daugherty, we’re leveraging Kafka in a variety of ways. For a leading uniform rental and linen supply company, we used Kafka to scale out and stabilize a brittle legacy system, and to enable real-time analytics to drive millions in savings. For the same client, we created a mobile application to replace paper contract renewals. Kafka provided a single point of access for data so that customer classifications, responsible employees and price-negotiation flexibility could be driven by analytics.

For one of the largest global biotech corporations, we leveraged Kafka at the center of their enterprise data hub, incorporating Spark clients for consumers and producers — on the consumer side, to integrate with an analytics database, and on the producer side, to pump data downstream to dependent systems. This eliminated data silos and enabled a cloud-based architecture for the client.

Kafka propels organizations into the latest evolution of handling data:

Replayability

Kafka sidesteps transactional roadblocks by integrating sources of data into publish/subscribe streams. Processing operations can be started and restarted at any point in the history of the stream, and the events can be replayed in order.

Stability of the system

Kafka can be used as version control for your data, ensuring a single source of truth to integrate between various systems.

Speed of processing

Business facts and events can be collected in near-real-time, so business processes can react just as quickly. Stream processing can be used to minimize latency in data processing.

Scalability and fault-tolerance

Kafka can be used to move your organization toward data processing capabilities that are scalable and fault-tolerant. Should a system go down, another picks up where it left off.