Key takeaways

• Learn about the power of events and unbounded data
• Find out why streaming is not just for real-time applications—it’s for everyone
• Understand where a streaming platform fits in an analytic architecture
• Discover how event-driven architectures can enable greater scalability and flexibility of systems both now and in the future

Data integration in architectures built on static, update-in-place datastores inevitably end up with pathologically high degrees of coupling and poor scalability. This has been the standard practice for decades, as we attempt to build data pipelines on top of databases that do a poor job modeling the fundamental objects that drive our businesses and systems: events. Events form a powerful primitive on which to build systems for developers and data engineers alike. Developers benefit from the asynchronous communication that events enable between services, and data engineers benefit from the integration capabilities. In this talk, we’ll discuss the concepts of events and their ability to unify architectures in a powerful way. We’ll see how stream processing makes sense in both a microservices and ETL environment, and why analytics, data integration and ETL fit naturally into a streaming world. The talk will include an example of the concepts in practice using Apache Kafka.