Here is the video of my talk on Modern Data Architecture from Java Day Kiev 2015
The slides are available here: Modern Data Architecture – JD Kiev v05
Here is the video of my talk on Modern Data Architecture from Java Day Kiev 2015
The slides are available here: Modern Data Architecture – JD Kiev v05
Pingback: Modern Data Architecture Talk | Filling the gaps in Big Data
Good talk, thanks.
1. But the architecture style of asynchronously writing and reading to a “queue” using APIs and giving the illusion to Front end users that things are happening synchronously isn’t new, right ? its called CQRS proposed by George young and it exists for a long time.
2. Also when you meant more than one consumer can read from the “queues” like Backend to load OLTP, EDW , Data lake and Streaming pipeline you meant a “Pub/Sub” semantic and NOT a “Worker-Queue” semantic right ?. The underlying implementation could be anything like RabbitMQ or Kafka or AWS Kinesis but the idea is to use Pub/Sub. So a single topic can be consumed by multiple consumers mentioned by you at their own speed and do their own processing on top of that.
I have implemented a very similar architecture and feel validated as smart people like you have a similar proposal. Although Kafka made more sense to our use cases.
Would love your thoughts on the above 2 points.