We have a batch process that runs every morning, processes a bunch of data gathered out of our central e-commerce database and publishes messages on a queue for another service to pick up for further processing. This batch process takes about 30 odd minutes to finish which is not at all bad since our time […]

This is the continuation of the previous article where I implemented the Pipes and Filters integration pattern but left out the continuous deployment aspect of it.

…the whole business process can be divided into smaller asynchronous sub-processes with each one feeding i.e. “piping” its output into the input of the subsequent process(es). At each stage some processing is done for e.g. data enriching, filtering etc and this processing generically is termed as a “filter”.