?How do you physically spit up the large files and track them in each thread?

?How is the master process receiving it's messages from the children processes?

FOR ABOVE TWO QUESTIONS, IN...

We can think of an ETL system where we expect around 5 million transaction coming in with around 25 columns.

I would request if any of the Technical Architects who worked...

Large volume means number of rows and data size of source(s) that need to be processed into the target system by applying good amount of complex validation/filtration/transformation logics. The...