Hi Apache Beam Dev Team, First of all, thank you for developing such an amazing project and making it open-source.
I have a use case where I encountered some limitations in using Apache Beam to solve my problem. I am working with workloads that are tied to specific sequences. My goal is to process workloads in order, where workloads with the same sequence should run concurrently, but workloads in the next sequence must wait until the previous sequence has fully completed. With Apache Beam, I found it difficult to achieve this balance. I was only able to process workloads either sequentially by enforcing strict ordering or run all workloads in parallel. Neither approach fully met my needs. I considered a workaround where one pipeline calls another, but I believe this would be inefficient and resource-intensive, leading to poor design and scalability. Is there a way to extend or customize Apache Beam to handle this use case more effectively? I would greatly appreciate any guidance or suggestions on this. I would also be happy to set up a call if further clarification is needed regarding my use case. Thanks & Regards Pramod
