GitHub user bamaer added a comment to the discussion: HOP Sizing
in addition to what Hans mentioned, make sure your pipelines are optimized before throwing more (virtual) hardware at them: - Measure what your actual resource usage is (rows per second, memory and CPU usages, ...) and tweak your scaling accordingly. You may be able to scale down if you only use 30% of the available RAM, Xmx or CPU. - Identify bottlenecks in your pipelines, fix the worst offenders and readjust your scaling needs accordingly. E.g. adding memory won't help if your pipeline is slow because of missing database indexes - Tweak for the resources you have available. A larger box that runs your big pipelines a lot faster in a short-lived container or pod may be cheaper than a smaller container or pod that needs more time to run GitHub link: https://github.com/apache/hop/discussions/4586#discussioncomment-11304536 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
