I wanted to put a perspective out there on environments. What I think we would want to end up with are small collections of environments.
For example, at Neo Corp, perhaps they have a single collection for their company consisting of 1) Production 2) Development 3) Test For a consultant type user, they work with multiple corporate entities and have a collection each. Nord Ltd. may have 1) Production 2) Test Taho US Inc. 1) Prod 2) Dev 3) Test etc. What each user would want to do is have the ability to tell an ETL project which collection and specific environment profile to use. It would probably be good to separate out the environment support and config files to hop/config/environment/(collection name)/(supporting flat files or configs). There could be a special folder hop/config/environment/relations.json which could keep track of client project / collection ties. This means you could ship projects with environment collections separately, but the Hop UI could parse and manage some of these relationships separate of the environment and projects themselves. The above would allow Hop to scan for these things. One other requirement is I would want some way to scan and report on a project to understand where all the environmental tentacles are touching. If I take any project right now, variables could be used or consumed anywhere. Only design best practices lead one to have a tendency of defining and using variables in particular places. It would be cool if Matt's fun reporting project might highlight those variables in a color, especially for variables with the same name. (patterned usage). Hope this contributes positively to the direction of decision-making. Another thing that would make Hop infinitely more useful is the return of syntax highlighting in steps. It is hard to overstate how difficult it is to write any kind of script in Hop at the moment. I have had to resort to outside tools for preparing and formatting scripts and queries. A prior ETL tool spoiled me. On Tue, Nov 2, 2021 at 3:19 AM Hans Van Akelyen <[email protected]> wrote: > Hi Hop enthusiasts, developers and philosophers > > I have had an interesting discussion with Sergio on Jira ticket HOP-3471 > [1] but we would like to get a bit of a broader consensus. > > So the main question is: do we see an environment as something linked to a > single project or are they objects that can be reused for several projects? > Both approaches have pro's and con's but we need to agree on this as it > would change user experience and maybe how they are stored. > > Currently an environment is a semi-standalone object, it is not linked to a > project in the hop-project.json but it is linked to a project in the > hop-config.json. In the GUI all environments are shown, not only the > environments linked to a project. When using hop-run I *think* you can > point to an environment that is not linked to the project in the > hop-config.json. > > My personal preference would be to unlink them completely and have > environments that could be used by multiple projects. This gives greater > flexibility when you would want to divide your projects on a lower level. > con would be that you would always see all available environments defined > in your hop-config.json. > > I would love for some opinions on the matter so it can be set in stone once > and for all. And we can update our docs to reflect this decision [2][3]. > > Cheers, > Hans > > > [1] https://issues.apache.org/jira/browse/HOP-3471 > [2] > > https://hop.apache.org/manual/latest/projects/projects-environments.html#top > [3] https://hop.apache.org/manual/latest/projects/index.html#_environments >
