Dear developers and users, After looking at this a bit more there is also a more fundamental problem about the approach we are taking; the need to create custom build directories in an easyblock file (where I do the iteration). I could do this in the build_step by modifying the self.builddir, but the path_step would not see this, etc.
It then seems to me that if we were to iterate, this should happen outside the easyblock. At least given the current configuration. As I do not see a way to specify such an iteration from the easyconfig (even by enabling extra parameters for the custom easyblock file), this iteration seem to need to happen outside EB. I am now thinking of simply adding a wrapper script that steps trough all the different versions we need by modifying the easyconfig file and passing custom arguments to an easyblock for each step and calling eb in each of these steps, collecting modules as I go. Is this the way to go or is there something I am missing? Thanks again for any comments or suggestions. Cheers, Espen On Mon, 2019-05-06 at 13:58 +0200, Espen wrote: > Dear developers and users, > > We are have a core software that need to be compiled with the > following > perturbations: > > - additional source files > - changes in the existing files > - additional libraries > > The first two can be done together or isolated. The latter is rather > easy to do; using dependencies in the top easyconfig and writing a > custom include file. > > However, we have not yet determined the best strategy to do the > additional files and changes in source files. > > In total we have about 50 raw perturbations of the same software at a > single toolchain that we need to maintain and deploy. We have defined > a > build matrix in a json file that we read in into our custom easyblock > file. The number of versions that needs to maintain will probably be > even larger as we allow for different versions also of each > perturbation. However, then starts the questions. Here is the first > one: > > What is the best strategy to iterate over these perturbations? Right > now we have overloaded the build_step. That works as long as we do > not > need to apply patches (say that one of these perturbations is a > simple > patch). Then it would make sense to overload patch_step, but I am not > sure how to coordinate that with the iteration we have created in > build_step. We have tried to look at the documentation to find > examples > of this, but have not yet managed to locate it. If it exists, can > anyone please point us in the right direction? > > Also, since the easyconfig file does not seem to allow generic python > code we would like to load parameters from a json and only make the > easyconfig start the easyblock file. Except for the simple > dependencies, all the iterative work will be done in the easyblock. > Is > this a sensible approach or should we halt immediately? > > Thanks in advance for any help. Also, keep up the good work! > > Best regards, > Espen
