Applied, sorry for being late On Mon, Mar 23, 2009 at 5:08 PM, J Wynia <[email protected]> wrote:
> The patch is attached. > > The deployment pattern is really my biggest question with RhinoETL. In > looking at the code and our situation, our whiteboard solution > (assuming it makes sense when it comes down off of the whiteboard) was > to have a central ETLPackageRunner that either *is* the Rhino.Etl.Cmd > directly or a derivative. That single console app would be scheduled > to execute named packages on the appropriate scheduled times. Those > named packages are just aliases to ETLPackage assembly paths and > provide the actual arguments to load the appropriate IProcess > implementation. > > It's pretty likely that our implementation would include 50-100 > packages that themselves would share half that many underlying common > IOperations. So, we were expecting to have to set up a set of nested > directories to organize them all and properly version things. One of > the reasons that we're looking to use RhinoETL is that the previous > regimes implemented DTS/SSIS without any real strategy for re-use or > versioning. > > Beyond that, our IOperations need to use the exact same business rules > that the web sites, web services, admin tools, etc. use and getting > SSIS to cooperate with the assemblies that contain that information > wasn't exactly a straightforward process. Things are definitely > looking up as RhinoETL turns the question from "Can we organize this > mess of scheduled ETL activity?" into "HOW should we organize this > mess?" > > On Sat, Mar 21, 2009 at 6:22 AM, Ayende Rahien <[email protected]> wrote: > > It looks like a bug. > > The reason that it works is related to how I am usually deploying this. > > I usually copy the exe to the same directory with the dlls. > > It sounds like a real problem, can you provide the patch as an attachment > > instead? > > > > On Fri, Mar 20, 2009 at 12:14 PM, J Wynia <[email protected]> wrote: > >> > >> After reading all of the articles I could find on RhinoETL, it was > >> clear that it was far closer to what I've been looking for on my > >> current project than any of the alternatives, so I sat down a few days > >> ago to do a prototype and work through what our pattern/workflow was > >> going to be. > >> > >> Not wanting to bite off too many pieces at once, I started with doing > >> all of the bits in C#. I can see how great using the Boo DSL is going > >> to be, but want to make sure I understand the actual ETL domain it is > >> the DSL for before using it. I'm also working through the DSL's In Boo > >> book to support really grokking what's going on before I make a mess. > >> This is especially true since many of our operations are going to have > >> to make use of classes from elsewhere in our project to determin > >> > >> Anyway, I ran into some problems that confused me. I spent some time > >> digging through the Rhino.Etl source and patched it in a way that made > >> it work, but I that change makes me unsure of how it was *supposed* to > >> work. Since I do NOT want to be one of those guys claiming that > >> "SELECT is broken", here's the details of what's going on my end and > >> my theories. > >> > >> ==What I did== > >> I created a C# class library project/solution. I took 2 of the > >> operations from the unit tests (the one to load from a file and the > >> one to write to a file) and added them to my project, tweaking so that > >> the reader loaded an "inputfile.txt" and the writer writes to an > >> "outputfile.txt". I set up the implementer of IProcess to Register() > >> both of those classes and built the whole thing with no problems. > >> > >> I took the output of building Rhino.Etl.Cmd in Debug mode and my > >> "package" assembly and put them together in a working directory. > >> > >> From a Powershell prompt, I ran > >> > >> .\Rhino.Etl.Cmd.exe -file:MyAssembly.dll -p:MyIProcessImplementer > >> > >> It complained about not being able to load the assembly or one of it's > >> dependencies via reflection, so I created a console app in my ETL > >> package solution and added my class library project as a reference. In > >> the console app's Main() method, I added a single line to do an > >> Assembly.Load() on my assembly. That worked with no problems. However, > >> as I'd used the assembly's name instead of the assembly's *file* name > >> in that console app and the Rhino.Etl.Cmd.exe version included the > >> .dll extension (thus making it about the filename), I went digging > >> into the Rhino.Etl source to see how the assembly was actually loaded. > >> > >> ==What confuses me== > >> The -file switch on Rhino.Etl.Cmd.exe looks for ".exe" or ".dll" on > >> the end of the argument and anything else is seen as a Boo file, > >> clearly the filename rather than the assembly name is what's supposed > >> to come in on the commandline. However, the GetFromAssembly() method > >> does an Assembly.Load() [which takes in the assembly name] rather than > >> an Assembly.LoadFromFile() [which takes in a filename]. That means > >> that anything that I pass in that GetFromAssembly() will load won't > >> get past the initial setup and anything that gets past the initial > >> setup fails on the Assembly.Load(). > >> > >> So, in an effort to see if that actually WAS the problem or not, I > >> tweaked GetFromAssembly() just a bit by adding a FileInfo lookup on > >> the input file to get the full filename and then swapped > >> Assembly.LoadFromFile() for the existing Assembly. When I ran that, as > >> long as I passed in something that could resolve to the filename of my > >> DLL, it would load and move on. The individual operations themselves > >> are still not 100% working the way I expect, but that to me is more a > >> matter of straightforward work and research into the various operation > >> types. > >> > >> Here's the diff/patch of what I did to get it to run: > >> > >> Index: RhinoEtlSetup.cs > >> =================================================================== > >> --- RhinoEtlSetup.cs (revision 2086) > >> +++ RhinoEtlSetup.cs (working copy) > >> @@ -66,7 +66,9 @@ > >> > >> private static Type GetFromAssembly(RhinoEtlCommandLineOptions > >> options) > >> { > >> - Assembly asm = Assembly.Load(options.File); > >> + FileInfo _assemblyInfo = new FileInfo(options.File); > >> + Assembly asm = Assembly.LoadFile(_assemblyInfo.FullName); > >> + //Assembly asm = Assembly.Load(options.File); > >> foreach (Type type in asm.GetTypes()) > >> { > >> if(typeof(EtlProcess).IsAssignableFrom(type) && > >> type.Name.Equals(options.Process, > >> StringComparison.InvariantCultureIgnoreCase)) > >> @@ -88,9 +90,11 @@ > >> { > >> try > >> { > >> + FileInfo _assemblyInfo = new FileInfo(options.File); > >> //we have to run the code in another appdomain, > >> because we want to > >> //setup our own app.config for it > >> AppDomainSetup appDomainSetup = new AppDomainSetup(); > >> + appDomainSetup.ApplicationBase = > >> _assemblyInfo.DirectoryName; > >> appDomainSetup.ConfigurationFile = options.File + > >> ".config"; > >> AppDomain appDomain = > >> AppDomain.CreateDomain("etl.domain", null, appDomainSetup); > >> appDomain.Load(processType.Assembly.GetName()); > >> > >> > >> ==So, What Am I Asking?== > >> What I'm wondering is if this is, indeed a bug or if I'm just doing it > >> wrong. Given what I'm seeing, I'm trying to understand how the > >> GetFromAssembly() method has worked in the past. The way it is right > >> now, I don't know how you'd get an assembly through and in to the > >> subsequent execution bits. > >> > >> ==Where To Go From Here?== > >> Because I've got a team of developers on this project, once I get this > >> figured out and can articulate how we integrate the use of RhinoETL > >> into our overall architecture, I'm going to have to put together an > >> article/presentation/screencast for their benefit and I'd like to put > >> it out there for public consumption too. However, that makes me want > >> to be sure I understand it even more. Otherwise, I'll just be sending > >> people out there to do it wrong as well. > >> > >> The bulk of the examples, both on Ayende's site and elsewhere, focus > >> on the operation classes mostly and the IProcess classes a little as > >> well. However, I haven't been able to find a higher level view of how > >> to put together/organize the ETL packages made up of those other bits > >> and how to run them, etc. > >> > >> Please note that I am NOT dogging the documentation or making any sort > >> of support "demand". The fact that the open source tool exists at all > >> is a great thing and I'm willing to do some legwork to get it running. > >> I just don't want to end up lost in the desert. > >> > >> -- > >> J Wynia > >> Software Consultant, Writer and Geek > >> Minneapolis, MN > >> [email protected] > >> "The glass isn't half full or half empty. It's just too big" > >> "Jack of all trades, master of none, though ofttimes better than master > of > >> one." > >> http://wynia.org > >> > >> > > > > > > > > > > > > > -- > J Wynia > Software Consultant, Writer and Geek > Minneapolis, MN > [email protected] > "The glass isn't half full or half empty. It's just too big" > "Jack of all trades, master of none, though ofttimes better than master of > one." > http://wynia.org > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Rhino Tools Dev" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/rhino-tools-dev?hl=en -~----------~----~----~----~------~----~------~--~---
