Hi Tammo, This is absolutely what in my mind. I think we should re-contract another copy of OModel classes to confirm they can be easily serialize and deserialize by Jackson. I have updated my proposal "schedule" part http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/tigergui/41002
Now, it may be more clear about this part. Please have a review of it. Yeah, once you finish pushing the experiment code into Github, please tell me. I will take a look into it and find useful parts to discuss with you :-) 2012/4/5 Tammo van Lessen <[email protected]> > Hi Tiger, > > serializing worked for me after defining some mixed-in classes for > Jackson with some annotations. I have it on a different laptop > currently, I try to fetch it and share it with you. Perhaps I just > push my test code to github. So at least it serialized the whole tree, > but I'm not entirely sure if really all elements and attributes were > included. > > Okay, yes. I think we're on the same page. The important step is: a > new version of the OModel (e.g. in a different package), that has a > Map for unknown fields and that has private default constructors so > that Jackson can easily serialize and deserialize them. This OModel > needs then to be included into compiler and to be used by the runtime. > The next step is to provide means to read an old cbp file and migrate > its contents to the new OModel. And we both believe that Jackson can > ease this job as well, since we could serialize the old OModel to > JSON, tweak the output a "little" and the deserialize it to the new > OModel. I tried this in my experiments as well and simply used > String.replace to change the package name. I'll write you a second > mail when I pushed that stuff to github. > > HTH, > Tammo > > On Wed, Apr 4, 2012 at 17:54, Tiger Gui <[email protected]> wrote: > > Hi Tammo, > > > > I mean, if we use Jackson to serialize OProcess directly, such as > following > > code: > > > > JsonGenerator jsonGenerator = null; > > ObjectMapper objectMapper = null; > > try { > > objectMapper = new ObjectMapper(); > > jsonGenerator = > > objectMapper.getJsonFactory().createJsonGenerator(System.out, > > JsonEncoding.UTF8); > > jsonGenerator.writeObject(OProcess instance here); > > } catch (IOException e) { > > e.printStackTrace(); > > } > > > > We will get some unsupport exception, right? Did you caught them in your > > experiment ? > > > > So, if we want to use Jackson to serialize the whole OProcess object > > successfully, we should extend Jackson and > > create another class (call it OProcessB) which store all the information > of > > the original OProcess object by key-value mode. > > > > This OProcessB may be similar with OProcess, but they are two different > > classes. I mean the transfrom job from OProcess -> OProcessB is the > > "refactor" job. > > > > Yeah, i miss deserialization part in the proposal, i will add this part. > But > > when i talked about the "Refactor OModel > > classes Oxxxx to confirm jackson can serialize them", i really mean : > > > > We should do some other Jackson extension job (even little OModel class > > refactor job) to confirm we can serialize and de-serialize these OModel > > classes to/from Json format file. > > > > Your suggestions? Thank you > > > > > > 2012/4/4 Tammo van Lessen <[email protected]> > >> > >> Hi Tiger, > >> > >> thanks, I'll have a look. One thing I found is "Refactor OModel > >> classes Oxxxx to confirm jackson can serialize them.". Serialization > >> seems to be much easier than deserialization. In my experiments, the > >> OModel could be serialized without significant changes. Also, > >> refactoring the original OModel is difficult because we need to keep > >> the binary compatibility to the cbp file. So we can only refactor the > >> duplicated, new OModel and make sure that this serializes and > >> deserializes correctly. I guess this is what you meant, but it could > >> be more clear in the proposal. > >> > >> Thanks, > >> Tammo > >> > >> On Wed, Apr 4, 2012 at 10:35, Tiger Gui <[email protected]> wrote: > >> > Hi Tammo, > >> > > >> > I have added project schedule and about me sections for my proposal > and > >> > submit it GSoC melange system, you can find the proposal details > >> > here[1], > >> > and we can keep improving it before April 6th, only 2 days left. If > you > >> > have > >> > any suggestions about it, please tell me, hope we can get an > acceptable > >> > proposal :-) > >> > > >> > Thank you > >> > > >> > > >> > [1] > http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/tigergui/41002 > >> > > >> > 2012/4/4 Tammo van Lessen <[email protected]> > >> >> > >> >> Hi Tiger, > >> >> > >> >> thanks for your proposal. I'll need to read it more thoroughly, I'll > >> >> come back to you tomorrow. But what is missing is a rough schedule > >> >> with defined milestones. This is something other applications provide > >> >> and are helpful and necessary for the ranking and also for measuring > >> >> your progress. It would be great if you could add a project plan. > >> >> > >> >> Thanks, > >> >> Tammo > >> >> > >> >> On Fri, Mar 30, 2012 at 10:11, Tiger Gui <[email protected]> > >> >> wrote: > >> >> > Hi all, > >> >> > > >> >> > I have drawed up draft version of GSoC 2012 project "Refactor > OModel > >> >> > and > >> >> > provide intelligent migration feature for Apache ODE" project > >> >> > proposal, > >> >> > you > >> >> > guys can find the details here[1]. > >> >> > > >> >> > This is only initial idea of Tammo and me, if you guys have any > good > >> >> > advises > >> >> > about this project, let's discuss here, and i will update the > >> >> > proposal > >> >> > according to the discussion result. As GSoC's student application > >> >> > period > >> >> > will close on April 6, we have still a week to improve the > proposal. > >> >> > Thank > >> >> > you all :-) > >> >> > > >> >> > [1] > >> >> > > >> >> > > >> >> > > http://code.google.com/p/tigergui-proposal-temp/wiki/Dynamic_OModel?ts=1333094366&updated=Dynamic_OModel > >> >> > > >> >> > > >> >> > > >> >> > ===============Refactor OModel and provide intelligent migration > >> >> > feature > >> >> > for > >> >> > Apache ODE==================== > >> >> > > >> >> > Introduction > >> >> > > >> >> > Apache ODE (Orchestration Director Engine) executes business > >> >> > processes > >> >> > written following the WS-BPEL standard. It talks to web services, > >> >> > sending > >> >> > and receiving messages, handling data manipulation and error > recovery > >> >> > as > >> >> > described by your process definition. It supports both long and > short > >> >> > living > >> >> > process executions to orchestrate all the services that are part of > >> >> > your > >> >> > application. > >> >> > > >> >> > Current ODE's OModel has a static structure and is serialized using > >> >> > Java's > >> >> > built-in mechanisms, which are not very tolerant against class > >> >> > changes > >> >> > and > >> >> > could even struggle with JDK upgrades. We want to create a GSoC > >> >> > project > >> >> > to > >> >> > refactor OModel to provide intelligent migration feature for ODE. > >> >> > > >> >> > Details Description > >> >> > > >> >> > Current ODE OModel Design > >> >> > > >> >> > Let's begin with Apache ODE's architectural: > >> >> > > >> >> > When a BPEL process is deployed to ODE, it first runs through the > >> >> > compiler, > >> >> > which parses the BPEL file into a BPEL Object Model (BOM), which > >> >> > basically > >> >> > aims at abstracting from different BPEL versions (1.1, 2.0 draft, > 2.0 > >> >> > final). > >> >> > > >> >> > ODE Compiler compile and create OModel instances(such as > >> >> > OProcess,OAssign,OInvoke etc) from .bpel, .wsdl and deploy files. > >> >> > Then > >> >> > it > >> >> > can use Java Object Serializable technology to serialize OModel > >> >> > objects > >> >> > into > >> >> > a .cbp file (Compiled Process Definitions part in the diagram, and > we > >> >> > can > >> >> > see OBase implements interface java.io.Serializable, and all the > >> >> > OModel > >> >> > classes are the sub-class of OBase). > >> >> > > >> >> > ODE runtime module can load a compiled BPEL model(the .cbp file in > >> >> > local > >> >> > disk) by de-serialize the .cbp file back into OModel classes. > >> >> > > >> >> > When a process instance is executed, the navigator operates on the > >> >> > OModel, > >> >> > i.e. it is loaded from disk into memory during execution and is > >> >> > unloaded > >> >> > afterwards, depending on hydration/dehydration policies to reduce > the > >> >> > memory > >> >> > footprint. The execution state of the OModel keeps in-memory > >> >> > references > >> >> > to > >> >> > the OModel, but when the state is persisted, it does not store a > copy > >> >> > of > >> >> > the > >> >> > OModel but only the IDs mentioned above. When the state is loaded, > >> >> > the > >> >> > IDs > >> >> > are looked up and replaced by proper references. > >> >> > > >> >> > Where Is The Problem > >> >> > > >> >> > Once you read OModel's serializer and de-serializer methods in > class > >> >> > org.apache.ode.bpel.o.Serializer: > >> >> > > >> >> > public void writeOProcess(OProcess process, OutputStream os) throws > >> >> > IOException { > >> >> > > >> >> > DataOutputStream out = new DataOutputStream(os); > >> >> > > >> >> > out.write(MAGIC_NUMBER); > >> >> > > >> >> > out.writeShort(format); > >> >> > > >> >> > out.writeLong(compileTime); > >> >> > > >> >> > out.writeUTF(process.guid); > >> >> > > >> >> > out.writeUTF(process.targetNamespace); > >> >> > > >> >> > out.writeUTF(process.processName); > >> >> > > >> >> > out.flush(); > >> >> > > >> >> > ObjectOutputStream oos = new CustomObjectOutputStream(os); > >> >> > > >> >> > oos.writeObject(process); > >> >> > > >> >> > oos.flush(); > >> >> > } > >> >> > > >> >> > public OProcess readOProcess() throws IOException, > >> >> > ClassNotFoundException { > >> >> > > >> >> > > >> >> > ObjectInputStream ois = new > >> >> > CustomObjectInputStream(_inputStream); > >> >> > > >> >> > OProcess oprocess; > >> >> > try { > >> >> > > >> >> > oprocess = (OProcess) ois.readObject(); > >> >> > > >> >> > } catch (ClassNotFoundException e) { > >> >> > > >> >> > throw new IOException("DataStream Error"); > >> >> > > >> >> > } > >> >> > return oprocess; > >> >> > > >> >> > } > >> >> > > >> >> > You will find the problem with the current OModel is that it has a > >> >> > static > >> >> > structure and is serialized using Java's built-in mechanisms, which > >> >> > are > >> >> > not > >> >> > very tolerant against class changes and could even struggle with > JDK > >> >> > upgrades. > >> >> > > >> >> > For example, there are these attributes in current OProcess: > >> >> > > >> >> > public final String version; > >> >> > > >> >> > public OConstants constants; > >> >> > > >> >> > public String uuid; > >> >> > > >> >> > public String targetNamespace; > >> >> > > >> >> > public String processName; > >> >> > > >> >> > public OScope procesScope; > >> >> > > >> >> > After ODE Compiler serialize it into .cbp file, the .cbp file will > >> >> > include > >> >> > these 6 attributes for OProcess instance. One day, we add another > >> >> > attribute > >> >> > for OProcess class, for example: > >> >> > > >> >> > pubic String newAttribute; > >> >> > > >> >> > If we try to de-serialize the old .cbp file into the new OProcess > >> >> > class, > >> >> > we > >> >> > will get a error. > >> >> > > >> >> > This is a problem because BPEL processes and their instances are > >> >> > potentially > >> >> > long-running,they usually run over years. Thus, a serialized OModel > >> >> > should > >> >> > survive ODE and JDK upgrades. Our problem is that even simple > changes > >> >> > (that > >> >> > are needed to implement new features or even to fix some bugs) > break > >> >> > backward compatibility in the OModel. We need a good solution to > >> >> > address > >> >> > that problem. > >> >> > > >> >> > What Shall We Do To Solve This Problem > >> >> > > >> >> > This project want to solve this problem, we will change .cbp file's > >> >> > format > >> >> > and use a brand new serialize solution to keep compatibility. After > >> >> > investigate possible approaches, we would like to use JSON to > >> >> > represent > >> >> > OModel(It means new .cbp file will be JSON format). About the above > >> >> > example, > >> >> > the old JSON format .cbp file can be de-serialized into the new > >> >> > OProcess > >> >> > class with a special migrate condition description string. > >> >> > > >> >> > Json Format > >> >> > > >> >> > Jackson is actually a JSON processor (JSON parser + JSON generator) > >> >> > written > >> >> > in Java. Beyond basic JSON reading/writing (parsing, generating), > it > >> >> > also > >> >> > offers full node-based Tree Model, as well as full OJM (Object/Json > >> >> > Mapper) > >> >> > data binding functionality. More important, it has good > performance, > >> >> > even > >> >> > faster than Java's built-in means. > >> >> > > >> >> > So we decided to use Json format file store OModel's serialize > >> >> > result, > >> >> > and > >> >> > Jackson 2.0 is the potential solution to reading/writing OModel > >> >> > instance > >> >> > from/to Json format .cbp file. > >> >> > > >> >> > We can use Jackson to store attributes of Current OProcess class to > >> >> > Json > >> >> > format file like this: > >> >> > > >> >> > Map<String,Object> userData1 = new > HashMap<String,Object>(); > >> >> > > >> >> > userData1.put("owner", process.getOwner()); > >> >> > > >> >> > userData1.put("name", process.getName()); > >> >> > > >> >> > userData1.put("targetNameSpace", process.targetNamespace); > >> >> > > >> >> > userData1.put("processName", process.processName); > >> >> > > >> >> > userData1.put("children", process.getChildren()); > >> >> > > >> >> > > >> >> > ObjectMapper mapper = new ObjectMapper(); > >> >> > > >> >> > StringWriter sw = new StringWriter(); > >> >> > > >> >> > org.codehaus.jackson.JsonGenerator gen; > >> >> > > >> >> > try { > >> >> > gen = new JsonFactory().createJsonGenerator(sw); > >> >> > > >> >> > mapper.writeValue(gen, userData1); > >> >> > > >> >> > gen.close(); > >> >> > String json = sw.toString(); > >> >> > > >> >> > System.out.println(json); > >> >> > > >> >> > } catch (IOException e) { > >> >> > > >> >> > e.printStackTrace(); > >> >> > } > >> >> > > >> >> > The target json file will be: > >> >> > > >> >> > > >> >> > > >> >> > {"targetNameSpace":"http://ode/bpel/unit-test > ","name":"HelloWorld2","processName":"HelloWorld2"} > >> >> > > >> >> > But not all the OModel instance can be serialized only using > Jacksons > >> >> > mixin > >> >> > feature, we have to extend Jackson and would probably need to write > >> >> > custom > >> >> > (de)serializers for WSDL4J etc. > >> >> > > >> >> > OModel Migrate Mechanism > >> >> > > >> >> > Here we use a simple example proposed by my potential mentor Tammo > >> >> > van > >> >> > Lessen to describe the migrate mechanism. > >> >> > > >> >> > Simple Example > >> >> > > >> >> > OProcess stores currently targetNamespace and processName as > Strings. > >> >> > Now > >> >> > imagine we change OProcess so that it stores a QName instead. When > >> >> > loading > >> >> > an old OModel, the QName field would be null and there would be the > >> >> > two > >> >> > String field in the "unknown fields" map. > >> >> > > >> >> > Migrate Solution > >> >> > > >> >> > Suppose the old version of OProcess is stored in following format: > >> >> > > >> >> > > >> >> > > >> >> > {"targetNameSpace":"http://ode/bpel/unit-test > ","name":"HelloWorld2","processName":"HelloWorld2"} > >> >> > > >> >> > and the corresponding new version of OProcess's json .cbp file will > >> >> > be: > >> >> > > >> >> > > >> >> > > >> >> > {"name":"HelloWorld2","namespace":"{ > http://ode/bpel/unit-test}HelloWorld2"} > >> >> > > >> >> > The migrate job is transform from one JSON to another JSON in a > >> >> > specific > >> >> > rule, We should consider how to define the migrate description > >> >> > language > >> >> > or > >> >> > just re-use some existing JSON migrate DSL. > >> >> > > >> >> > This migrate DSL may be the most important part of this project. In > >> >> > this > >> >> > simple demo, maybe looks like this: > >> >> > > >> >> > { > >> >> > "from":{ > >> >> > > >> >> > "name":"$value0", > >> >> > > >> >> > "targetNameSpace":"$value1", > >> >> > > >> >> > "processName":"$value2" > >> >> > > >> >> > }, > >> >> > "to":{ > >> >> > > >> >> > "name":"$value0", > >> >> > > >> >> > "namespace":"{$value1}$value2" > >> >> > > >> >> > } > >> >> > } > >> >> > > >> >> > Use $value to represent migrate attribute values. > >> >> > > >> >> > Domain-specific language(DSL) > >> >> > > >> >> > Once we update OModel, and want to migrate the old version to new > >> >> > version. > >> >> > If we have both the old version Json format .cbp file and the > >> >> > specificed > >> >> > migrate rule, we can get the new version Json format .cbp file, > then > >> >> > load as > >> >> > new OProcess object, finish the migrate job easily. > >> >> > > >> >> > We are still survey for exist tools like XSLT, working on JSON > Trees. > >> >> > If > >> >> > can > >> >> > not find such a tool, have both the old OModel and the new OModel, > we > >> >> > can > >> >> > write a tool to generate migrate DSL string. > >> >> > > >> >> > Of course, we must specific the migrate rule for different version > >> >> > change, > >> >> > such as v1.0->v1.1, v1.1->v1.2 and v1.0->v1.2. > >> >> > > >> >> > > >> >> > -- > >> >> > Best Regards From Tiger Gui > >> >> > -------------------------------------- > >> >> > Open source is some kind of life attitude > >> >> > >> >> > >> >> > >> >> -- > >> >> Tammo van Lessen - http://www.taval.de > >> > > >> > > >> > > >> > > >> > -- > >> > Best Regards From Tiger Gui > >> > -------------------------------------- > >> > Open source is some kind of life attitude > >> > >> > >> > >> -- > >> Tammo van Lessen - http://www.taval.de > > > > > > > > > > -- > > Best Regards From Tiger Gui > > -------------------------------------- > > Open source is some kind of life attitude > > > > -- > Tammo van Lessen - http://www.taval.de > -- Best Regards From Tiger Gui -------------------------------------- Open source is some kind of life attitude
