Hi Tiger, thanks, I'll have a look. One thing I found is "Refactor OModel classes Oxxxx to confirm jackson can serialize them.". Serialization seems to be much easier than deserialization. In my experiments, the OModel could be serialized without significant changes. Also, refactoring the original OModel is difficult because we need to keep the binary compatibility to the cbp file. So we can only refactor the duplicated, new OModel and make sure that this serializes and deserializes correctly. I guess this is what you meant, but it could be more clear in the proposal.
Thanks, Tammo On Wed, Apr 4, 2012 at 10:35, Tiger Gui <[email protected]> wrote: > Hi Tammo, > > I have added project schedule and about me sections for my proposal and > submit it GSoC melange system, you can find the proposal details here[1], > and we can keep improving it before April 6th, only 2 days left. If you have > any suggestions about it, please tell me, hope we can get an acceptable > proposal :-) > > Thank you > > [1] http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/tigergui/41002 > > 2012/4/4 Tammo van Lessen <[email protected]> >> >> Hi Tiger, >> >> thanks for your proposal. I'll need to read it more thoroughly, I'll >> come back to you tomorrow. But what is missing is a rough schedule >> with defined milestones. This is something other applications provide >> and are helpful and necessary for the ranking and also for measuring >> your progress. It would be great if you could add a project plan. >> >> Thanks, >> Tammo >> >> On Fri, Mar 30, 2012 at 10:11, Tiger Gui <[email protected]> wrote: >> > Hi all, >> > >> > I have drawed up draft version of GSoC 2012 project "Refactor OModel and >> > provide intelligent migration feature for Apache ODE" project proposal, >> > you >> > guys can find the details here[1]. >> > >> > This is only initial idea of Tammo and me, if you guys have any good >> > advises >> > about this project, let's discuss here, and i will update the proposal >> > according to the discussion result. As GSoC's student application period >> > will close on April 6, we have still a week to improve the proposal. >> > Thank >> > you all :-) >> > >> > [1] >> > >> > http://code.google.com/p/tigergui-proposal-temp/wiki/Dynamic_OModel?ts=1333094366&updated=Dynamic_OModel >> > >> > >> > >> > ===============Refactor OModel and provide intelligent migration feature >> > for >> > Apache ODE==================== >> > >> > Introduction >> > >> > Apache ODE (Orchestration Director Engine) executes business processes >> > written following the WS-BPEL standard. It talks to web services, >> > sending >> > and receiving messages, handling data manipulation and error recovery as >> > described by your process definition. It supports both long and short >> > living >> > process executions to orchestrate all the services that are part of your >> > application. >> > >> > Current ODE's OModel has a static structure and is serialized using >> > Java's >> > built-in mechanisms, which are not very tolerant against class changes >> > and >> > could even struggle with JDK upgrades. We want to create a GSoC project >> > to >> > refactor OModel to provide intelligent migration feature for ODE. >> > >> > Details Description >> > >> > Current ODE OModel Design >> > >> > Let's begin with Apache ODE's architectural: >> > >> > When a BPEL process is deployed to ODE, it first runs through the >> > compiler, >> > which parses the BPEL file into a BPEL Object Model (BOM), which >> > basically >> > aims at abstracting from different BPEL versions (1.1, 2.0 draft, 2.0 >> > final). >> > >> > ODE Compiler compile and create OModel instances(such as >> > OProcess,OAssign,OInvoke etc) from .bpel, .wsdl and deploy files. Then >> > it >> > can use Java Object Serializable technology to serialize OModel objects >> > into >> > a .cbp file (Compiled Process Definitions part in the diagram, and we >> > can >> > see OBase implements interface java.io.Serializable, and all the OModel >> > classes are the sub-class of OBase). >> > >> > ODE runtime module can load a compiled BPEL model(the .cbp file in local >> > disk) by de-serialize the .cbp file back into OModel classes. >> > >> > When a process instance is executed, the navigator operates on the >> > OModel, >> > i.e. it is loaded from disk into memory during execution and is unloaded >> > afterwards, depending on hydration/dehydration policies to reduce the >> > memory >> > footprint. The execution state of the OModel keeps in-memory references >> > to >> > the OModel, but when the state is persisted, it does not store a copy of >> > the >> > OModel but only the IDs mentioned above. When the state is loaded, the >> > IDs >> > are looked up and replaced by proper references. >> > >> > Where Is The Problem >> > >> > Once you read OModel's serializer and de-serializer methods in class >> > org.apache.ode.bpel.o.Serializer: >> > >> > public void writeOProcess(OProcess process, OutputStream os) throws >> > IOException { >> > >> > DataOutputStream out = new DataOutputStream(os); >> > >> > out.write(MAGIC_NUMBER); >> > >> > out.writeShort(format); >> > >> > out.writeLong(compileTime); >> > >> > out.writeUTF(process.guid); >> > >> > out.writeUTF(process.targetNamespace); >> > >> > out.writeUTF(process.processName); >> > >> > out.flush(); >> > >> > ObjectOutputStream oos = new CustomObjectOutputStream(os); >> > >> > oos.writeObject(process); >> > >> > oos.flush(); >> > } >> > >> > public OProcess readOProcess() throws IOException, >> > ClassNotFoundException { >> > >> > >> > ObjectInputStream ois = new >> > CustomObjectInputStream(_inputStream); >> > >> > OProcess oprocess; >> > try { >> > >> > oprocess = (OProcess) ois.readObject(); >> > >> > } catch (ClassNotFoundException e) { >> > >> > throw new IOException("DataStream Error"); >> > >> > } >> > return oprocess; >> > >> > } >> > >> > You will find the problem with the current OModel is that it has a >> > static >> > structure and is serialized using Java's built-in mechanisms, which are >> > not >> > very tolerant against class changes and could even struggle with JDK >> > upgrades. >> > >> > For example, there are these attributes in current OProcess: >> > >> > public final String version; >> > >> > public OConstants constants; >> > >> > public String uuid; >> > >> > public String targetNamespace; >> > >> > public String processName; >> > >> > public OScope procesScope; >> > >> > After ODE Compiler serialize it into .cbp file, the .cbp file will >> > include >> > these 6 attributes for OProcess instance. One day, we add another >> > attribute >> > for OProcess class, for example: >> > >> > pubic String newAttribute; >> > >> > If we try to de-serialize the old .cbp file into the new OProcess class, >> > we >> > will get a error. >> > >> > This is a problem because BPEL processes and their instances are >> > potentially >> > long-running,they usually run over years. Thus, a serialized OModel >> > should >> > survive ODE and JDK upgrades. Our problem is that even simple changes >> > (that >> > are needed to implement new features or even to fix some bugs) break >> > backward compatibility in the OModel. We need a good solution to address >> > that problem. >> > >> > What Shall We Do To Solve This Problem >> > >> > This project want to solve this problem, we will change .cbp file's >> > format >> > and use a brand new serialize solution to keep compatibility. After >> > investigate possible approaches, we would like to use JSON to represent >> > OModel(It means new .cbp file will be JSON format). About the above >> > example, >> > the old JSON format .cbp file can be de-serialized into the new OProcess >> > class with a special migrate condition description string. >> > >> > Json Format >> > >> > Jackson is actually a JSON processor (JSON parser + JSON generator) >> > written >> > in Java. Beyond basic JSON reading/writing (parsing, generating), it >> > also >> > offers full node-based Tree Model, as well as full OJM (Object/Json >> > Mapper) >> > data binding functionality. More important, it has good performance, >> > even >> > faster than Java's built-in means. >> > >> > So we decided to use Json format file store OModel's serialize result, >> > and >> > Jackson 2.0 is the potential solution to reading/writing OModel instance >> > from/to Json format .cbp file. >> > >> > We can use Jackson to store attributes of Current OProcess class to Json >> > format file like this: >> > >> > Map<String,Object> userData1 = new HashMap<String,Object>(); >> > >> > userData1.put("owner", process.getOwner()); >> > >> > userData1.put("name", process.getName()); >> > >> > userData1.put("targetNameSpace", process.targetNamespace); >> > >> > userData1.put("processName", process.processName); >> > >> > userData1.put("children", process.getChildren()); >> > >> > >> > ObjectMapper mapper = new ObjectMapper(); >> > >> > StringWriter sw = new StringWriter(); >> > >> > org.codehaus.jackson.JsonGenerator gen; >> > >> > try { >> > gen = new JsonFactory().createJsonGenerator(sw); >> > >> > mapper.writeValue(gen, userData1); >> > >> > gen.close(); >> > String json = sw.toString(); >> > >> > System.out.println(json); >> > >> > } catch (IOException e) { >> > >> > e.printStackTrace(); >> > } >> > >> > The target json file will be: >> > >> > >> > {"targetNameSpace":"http://ode/bpel/unit-test","name":"HelloWorld2","processName":"HelloWorld2"} >> > >> > But not all the OModel instance can be serialized only using Jacksons >> > mixin >> > feature, we have to extend Jackson and would probably need to write >> > custom >> > (de)serializers for WSDL4J etc. >> > >> > OModel Migrate Mechanism >> > >> > Here we use a simple example proposed by my potential mentor Tammo van >> > Lessen to describe the migrate mechanism. >> > >> > Simple Example >> > >> > OProcess stores currently targetNamespace and processName as Strings. >> > Now >> > imagine we change OProcess so that it stores a QName instead. When >> > loading >> > an old OModel, the QName field would be null and there would be the two >> > String field in the "unknown fields" map. >> > >> > Migrate Solution >> > >> > Suppose the old version of OProcess is stored in following format: >> > >> > >> > {"targetNameSpace":"http://ode/bpel/unit-test","name":"HelloWorld2","processName":"HelloWorld2"} >> > >> > and the corresponding new version of OProcess's json .cbp file will be: >> > >> > >> > {"name":"HelloWorld2","namespace":"{http://ode/bpel/unit-test}HelloWorld2"} >> > >> > The migrate job is transform from one JSON to another JSON in a specific >> > rule, We should consider how to define the migrate description language >> > or >> > just re-use some existing JSON migrate DSL. >> > >> > This migrate DSL may be the most important part of this project. In this >> > simple demo, maybe looks like this: >> > >> > { >> > "from":{ >> > >> > "name":"$value0", >> > >> > "targetNameSpace":"$value1", >> > >> > "processName":"$value2" >> > >> > }, >> > "to":{ >> > >> > "name":"$value0", >> > >> > "namespace":"{$value1}$value2" >> > >> > } >> > } >> > >> > Use $value to represent migrate attribute values. >> > >> > Domain-specific language(DSL) >> > >> > Once we update OModel, and want to migrate the old version to new >> > version. >> > If we have both the old version Json format .cbp file and the specificed >> > migrate rule, we can get the new version Json format .cbp file, then >> > load as >> > new OProcess object, finish the migrate job easily. >> > >> > We are still survey for exist tools like XSLT, working on JSON Trees. If >> > can >> > not find such a tool, have both the old OModel and the new OModel, we >> > can >> > write a tool to generate migrate DSL string. >> > >> > Of course, we must specific the migrate rule for different version >> > change, >> > such as v1.0->v1.1, v1.1->v1.2 and v1.0->v1.2. >> > >> > >> > -- >> > Best Regards From Tiger Gui >> > -------------------------------------- >> > Open source is some kind of life attitude >> >> >> >> -- >> Tammo van Lessen - http://www.taval.de > > > > > -- > Best Regards From Tiger Gui > -------------------------------------- > Open source is some kind of life attitude -- Tammo van Lessen - http://www.taval.de
