Marko, no problem. If you have the time, then defining your own
protocol/schema is the right approach.
I did apply my changes from 1.0 to 1.2 on my local git repo. Here is a
simple use-case I tested with:
public class HelloWorld {
public String m_strName="myname";
public String m_strLocation="mylocation";
public List<String> m_listString = new ArrayList<String>();
public Date m_Date = new Date();
}
The above gets converted to the following schema using ReflectData:
{ "fields" : [ { "name" : "m_strName",
"type" : "string"
},
{ "name" : "m_strLocation",
"type" : "string"
},
{ "name" : "m_listString",
"type" : { "items" : "string",
"type" : "array"
}
},
{ "name" : "m_Date",
"type" : { "fields" : [ { "name" : "data",
"type" : "long"
} ],
"name" : "Date",
"namespace" : "java.util",
"type" : "record"
}
}
],
"name" : "HelloWorld",
"namespace" : "com.company.project",
"type" : "record"
}
Good luck with your Avro adventures :)
-Hiral
On Tue, Dec 1, 2009 at 12:54 PM, Marko Milicevic <[email protected]>wrote:
> Thanks for the offer Hiral, but without doc i'm not sure what i need.
>
> My original usecase was to serialize arbitrary Java objects over the wire.
> I was looking for something more efficient than Java serialization.
>
> I read of several benchmarks that showed favorable performance for Avro, so
> here i am.
>
> But it seems that there are significant constraints on the type of Java
> object that can be serialized. If that is the case then i tend to think
> that Doug's approach would make the most sense in the long run (ie. first
> define a Avro schema/protocol, then generate the Java).
>
> Thanks again.
>
> Marko.
> .
>
> ------------------------------
> *From:* tazan007 [mailto:[email protected]]
> *Sent:* Tuesday, December 01, 2009 2:57 PM
>
> *To:* [email protected]
> *Subject:* Re: reflected char array returning null package.
>
> Marko, I did make some changes to avro 1.0 back when I was evaluating it
> for RPC and added support for it to convert String to Utf8 and Date to long
> for converting Java to Avro but I never had to go from Avro back to Java
> once I had the data in Avro.
>
> If you want, I could try to apply the changes I made to 1.2 if it fits your
> use-case.
>
> -Hiral
>
> On Tue, Dec 1, 2009 at 11:42 AM, Marko Milicevic <[email protected]>wrote:
>
>> Thanks Doug.
>>
>> Is there any documentation on the Specific API and the Specific schema
>> format?
>>
>> Marko.
>> .
>>
>> -----Original Message-----
>> From: Doug Cutting [mailto:[email protected]]
>> Sent: Tuesday, December 01, 2009 1:22 PM
>> To: [email protected]
>> Subject: Re: reflected char array returning null package.
>>
>> In Avro 1.2's reflect implementation, Strings must be represented with
>> Avro's Utf8 class. Folks reasonably complained that this is not very
>> useful, and it's being fixed for Avro 1.3.
>>
>> https://issues.apache.org/jira/browse/AVRO-80
>>
>> I'm currently investing a lot of effort improving reflect, and reflect
>> should work considerably better in 1.3. That said, unless you have an
>> existing RPC codebase that you need to move onto Avro with minimal
>> disruption, the specific API is much easier to use than the reflect API.
>>
>> Avro schemas are mostly an expressive subset of Java, so mapping Avro
>> to Java is much simpler than mapping Java to Avro.
>>
>> Doug
>>
>> Marko Milicevic wrote:
>> > Hello. I'm running the following code under the avro 1.2.0 release
>> > and jdk 1.6.0_10.
>> >
>> > public interface ISink
>> > {
>> > public void send( String msg );
>> > }
>> >
>> > public class Sink implements ISink
>> > {
>> > public void send( String msg )
>> > {
>> > }
>> > }
>> >
>> >
>> > public void test()
>> > {
>> > SocketServer server =
>> > new SocketServer(new SpecificResponder(
>> > ISink.class, new Sink()), new InetSocketAddress(3333));
>> >
>> > SocketTransceiver client =
>> > new SocketTransceiver(new InetSocketAddress(server.getPort()));
>> >
>> > ISink proxy = (ISink)SpecificRequestor.getClient(ISink.class,
>> > client);
>> >
>> > for (int i = 0; i < 1000; i++ )
>> > {
>> > proxy.send("testing");
>> > }
>> >
>> > client.close();
>> > server.close();
>> > }
>> >
>> >
>> > First I was getting errors until I learned of paranamer.
>> > I thought I read in the doc, that no pre-processing would be
>> necessary?
>> >
>> > I'm fine with having to run paranamer, but want to be sure that it is
>> > required?
>> >
>> >
>> > Now after processing with paranamer I get the following error trying
>> > to run...
>> >
>> > Exception in thread "main" java.lang.NullPointerException
>> > at
>> > org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:259)
>> > at
>> > org.apache.avro.reflect.ReflectData.createFieldSchema(ReflectData.java
>> > :3
>> > 03)
>> > at
>> > org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:291)
>> > at
>> > org.apache.avro.reflect.ReflectData.createFieldSchema(ReflectData.java
>> > :3
>> > 03)
>> > at
>> > org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:291)
>> > at
>> > org.apache.avro.reflect.ReflectData.getMessage(ReflectData.java:339)
>> > at
>> > org.apache.avro.reflect.ReflectData.getProtocol(ReflectData.java:318)
>> > at
>> > org.apache.avro.reflect.ReflectResponder.<init>(ReflectResponder.java:
>> > 45
>> > )
>> > at
>> > org.apache.avro.reflect.ReflectResponder.<init>(ReflectResponder.java:
>> > 41
>> > )
>> >
>> > //
>> > org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:259)
>> > } else if (type instanceof Class) {
>> > Class c = (Class)type;
>> > String name = c.getSimpleName();
>> >>> String space = c.getPackage().getName();
>> > if (c.getEnclosingClass() != null) // nested class
>> > space = c.getEnclosingClass().getName() + "$";
>> > String fullName = c.getName();
>> > Schema schema = names.get(fullName);
>> > if (schema == null) {
>> >
>> > Stepping though the debugger, the type appears to be "class [C", with
>> > the name "value". I assume that is the "value" char[] in the String
>> > msg.
>> >
>> > Should a Class of type "class [C", return a package of null?
>> >
>> > If so, any idea what I'm doing incorrectly?
>> >
>> > Thank you.
>> >
>> > Marko.
>> > .
>>
>
>
