Hi Colin,
I am glad to provide that.
I have attached the Serializer (don't laugh too hard if it is rubish)
here is the entire stack trace:
Feb 25, 2015 3:35:08 PM com.orientechnologies.common.log.OLogManager log
INFO: OrientDB auto-config DISKCACHE=10,695MB (heap=3,641MB os=16,384MB
disk=120,600MB)
com.orientechnologies.orient.core.exception.OConfigurationException:
Runtime index definition cannot find binary serializer with id=100. Assure
to plug custom serializer into the server.
at
com.orientechnologies.orient.core.index.ORuntimeKeyIndexDefinition.<init>(ORuntimeKeyIndexDefinition.java:48)
at
com.orientechnologies.orient.core.sql.OCommandExecutorSQLCreateIndex.execute(OCommandExecutorSQLCreateIndex.java:255)
at
com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:64)
at
com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:1184)
at
com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:1173)
at
com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:63)
at
com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.command(ONetworkProtocolBinary.java:1178)
at
com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:385)
at
com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:220)
at
com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:69)
On Thursday, February 26, 2015 at 9:42:46 AM UTC-6, Colin wrote:
>
> Hi Russ.
>
> Would you be able to send me a complete stack trace of the exception and
> the source to your CFISerializer code? (You can send it privately or post
> it here, whichever you prefer.)
>
> Thanks for your patience.
>
> -Colin
>
> Colin Leister
>
> *Orient Technologies*
>
> *The Company behind OrientDB*
>
>
>
>
>
> On Thursday, February 26, 2015 at 8:46:39 AM UTC-6, Russ White wrote:
>>
>> Sorry if this is a noob question.
>>
>> I am trying to use a custom index key type this against a remote server.
>> How do I register the serializer?
>>
>> I have put the jar containing the serializer in the remote server's lib
>> directory - but when I try this:
>>
>> OBinarySerializerFactory.getInstance().registerSerializer(new
>> CFISerializer(), null);
>> val index = db.getMetadata.getIndexManager.createIndex("cfi","UNIQUE",new
>> ORuntimeKeyIndexDefinition(CFISerializer.ID),null,null,null)
>>
>> Enter code here...
>>
>> I get the following exception:
>>
>> com.orientechnologies.orient.core.exception.OConfigurationException:
>> Runtime index definition cannot find binary serializer with id=100. Assure
>> to plug custom serializer into the server.
>> at
>> com.orientechnologies.orient.core.index.ORuntimeKeyIndexDefinition.<init>(ORuntimeKeyIndexDefinition.java:48)
>> at
>> com.orientechnologies.orient.core.sql.OCommandExecutorSQLCreateIndex.execute(OCommandExecutorSQLCreateIndex.java:255)
>>
>> The problem is I can't seem to find any mention on "how" to plug in the
>> custom serializer...
>>
>> Any help appreciated
>>
>> Also - as long as my serialized key implements Comparable can I use it in
>> a ranged query?
>>
>> What about using custom keys in Vertex key indexes? Will the key index
>> use the custom serializer? Can I make SQL queries use the serialized
>> objects compare method for ordering and range queries?
>>
>> Thanks!
>> Russ
>>
>> On Thursday, April 26, 2012 at 12:23:24 PM UTC-5, Lvc@ wrote:
>>>
>>> Hi all,
>>> I've just committed a quite internal feature someone of you could use:
>>> the support for custom serializer for index keys.
>>>
>>> In order to let to the developer to optimize at the best the usage of
>>> memory, disk space and cpu, it's now possible to have custom binary
>>> serializer.
>>>
>>> Example: index documents per SHA-256 key. Before now you had to use a
>>> String to represent it. A SHA-256 string occupies at least 64bytes in
>>> memory. Using your own byte[] you can save half space because SHA-256 is
>>> 256bits long = 32 bytes.
>>>
>>> Unfortunately you can't use a raw byte[] because Index needs object that
>>> implements the Java Comparable interface. For this purpose let's create own
>>> binary type:
>>>
>>> public static class ComparableBinary implements
>>> Comparable<ComparableBinary> {
>>>
>>>
>>> private byte[] value;
>>>
>>>
>>> public ComparableBinary(byte[] buffer) {
>>>
>>>
>>> value = buffer;
>>> }
>>>
>>>
>>> public int compareTo(ComparableBinary o) {
>>>
>>>
>>> final int size = value.length;
>>>
>>>
>>> for (int i = 0; i < size; ++i) {
>>>
>>>
>>> if (value[i] > o.value[i])
>>>
>>>
>>> return 1;
>>>
>>>
>>> else if (value[i] < o.value[i])
>>>
>>>
>>> return -1;
>>>
>>>
>>> }
>>> return 0;
>>>
>>>
>>> }
>>>
>>> public byte[] toByteArray() {
>>>
>>>
>>> return value;
>>> }
>>>
>>> }
>>>
>>> Now create the serializer for this type using 32bytes long:
>>>
>>> public static class OHash256Serializer implements
>>> OBinarySerializer<ComparableBinary> {
>>>
>>>
>>> public static final OBinaryTypeSerializer INSTANCE = new
>>> OBinaryTypeSerializer();
>>>
>>>
>>> public static final byte ID = 100;
>>>
>>>
>>> public static final int LENGTH = 32;
>>>
>>>
>>> public int getObjectSize(final int length) {
>>>
>>>
>>> return length;
>>> }
>>>
>>>
>>> public int getObjectSize(final ComparableBinary object) {
>>>
>>>
>>> return object.toByteArray().length;
>>>
>>>
>>> }
>>>
>>> public void serialize(final ComparableBinary object, final byte[] stream,
>>> final int startPosition) {
>>>
>>>
>>> final byte[] buffer = object.toByteArray();
>>>
>>>
>>> System.arraycopy(buffer, 0, stream, startPosition, buffer.length);
>>>
>>>
>>> }
>>>
>>> public ComparableBinary deserialize(final byte[] stream, final int
>>> startPosition) {
>>>
>>>
>>> final byte[] buffer = Arrays.copyOfRange(stream, startPosition,
>>> startPosition + LENGTH);
>>>
>>>
>>> return new ComparableBinary(buffer);
>>>
>>>
>>> }
>>>
>>> public int getObjectSize(byte[] stream, int startPosition) {
>>>
>>>
>>> return LENGTH;
>>> }
>>>
>>>
>>> public byte getId() {
>>>
>>>
>>> return ID;
>>> }
>>>
>>> }
>>>
>>> Now to use it register to the OrientDB kernel and create an index with
>>> it:
>>>
>>> OBinarySerializerFactory.INSTANCE.registerSerializer(new
>>> OHash256Serializer(), null);
>>>
>>>
>>> index = database.getMetadata().getIndexManager().createIndex("custom-hash",
>>> "UNIQUE", new ORuntimeKeyIndexDefinition(OHash256Serializer.ID), null,
>>> null);
>>>
>>> How to use it? Like a normal value, OrientDB will do all the rest:
>>>
>>> ComparableBinary key1 = new ComparableBinary(new byte[] { 0, 1, 2, 3, 4, 5,
>>> 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0,
>>> 1 });
>>>
>>> ODocument doc1 = new ODocument().field("k", "key1");
>>>
>>>
>>> index.put(key1, doc1);
>>>
>>>
>>> For more information:
>>> http://code.google.com/p/orient/wiki/Indexes#Custom_keys
>>>
>>> Lvc@
>>>
>>
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
package com.lifeway.xcite;
import com.lifeway.epublican.epub.cfi.CFI;
import com.lifeway.epublican.epub.cfi.CFIParser;
import com.lifeway.epublican.epub.epub3.EpublicanException;
import com.orientechnologies.common.directmemory.ODirectMemoryPointer;
import com.orientechnologies.common.serialization.types.OBinarySerializer;
import com.orientechnologies.common.serialization.types.OBinaryTypeSerializer;
import java.io.Serializable;
import java.util.Arrays;
/**
* Created by Russ White on 2/25/15.
*/
public class CFISerializer implements OBinarySerializer<CFI>{
private static int INTEGER_SIZE = 4;
public static byte ID = 100;
public static final OBinaryTypeSerializer INSTANCE = new OBinaryTypeSerializer();
@Override
public int getObjectSize(final CFI object, final Object... hints) {
return object.toString().getBytes().length + INTEGER_SIZE;
}
@Override
public int getObjectSize(final byte[] stream, final int startPosition) {
return byteArrayToInt(Arrays.copyOfRange(stream, startPosition, startPosition + INTEGER_SIZE)) + INTEGER_SIZE;
}
@Override
public void serialize(final CFI object, final byte[] stream, final int startPosition, final Object... hints) {
final byte[] payload = object.toString().getBytes();
final byte[] size = intToByteArray(getObjectSize(object));
final byte[] buffer = new byte[payload.length + INTEGER_SIZE];
System.arraycopy(size, 0, buffer, 0, size.length);
System.arraycopy(payload, 0, buffer, size.length, payload.length);
System.arraycopy(buffer, 0, stream, startPosition, buffer.length);
}
@Override
public CFI deserialize(final byte[] stream, final int startPosition) {
int size = getObjectSize(stream,startPosition) - INTEGER_SIZE;
byte[] buffer = Arrays.copyOfRange(stream, startPosition + INTEGER_SIZE, startPosition + size);
CFI result = null;
try {
result = CFIParser.parse(new String(buffer));
} catch (EpublicanException ex) {
ex.printStackTrace();
}
System.out.println(result);
return result;
}
@Override
public byte getId() {
return ID;
}
@Override
public boolean isFixedLength() {
return false;
}
@Override
public int getFixedLength() {
return 0;
}
@Override
public void serializeNativeObject(final CFI object, final byte[] stream, final int startPosition, final Object... hints) {
}
@Override
public CFI deserializeNativeObject(final byte[] stream, final int startPosition) {
return null;
}
@Override
public int getObjectSizeNative(final byte[] stream, final int startPosition) {
return 0;
}
@Override
public void serializeInDirectMemoryObject(final CFI object, final ODirectMemoryPointer pointer, final long offset, final Object... hints) {
}
@Override
public CFI deserializeFromDirectMemoryObject(final ODirectMemoryPointer pointer, final long offset) {
return null;
}
@Override
public int getObjectSizeInDirectMemory(final ODirectMemoryPointer pointer, final long offset) {
return 0;
}
@Override
public CFI preprocess(final CFI value, final Object... hints) {
return null;
}
public static int byteArrayToInt(byte[] encodedValue) {
int index = 0;
int value = encodedValue[index++] << Byte.SIZE * 3;
value ^= (encodedValue[index++] & 0xFF) << Byte.SIZE * 2;
value ^= (encodedValue[index++] & 0xFF) << Byte.SIZE * 1;
value ^= (encodedValue[index++] & 0xFF);
return value;
}
public static byte[] intToByteArray(int value) {
int index = 0;
byte[] encodedValue = new byte[Integer.SIZE / Byte.SIZE];
encodedValue[index++] = (byte) (value >> Byte.SIZE * 3);
encodedValue[index++] = (byte) (value >> Byte.SIZE * 2);
encodedValue[index++] = (byte) (value >> Byte.SIZE);
encodedValue[index++] = (byte) value;
return encodedValue;
}
}
