Hello,
I'd like to share my code for TeraSort on Flink and Spark which uses
the same range partitioner as Hadoop TeraSort:
https://github.com/eastcirclek/terasort
I also write a short report on it:
http://eastcirclek.blogspot.kr/2015/06/terasort-for-spark-and-flink-with-range.html
In the
verting Text to OptimizedText (Java code)
map(new MapFunction, Tuple2>() {
@Override
public Tuple2 map(Tuple2 value) {
return new Tuple2(new OptimizedText(value.f0),
value.f1);
}
})
On Thu, Jul 2, 2015 at 6:47 PM, Dongwon Kim
wrote:
> Hello,
>
> I'd like to share my code fo
gt; public int compareTo(OptimizedText o) {
> return this.text.compareTo(o.text);
> }
> }
>
> -
> Converting Text to OptimizedText (Java code)
>
> map(new MapFunction, Tuple2>() {
> @Override
> public Tuple2
t; public void write(DataOutputView out) throws IOException {
>> text.write(out);
>> }
>>
>> @Override
>> public void read(DataInputView in) throws IOException {
>> text.readFields(in);
>> }
>>
>> @Override
>> public int compareTo(Opti
ew out) throws IOException {
> > text.write(out);
> > }
> >
> > @Override
> > public void read(DataInputView in) throws IOException {
> > text.readFields(in);
> > }
> >
> > @Override
> > public int compareTo(OptimizedText o) {
> > return
ew out) throws IOException {
> > text.write(out);
> > }
> >
> > @Override
> > public void read(DataInputView in) throws IOException {
> > text.readFields(in);
> > }
> >
> > @Override
> > public int compareTo(OptimizedText o) {
>
>> > this.text = from;
>> > }
>> >
>> > public Text getText() {
>> > return text;
>> > }
>> >
>> > @Override
>> > public int getMaxNormalizedKeyLen() {
>> > return 10;
>> > }
>> >
&g
t; Stephan
>>> >
>>> > -
>>> > Code for optimized sortable (Java):
>>> >
>>> > public final class OptimizedText implements
>>> NormalizableKey
>>> > {
>>>
