I forgot to mention that I set ozone.datanode.pipeline.limit to 5. Michel Le mar. 28 juil. 2020 à 20:22, Michel Sumbul <[email protected]> a écrit :
> Hi guys, > > I would like to know if you have any advice tips/tricks to get the best > performance of Ozone? (Memory tuning / thread / specific settings / etc..) > > I did a few teragen/terasort on it and the results are really surprising > compared to HDFS,Ozone (using the hadoopFS) is almost 2 times slower than > HDFS. > > [image: image.png] > > [image: image.png] > > > The clusters were exactly the same for both: > - 3 masters and 4 slaves (8core/32GB) (its a small cluster but that should > matter here) > - Backend storage is a CEPH storage (80 servers) > - NIC: 2 x 25Gb/S > - Ozone version 0.5 > - Each job was executed 5 times > > HDFS and Ozone were installed on the same nodes, one was down where the > other one was up, to guarantee no other differences of configuration that > the distributed FS. > > I was not expecting a big difference like this, do you have any idea how > to improve this? > Or what can be the reason for that? I saw a few jira regarding data > locality at read, can it be linked to that? > > Thanks, > Michel > >
