Thank you Todd. Thanks, Amit
On Thu, Apr 28, 2016 at 9:25 PM, Todd Lipcon <t...@cloudera.com> wrote: > Hi Amit, > > What you're probably seeing is container pre-allocation. We use > 'fallocate' to preallocate space in the block container files, to avoid > fragmentation on disk. So, the disk usage will increase in multiples of > 32MB within each data container file. You can see the number of active > containers by looking at the 'log_block_manager_containers' metric and > subtract 'log_block_manager_full_containers' > > -Todd > > On Thu, Apr 28, 2016 at 6:03 AM, Amit Adhau <amit.ad...@globant.com> > wrote: > >> Hi Kudu team, >> >> I have below observations for kudu data storage size mismatch; >> >> I was having similar observation last week when total on-disk size was >> between 4-5GB and data folder was showing 180GB. Hence, we have cleaned all >> kudu data and created new master and tablet data directories. That means >> kudu was having zero table. After that, we created a new table in kudu and >> inserted just single record as 1461842027,'Test Event'. >> >> 1] Now, On Kudu Dashboard - TOTAL On-Disk Size in Kudu is 250B, for a >> single table having single record[1461842027,'Test Event'] >> >> 2] >> >> Result for bytes_under is as per below; >> >> For Link:- http://kuduserver:8051/metrics?metrics=bytes_under >> >> "type": "server", >> "id": "kudu.master", >> "attributes": {}, >> "metrics": [ >> { >> "name": "log_block_manager_bytes_under_management", >> "value": 48549 >> } >> ] >> >> For Link:- http://kuduserver:8050/metrics?metrics=bytes_under >> >> "type": "server", >> "id": "kudu.tabletserver", >> "attributes": {}, >> "metrics": [ >> { >> "name": "log_block_manager_bytes_under_management", >> "value": 4657 >> } >> ] >> >> 3] And data folder size on both tablet server is 161MB. >> >> as confirmed earlier, Total on-disk size[1] should be equal to Data >> folder size[3] and probably [2] should be also in sync. >> >> Can you please suggest, if this is normal or if it is an issue. This >> would be important factor while planning for capacity. >> >> FYI, On our Kudu cluster, we have 2 tablet server and a master[running as >> tablet as well] >> >> >> -- >> Thanks & Regards, >> >> *Amit Adhau* | Data Architect >> >> *GLOBANT* | IND:+91 9821518132 >> >> [image: Facebook] <https://www.facebook.com/Globant> >> >> [image: Twitter] <http://www.twitter.com/globant> >> >> [image: Youtube] <http://www.youtube.com/Globant> >> >> [image: Linkedin] <http://www.linkedin.com/company/globant> >> >> [image: Pinterest] <http://pinterest.com/globant/> >> >> [image: Globant] <http://www.globant.com/> >> >> The information contained in this e-mail may be confidential. It has been >> sent for the sole use of the intended recipient(s). If the reader of this >> message is not an intended recipient, you are hereby notified that any >> unauthorized review, use, disclosure, dissemination, distribution or >> copying of this communication, or any of its contents, >> is strictly prohibited. If you have received it by mistake please let us >> know by e-mail immediately and delete it from your system. Many thanks. >> >> >> >> La información contenida en este mensaje puede ser confidencial. Ha sido >> enviada para el uso exclusivo del destinatario(s) previsto. Si el lector de >> este mensaje no fuera el destinatario previsto, por el presente queda Ud. >> notificado que cualquier lectura, uso, publicación, diseminación, >> distribución o copiado de esta comunicación o su contenido está >> estrictamente prohibido. En caso de que Ud. hubiera recibido este mensaje >> por error le agradeceremos notificarnos por e-mail inmediatamente y >> eliminarlo de su sistema. Muchas gracias. >> >> > > > -- > Todd Lipcon > Software Engineer, Cloudera > -- Thanks & Regards, *Amit Adhau* | Data Architect *GLOBANT* | IND:+91 9821518132 [image: Facebook] <https://www.facebook.com/Globant> [image: Twitter] <http://www.twitter.com/globant> [image: Youtube] <http://www.youtube.com/Globant> [image: Linkedin] <http://www.linkedin.com/company/globant> [image: Pinterest] <http://pinterest.com/globant/> [image: Globant] <http://www.globant.com/> -- The information contained in this e-mail may be confidential. It has been sent for the sole use of the intended recipient(s). If the reader of this message is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution or copying of this communication, or any of its contents, is strictly prohibited. If you have received it by mistake please let us know by e-mail immediately and delete it from your system. Many thanks. La información contenida en este mensaje puede ser confidencial. Ha sido enviada para el uso exclusivo del destinatario(s) previsto. Si el lector de este mensaje no fuera el destinatario previsto, por el presente queda Ud. notificado que cualquier lectura, uso, publicación, diseminación, distribución o copiado de esta comunicación o su contenido está estrictamente prohibido. En caso de que Ud. hubiera recibido este mensaje por error le agradeceremos notificarnos por e-mail inmediatamente y eliminarlo de su sistema. Muchas gracias.
