Thank you. Are there tools for parsing the Hive logs for errors? If not,
can you talk about the strategy used at Facebook to deal with detection and
resolution of MR errors?
Perhaps I can write a script to identify errors. First I have to solve the
mystery of why there are no logs on my hadoop
Hive queries are compiled to different types tasks (MapReduce, MoveTask, etc),
so a successful MR task as indicated in the JT doesn't mean the whole query
succeeded. So you need to examine the status of the hive query to see if it
succeeded or not. You can also check the hive's log file under
/
I've been using Hive in production for two months now. We're mainly using
it for processing server logs, about 1-2GB per day (2-2.5 million
requests). Typically we import a day's worth of logs at once. That said,
sometimes we decide to tweak a calculated column. When that happens, we
modify our
