Re: Major compactions during map reduce

2022-04-19 Thread Vincent Russell
Thank you Christopher. No. We aren't seeing any errors, the results of the map reduce job just seem to be less than what we expected so it seemed like possible records were being skipped by the mapper even though we aren't passing in any ranges at all. On Tue, Apr 19, 2022 at 5:33 AM Christopher

Re: Major compactions during map reduce

2022-04-19 Thread Dave Marion
I was initially thinking about the case where the splits change between the job setup and the Map execution, but given more thought I think I went down the wrong path. Tablet splitting should not affect the overall range of keys for the MR job. If a Tablet splits after the job computes the splits,

Re: Major compactions during map reduce

2022-04-19 Thread Christopher
Isolation should only give you consistency within a row, to ensure you're not scanning over partial changes from a mutation that is currently being written to a row. It shouldn't have anything to do with compactions or missing data that has already been written before the MapReduce scan has started