Re: [PATCHv1 0/3] git-p4: fixing --changes-block-size support
On 07/06/15 17:01, Lex Spoon wrote: Great work. Thanks! I actually found the problem in my day job, so it was very handy having all the infrastructure already in place! For curiosity's sake, the -m solution has been observed to work on at least one Perforce installation. However clearly it doesn't work on others, so the batch ranges approach looks like it will be better. Yes, I can easily imagine that it's changed from one version to the next. I tried going back to a 2014.2 server which still had the same problem (with maxresults), but my investigations were not very exhaustive! Based on what has been seen so far, the Perforce maxscanrows setting must be applying the low-level database queries that Perforce uses internally in its implementation. That makes the precise effect on external queries rather hard to predict. It likely also depends on the version of Perforce. Indeed. All sorts of things can cause it to fail; I've seen it reject p4 files and p4 print, albeit with artificially low maxscanrows and maxresults values. I think this means there's no way to ever make it reliably work for all possible sizes of depot and values of maxscanrows/maxresults. Luke -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCHv1 0/3] git-p4: fixing --changes-block-size support
Great work. For curiosity's sake, the -m solution has been observed to work on at least one Perforce installation. However clearly it doesn't work on others, so the batch ranges approach looks like it will be better. Based on what has been seen so far, the Perforce maxscanrows setting must be applying the low-level database queries that Perforce uses internally in its implementation. That makes the precise effect on external queries rather hard to predict. It likely also depends on the version of Perforce. Lex Spoon -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCHv1 0/3] git-p4: fixing --changes-block-size support
We recently added support to git-p4 to limit the number of changes it would try to import at a time. That was to help clients who were being limited by the maxscanrows limit. This used the -m maxchanges argument to p4 changes to limit the number of results returned to git-p4. Unfortunately it turns out that in practice, the server limits the number of results returned *before* the -m maxchanges argument is considered. Even supplying a -m 1 argument doesn't help. This affects both the maxscanrows and maxresults group options. This set of patches updates the t9818 git-p4 tests to show the problem, and then adds a fix which works by iterating over the changes in batches (as at present) but using a revision range to limit the number of changes, rather than -m $BATCHSIZE. That means it will in most cases require more transactions with the server, but usually the effect will be small. Along the way I also found that p4 print can fail if you have a file with too many changes in it, but there's unfortunately no way to workaround this. It's fairly unlikely to ever happen in practice. I think I've covered everything in this fix, but it's possible that there are still bugs to be uncovered; I find the way that these limits interact somewhat tricky to understand. Thanks, Luke Luke Diamand (3): git-p4: additional testing of --changes-block-size git-p4: test with limited p4 server results git-p4: fixing --changes-block-size handling git-p4.py | 48 +++- t/t9818-git-p4-block.sh | 73 +++-- 2 files changed, 99 insertions(+), 22 deletions(-) -- 2.3.4.48.g223ab37 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
