[ https://issues.apache.org/jira/browse/PIG-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766985#action_12766985 ]
Thejas M Nair commented on PIG-965: ----------------------------------- I found another regex library that is supposed to be faster than java.util.regex . - dk.brics.automaton.RegExp (BSD license, used in apache nutch). It does not support all features of java regex, but it is a candidate that can be used for purposes of this patch (common simpler regexes). It is faster than java regex, but much slower than 'optimization2' (see numbers in code comments below) {code} String prefix = "123"; Pattern p = Pattern.compile("123.*"); RegExp r = new RegExp("123.*"); Automaton a = r.toAutomaton(); while((str = in.readLine()) != null ){ // optimization 1 - takes 30 secs // if((p.matcher(str).matches())) // matches++; //optimization 2 - takes 15 secs // int len = prefix.length(); // boolean matched = true; // for(int i=0; i<len; i++){ // if(prefix.charAt(i) != str.charAt(i)){ // matched = false; // break; // } // } // if(matched) // matches++; // dk.brics.automaton - takes 25 secs // if(a.run(str)) // matches++; tot++; } {code} > PERFORMANCE: optimize common case in matches (PORegex) > ------------------------------------------------------ > > Key: PIG-965 > URL: https://issues.apache.org/jira/browse/PIG-965 > Project: Pig > Issue Type: Improvement > Components: impl > Reporter: Thejas M Nair > > Some frequently seen use cases of 'matches' comparison operator have follow > properties - > 1. The rhs is a constant string . eg "c1 matches 'abc%' " > 2. Regexes such that look for matching prefix , suffix etc are very common. > eg - "abc%', "%abc", '%abc%' > To optimize for these common cases , PORegex.java can be changed to - > 1. Compile the pattern (rhs of matches) re-use it if the pattern string has > not changed. > 2. Use string comparisons for simple common regexes (in 2 above). > The implementation of Hive like clause uses similar optimizations. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.