Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread Josh Elser
On 4/13/14, 11:32 PM, Sean Busbey wrote: Oh! that's ACCUMULO-1994. The ticket explains the root problem. In the 1.5.0 Proxy, the timestamp defaults to 0. Since columns sort on timestamp in reverse order, that means nothing in the start row gets included (unless the rest of the key components are

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread David O'Gwynn
Lol. Search JIRA before banging head against wall. Noted. :-D On Sun, Apr 13, 2014 at 11:32 PM, Sean Busbey wrote: > Oh! that's ACCUMULO-1994. The ticket explains the root problem. In the 1.5.0 > Proxy, the timestamp defaults to 0. Since columns sort on timestamp in > reverse order, that means no

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread Sean Busbey
Oh! that's ACCUMULO-1994. The ticket explains the root problem. In the 1.5.0 Proxy, the timestamp defaults to 0. Since columns sort on timestamp in reverse order, that means nothing in the start row gets included (unless the rest of the key components are 0). [1]: https://issues.apache.org/jira/br

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread David O'Gwynn
Urgh. If I run your code with the 1.5.1 Thrift interface, behavior's not there. If I run it with the one I downloaded, I get the behavior. I diff'ed the ttypes.py from the old and new and got this: < (5, TType.I64, 'timestamp', None, None, ), # 5 --- > (5, TType.I64, 'timestamp', None, 922

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread Josh Elser
On 4/13/14, 9:13 PM, David O'Gwynn wrote: If the versioning iterator is attached, and the WRI's priority is <= the versioning iterator's priority, then you see this behavior (the first row of a WRI scan gets dropped). If you change the priority for the WRI in your code to <=20, then you'll see it

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread David O'Gwynn
Ok, so I went back to my IPython console to rerun my scan to prove to myself that I wasn't crazy. Well, I ran it and it worked like you just said, contra to my original point. Started to think I was on the crazy train. Then I remembered that the table I'd been working on, I'd removed the versionin

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread Josh Elser
David, Not quite sure what you're seeing. Using the "plain" python bindings, I think I emulated what you described. I created a table with the following data: 1 => ['col1: [] 1397241795 => val1', 'col2: [] 1397241797 => val2', 'col3: [] 1397241800 => val3'] 2 => ['col1: [] 1397241803 => val1

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread David O'Gwynn
Hi Russ, I ported it: def decode_row(cell): value = StringIO.StringIO(cell.value) numCells = struct.unpack('!i',value.read(4))[0] key = cell.row for i in range(numCells): if value.pos == value.len: raise Exception( 'Reached the end of the parsab

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread Russ Weeks
Just curious, David, did you port the logic of WholeRowIterator.decodeRow over to Python, or is that functionality available somewhere in the pyaccumulo API and I just missed it? -Russ On Sun, Apr 13, 2014 at 10:48 AM, David O'Gwynn wrote: > 1.5.0 > > Btw, the pyaccumulo library: > > https://g

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread David O'Gwynn
1.5.0 Btw, the pyaccumulo library: https://github.com/accumulo/pyaccumulo is the basis of my codebase. You should be able to use that to replicate the issue. Thanks for looking into this! On Sun, Apr 13, 2014 at 12:51 PM, Josh Elser wrote: > Ah, gotcha. > > That definitely does not seem right

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-13 Thread Josh Elser
Ah, gotcha. That definitely does not seem right. I'll see if I can poke around at this today. Are you using 1.5.0 or 1.5.1? (1.5.1 was just released a few weeks ago) On 4/12/14, 4:13 PM, David O'Gwynn wrote: Hi Josh, I guess I misspoke, the Range I'm passing is this: Range('row0', true, 'r

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-12 Thread David O'Gwynn
Hi Josh, I guess I misspoke, the Range I'm passing is this: Range('row0', true, 'row0\0',true) Keeping in mind that the Thrift interface only exposes one Range constructor (Range(Key,bool,Key,bool)), the actual call I'm passing is this: Range( Key('row0',null,...), true, Key('row0\0',null,...),

Re: Thrift proxy: Python WholeRowIterator behavior

2014-04-12 Thread Josh Elser
Hi David, Looks like you're just mis-using the Range here. If you create a range that is ["row0", "row0"] as you denote below, that will only include Keys that have a rowId of "row0" with an empty colfam, colqual, etc. Since you want to use the WholeRowIterator, I can assume you want all colu

Thrift proxy: Python WholeRowIterator behavior

2014-04-12 Thread David O'Gwynn
Hi all, I'm working with the Python Thrift API for the Accumulo proxy service, and I have a bit of odd behavior happening. I'm using Accumulo 1.5 (the standard one from the Accumulo website). Whenever I use the WholeRowIterator with a Scanner, I cannot configure the Range for that Scanner to corr