[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705457#comment-14705457 ] Jeremiah Jordan commented on CASSANDRA-7410: ping > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.x > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt, CASSANDRA-7410-v3-2.1-branch.txt, > CASSANDRA-7410-v4-2.0-branch.txt, CASSANDRA-7410-v4-2.1-branch.txt, > CASSANDRA-7410-v5-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14712687#comment-14712687 ] Piotr Kołaczkowski commented on CASSANDRA-7410: --- +1 > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.x > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt, CASSANDRA-7410-v3-2.1-branch.txt, > CASSANDRA-7410-v4-2.0-branch.txt, CASSANDRA-7410-v4-2.1-branch.txt, > CASSANDRA-7410-v5-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745593#comment-14745593 ] Aleksey Yeschenko commented on CASSANDRA-7410: -- Committed to cassandra-2.1 as [c7b40735789c840529002eb3c11d8731f460d61c|https://github.com/apache/cassandra/commit/c7b40735789c840529002eb3c11d8731f460d61c] and merged upstream. 3.0 merge went without conflicts, but I did not verify if it works or not. If something is broken there, it's gonna be on Alex and Piotr. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.1.x, 2.2.x > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, CASSANDRA-7410-v1-2.2.txt, > CASSANDRA-7410-v2-2.1-branch.txt, CASSANDRA-7410-v3-2.1-branch.txt, > CASSANDRA-7410-v4-2.0-branch.txt, CASSANDRA-7410-v4-2.1-branch.txt, > CASSANDRA-7410-v5-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096201#comment-14096201 ] Brandon Williams commented on CASSANDRA-7410: - Can you rebase? Doesn't quite work on 2.0 with the latest CASSANDRA-6927 > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.10, 2.1.1 > > Attachments: 7410-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14036614#comment-14036614 ] Alex Liu commented on CASSANDRA-7410: - The patch is on top of CASSANDRA-6927. The following url parameters are added to CqlStorage {code} [&bulk_output_format=true|false] [&bulk_cf_schema=bulk_cf_schema] [&bulk_insert_statement=bulk_insert_statement] [&bulk_output_location=] [&bulk_buff_size=] [&bulk_stream_throttle=] [&bulk_max_failed_hosts=] {code} Note: legacy thrift based BulkOutputFormat is not supported. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.9 > > Attachments: 7410-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347505#comment-14347505 ] Alex Liu commented on CASSANDRA-7410: - {{CqlBulkRecordWriter}} comes with its own {{ExternalClient}} which uses a wrong approach for generating CFMetaData (and cfId). We can fix the issue by using the approach found in {{BulkLoader.ExternalClient}} which fetches CFMetadata from the cluster using CQL. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.13 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132792#comment-14132792 ] Brandon Williams commented on CASSANDRA-7410: - Ping [~alexliu68] > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.11 > > Attachments: 7410-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14135690#comment-14135690 ] Alex Liu commented on CASSANDRA-7410: - checking > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.11 > > Attachments: 7410-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14138154#comment-14138154 ] Brandon Williams commented on CASSANDRA-7410: - I'd prefer just rebasing the first much smaller patch, rather than do such a large refactoring in a minor release. We can do that in 2.1 if that's what we want. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.11 > > Attachments: 7410-2.0-branch.txt, 7410-v2-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284045#comment-14284045 ] Brandon Williams commented on CASSANDRA-7410: - Sorry, missed this one because it wasn't marked PA. Can you rebase and also push a 2.1 version? > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.13 > > Attachments: 7410-2.0-branch.txt, 7410-v2-2.0-branch.txt, > 7410-v3-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284918#comment-14284918 ] Alex Liu commented on CASSANDRA-7410: - I got following sstable loading error after bulk writing. {code} DEBUG [Thread-193] 2015-01-20 16:43:53,751 Renaming /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-TOC.txt to /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-TOC.txt DEBUG [Thread-193] 2015-01-20 16:43:53,751 Renaming /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-Statistics.db to /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-Statistics.db DEBUG [Thread-193] 2015-01-20 16:43:53,752 Renaming /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-Digest.sha1 to /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-Digest.sha1 DEBUG [Thread-193] 2015-01-20 16:43:53,752 Renaming /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-Filter.db to /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-Filter.db DEBUG [Thread-193] 2015-01-20 16:43:53,752 Renaming /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-Index.db to /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-Index.db DEBUG [Thread-193] 2015-01-20 16:43:53,752 Renaming /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-CompressionInfo.db to /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-CompressionInfo.db DEBUG [Thread-193] 2015-01-20 16:43:53,752 Renaming /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-tmp-ka-1-Data.db to /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1-Data.db DEBUG [Thrift:9] 2015-01-20 16:43:53,758 computing ranges for 8e56ee4c6ef2c35b3d97c6abeefe8b92 DEBUG [Thread-191] 2015-01-20 16:43:53,786 Load metadata for /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1 INFO [Thread-191] 2015-01-20 16:43:53,786 Opening /var/folders/92/cw97kmh10dxf6kj0b4ly1mbhgn/T/cql3ks/test_bulk-46af6a92-6c4c-44be-9b14-d817f7d63174/cql3ks-test_bulk-ka-1 (52 bytes) INFO [Thread-191] 2015-01-20 16:43:53,802 [Stream #931af290-a106-11e4-ae05-d11cec192498] Executing streaming plan for Bulk Load INFO [StreamConnectionEstablisher:1] 2015-01-20 16:43:53,802 [Stream #931af290-a106-11e4-ae05-d11cec192498] Starting streaming to /127.0.0.1 DEBUG [StreamConnectionEstablisher:1] 2015-01-20 16:43:53,802 [Stream #931af290-a106-11e4-ae05-d11cec192498] Sending stream init for incoming stream DEBUG [StreamConnectionEstablisher:1] 2015-01-20 16:43:53,805 [Stream #931af290-a106-11e4-ae05-d11cec192498] Sending stream init for outgoing stream DEBUG [ACCEPT-/127.0.0.1] 2015-01-20 16:43:53,805 Connection version 2 from /127.0.0.1 DEBUG [ACCEPT-/127.0.0.1] 2015-01-20 16:43:53,806 Connection version 2 from /127.0.0.1 INFO [STREAM-INIT-/127.0.0.1:52052] 2015-01-20 16:43:53,806 [Stream #931af290-a106-11e4-ae05-d11cec192498 ID#0] Creating new streaming plan for Bulk Load DEBUG [STREAM-OUT-/127.0.0.1] 2015-01-20 16:43:53,806 [Stream #931af290-a106-11e4-ae05-d11cec192498] Sending Prepare (0 requests, 1 files} INFO [StreamConnectionEstablisher:1] 2015-01-20 16:43:53,807 [Stream #931af290-a106-11e4-ae05-d11cec192498 ID#0] Prepare completed. Receiving 0 files(0 bytes), sending 1 files(52 bytes) INFO [STREAM-INIT-/127.0.0.1:52052] 2015-01-20 16:43:53,807 [Stream #931af290-a106-11e4-ae05-d11cec192498, ID#0] Received streaming plan for Bulk Load INFO [STREAM-INIT-/127.0.0.1:52053] 2015-01-20 16:43:53,808 [Stream #931af290-a106-11e4-ae05-d11cec192498, ID#0] Received streaming plan for Bulk Load DEBUG [STREAM-IN-/127.0.0.1] 2015-01-20 16:43:53,808 [Stream #931af290-a106-11e4-ae05-d11cec192498] Received Prepare (0 requests, 1 files} INFO [STREAM-IN-/127.0.0.1] 2015-01-20 16:43:53,808 [Stream #931af290-a106-11e4-ae05-d11cec192498 ID#0] Prepare completed. Receiving 1 files(52 bytes), sending 0 files(0 byt
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365884#comment-14365884 ] Alex Liu commented on CASSANDRA-7410: - Waiting for CASSANDRA-8358 > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365910#comment-14365910 ] Alex Liu commented on CASSANDRA-7410: - [~brandon.williams] Do you have time to review it? > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365925#comment-14365925 ] Brandon Williams commented on CASSANDRA-7410: - Seems like waiting on CASSANDRA-8358 is the right move, at least for trunk, so we don't compound a problem. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484096#comment-14484096 ] Aleksey Yeschenko commented on CASSANDRA-7410: -- CASSANDRA-8358 is taking a bit longer than I expected to review/commit. Could be delayed by a week or so more. Can you guys go ahead and review/commit this without 8358? I'll rebase CASSANDRA-8358 afterwards. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.15 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496547#comment-14496547 ] Piotr Kołaczkowski commented on CASSANDRA-7410: --- org/apache/cassandra/hadoop/pig/CqlNativeStorage.java:149 {noformat} if (t.getType(0) == DataType.TUPLE) { if (bulkOutputFormat) { cqlQueryFromTuple(null, t, 0); } else if (t.getType(1) == DataType.TUPLE) { Map key = tupleToKeyMap((Tuple)t.get(0)); cqlQueryFromTuple(key, t, 1); } else throw new IOException("Second argument in output must be a tuple"); } else throw new IOException("First argument in output must be a tuple"); {noformat} Personally, I don't like this input validation style. Much better to validate input in a flat way at the beginning: {noformat} if (t.getType(0) != DataType.TUPLE) throw if (t.getType(1) != DataType.TUPLE) throw // now we know input is ok, so we can focus on doing real stuff {noformat} Moreover, {{cqlQueryFromTuple}} does the same validation again... org.apache.cassandra.hadoop.pig.CqlNativeStorage#setStoreLocation: This method is a copy-paste from org.apache.cassandra.hadoop.pig.CqlStorage#setStoreLocation with only a minor section related to bulkOutputFormat added. Any reason for not using super.setStoreLocation()? org/apache/cassandra/io/sstable/CQLSSTableWriter.java: {noformat} try { Schema.instance.load(ksm); } catch (Exception e) { //It may get an exception of Attempting to load already loaded column family } {noformat} Ok, I get it, but what if it tries to load it for the first time and fails? It doesn't even inform the user that something bad happened and why. Also, can you elaborate more on why it may want to load it multiple times? > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.15 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498218#comment-14498218 ] Alex Liu commented on CASSANDRA-7410: - Any reason for not using super.setStoreLocation()? -- We will remove CqlStorage class soon, so try to not couple with it any more. > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.15 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498768#comment-14498768 ] Alex Liu commented on CASSANDRA-7410: - This change is no longer need, because I port the patch on 2.1 back to 2.0 which uses BulkLoader.ExternalClient . > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.15 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503144#comment-14503144 ] Piotr Kołaczkowski commented on CASSANDRA-7410: --- Patch doesn't apply to 2.0 branch: {noformat} pkolaczk@m4600 ~/Projekty/DataStax/cassandra $ git fetch pkolaczk@m4600 ~/Projekty/DataStax/cassandra $ git checkout cassandra-2.0 Already on 'cassandra-2.0' Your branch is up-to-date with 'origin/cassandra-2.0'. pkolaczk@m4600 ~/Projekty/DataStax/cassandra $ git apply 7410-v3-2.0-branch.txt 7410-v3-2.0-branch.txt:195: trailing whitespace. "[columns=][where_clause=]" + error: patch failed: src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java:345 error: src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java: patch does not apply {noformat} > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.15 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt, CASSANDRA-7410-v3-2.1-branch.txt, > CASSANDRA-7410-v4-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url
[ https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503183#comment-14503183 ] Piotr Kołaczkowski commented on CASSANDRA-7410: --- org/apache/cassandra/hadoop/pig/CqlNativeStorage.java:342 {noformat} private boolean serverEntryped() { if (!StringUtils.isEmpty(internodeEncrypt)) return InternodeEncryption.none != InternodeEncryption.valueOf(internodeEncrypt.toLowerCase()); return false; } {noformat} Typo: serverEntryped -> serverEncrypted. Also, the if can be slightly simplified: {noformat} private boolean serverEntryped() { return !StringUtils.isEmpty(internodeEncrypt) && InternodeEncryption.none != InternodeEncryption.valueOf(internodeEncrypt.toLowerCase()); } {noformat} Other than this, it looks good. +1 > Pig support for BulkOutputFormat as a parameter in url > -- > > Key: CASSANDRA-7410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7410 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.15 > > Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, > 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, > CASSANDRA-7410-v2-2.1-branch.txt, CASSANDRA-7410-v3-2.1-branch.txt, > CASSANDRA-7410-v4-2.0-branch.txt > > > Add BulkOutputFormat support in Pig url -- This message was sent by Atlassian JIRA (v6.3.4#6332)